Kohya S
c7c666b182
fix typo
2024-09-11 22:12:31 +09:00
Kohya S
6dbfd47a59
Fix to work PIECEWISE_CONSTANT, update requirement.txt and README #1393
2024-09-11 21:44:36 +09:00
青龍聖者@bdsqlsz
fd68703f37
Add New lr scheduler ( #1393 )
...
* add new lr scheduler
* fix bugs and use num_cycles / 2
* Update requirements.txt
* add num_cycles for min lr
* keep PIECEWISE_CONSTANT
* allow use float with warmup or decay ratio.
* Update train_util.py
2024-09-11 21:25:45 +09:00
Kohya S
62ec3e6424
Merge branch 'main' into dev
2024-09-07 10:52:49 +09:00
Kohya S
0005867ba5
update README, format code
2024-09-07 10:45:18 +09:00
Kohya S.
16bb5699ac
Merge pull request #1426 from sdbds/resize
...
Replacing CV2 resize to Pil resize
2024-09-07 10:22:52 +09:00
Kohya S.
319e4d9831
Merge pull request #1433 from millie-v/sample-image-without-cuda
...
Generate sample images without having CUDA (such as on Macs)
2024-09-07 10:19:55 +09:00
Nando Metzger
2a3aefb4e4
Update train_util.py, bug fix
2024-08-30 08:15:05 +02:00
sdbds
9ca7a5b6cc
instead cv2 LANCZOS4 resize to pil resize
2024-07-20 21:59:11 +08:00
sdbds
1f16b80e88
Revert "judge image size for using diff interpolation"
...
This reverts commit 87526942a6
2024-07-20 21:35:24 +08:00
Millie
2e67978ee2
Generate sample images without having CUDA (such as on Macs)
2024-07-18 11:52:58 -07:00
sdbds
87526942a6
judge image size for using diff interpolation
2024-07-12 22:56:38 +08:00
Kohya S
0b3e4f7ab6
show file name if error in load_image ref #1385
2024-06-25 20:03:09 +09:00
Kohya S
4a441889d4
Merge branch 'dev' into train_resume_step
2024-06-11 19:27:37 +09:00
Yuta Hayashibe
4ecbac131a
Bump crate-ci/typos from 1.19.0 to 1.21.0, fix typos, and updated _typos.toml ( Close #1307 )
2024-06-05 16:31:55 +09:00
Kohya S
4dbcef429b
update for corner cases
2024-06-04 21:26:55 +09:00
Kohya S
321e24d83b
Merge pull request #1353 from KohakuBlueleaf/train_resume_step
...
Resume correct step for "resume from state" feature.
2024-06-04 19:30:11 +09:00
Kohya S
e5bab69e3a
fix alpha mask without disk cache closes #1351 , ref #1339
2024-06-02 21:11:40 +09:00
Kohaku-Blueleaf
b2363f1021
Final implementation
2024-05-31 12:20:20 +08:00
Kohya S
e8cfd4ba1d
fix to work cond mask and alpha mask
2024-05-26 22:01:37 +09:00
Kohya S
da6fea3d97
simplify and update alpha mask to work with various cases
2024-05-19 21:26:18 +09:00
Kohya S
f2dd43e198
revert kwargs to explicit declaration
2024-05-19 19:23:59 +09:00
u-haru
db6752901f
画像のアルファチャンネルをlossのマスクとして使用するオプションを追加 ( #1223 )
...
* Add alpha_mask parameter and apply masked loss
* Fix type hint in trim_and_resize_if_required function
* Refactor code to use keyword arguments in train_util.py
* Fix alpha mask flipping logic
* Fix alpha mask initialization
* Fix alpha_mask transformation
* Cache alpha_mask
* Update alpha_masks to be on CPU
* Set flipped_alpha_masks to Null if option disabled
* Check if alpha_mask is None
* Set alpha_mask to None if option disabled
* Add description of alpha_mask option to docs
2024-05-19 19:07:25 +09:00
Kohya S
c68baae480
add --log_config option to enable/disable output training config
2024-05-19 17:21:04 +09:00
Kohya S
47187f7079
Merge pull request #1285 from ccharest93/main
...
Hyperparameter tracking
2024-05-19 16:31:33 +09:00
Kohya S
a384bf2187
Merge pull request #1313 from rockerBOO/patch-3
...
Add caption_separator to output for subset
2024-05-12 21:36:56 +09:00
Kohya S
1c296f7229
Merge pull request #1312 from rockerBOO/patch-2
...
Fix caption_separator missing in subset schema
2024-05-12 21:33:12 +09:00
Kohya S
3701507874
raise original error if error is occured in checking latents
2024-05-12 20:56:56 +09:00
Kohya S
78020936d2
Merge pull request #1278 from Cauldrath/catch_latent_error_file
...
Display name of error latent file
2024-05-12 20:46:25 +09:00
Kohya S
9ddb4d7a01
update readme and help message etc.
2024-05-12 17:55:08 +09:00
Kohya S
8d1b1acd33
Merge pull request #1266 from Zovjsra/feature/disable-mmap
...
Add "--disable_mmap_load_safetensors" parameter
2024-05-12 17:43:44 +09:00
Kohya S
1ffc0b330a
fix typo
2024-05-12 16:18:43 +09:00
Kohya S
017b82ebe3
update help message for fused_backward_pass
2024-05-06 15:05:42 +09:00
Dave Lage
8db0cadcee
Add caption_separator to output for subset
2024-05-02 18:08:28 -04:00
Dave Lage
dbb7bb288e
Fix caption_separator missing in subset schema
2024-05-02 17:39:35 -04:00
Cauldrath
040e26ff1d
Regenerate failed file
...
If a latent file fails to load, print out the path and the error, then return false to regenerate it
2024-04-21 13:46:31 -04:00
Maatra
b886d0a359
Cleaned typing to be in line with accelerate hyperparameters type resctrictions
2024-04-20 14:36:47 +01:00
Maatra
2c9db5d9f2
passing filtered hyperparameters to accelerate
2024-04-20 14:11:43 +01:00
Cauldrath
feefcf256e
Display name of error latent file
...
When trying to load stored latents, if an error occurs, this change will tell you what file failed to load
Currently it will just tell you that something failed without telling you which file
2024-04-18 23:15:36 -04:00
Zovjsra
64916a35b2
add disable_mmap to args
2024-04-16 16:40:08 +08:00
2kpr
4f203ce40d
Fused backward pass
2024-04-14 09:56:58 -05:00
Kohya S
bfb352bc43
change huber_schedule from exponential to snr
2024-04-07 21:07:52 +09:00
Kohya S
d30ebb205c
update readme, add metadata for network module
2024-04-07 14:58:17 +09:00
kabachuha
90b18795fc
Add option to use Scheduled Huber Loss in all training pipelines to improve resilience to data corruption ( #1228 )
...
* add huber loss and huber_c compute to train_util
* add reduction modes
* add huber_c retrieval from timestep getter
* move get timesteps and huber to own function
* add conditional loss to all training scripts
* add cond loss to train network
* add (scheduled) huber_loss to args
* fixup twice timesteps getting
* PHL-schedule should depend on noise scheduler's num timesteps
* *2 multiplier to huber loss cause of 1/2 a^2 conv.
The Taylor expansion of sqrt near zero gives 1/2 a^2, which differs from a^2 of the standard MSE loss. This change scales them better against one another
* add option for smooth l1 (huber / delta)
* unify huber scheduling
* add snr huber scheduler
---------
Co-authored-by: Kohya S <52813779+kohya-ss@users.noreply.github.com >
2024-04-07 13:54:21 +09:00
ykume
cd587ce62c
verify command line args if wandb is enabled
2024-04-05 08:23:03 +09:00
Disty0
4012fd24f6
IPEX fix pin_memory
2024-03-28 21:08:16 +03:00
Disty0
6f7e93d5cc
Add OpenVINO and ROCm ONNX Runtime for WD14
2024-03-27 03:21:13 +03:00
Kohya S
c86e356013
Merge branch 'dev' into dataset-cache
2024-03-26 19:43:40 +09:00
Kohya S
ab1e389347
Merge branch 'dev' into masked-loss
2024-03-26 19:39:30 +09:00
Kohya S
a2b8531627
make each script consistent, fix to work w/o DeepSpeed
2024-03-25 22:28:46 +09:00
