Kohya S
0005867ba5
update README, format code
2024-09-07 10:45:18 +09:00
Kohya S.
16bb5699ac
Merge pull request #1426 from sdbds/resize
...
Replacing CV2 resize to Pil resize
2024-09-07 10:22:52 +09:00
Kohya S.
319e4d9831
Merge pull request #1433 from millie-v/sample-image-without-cuda
...
Generate sample images without having CUDA (such as on Macs)
2024-09-07 10:19:55 +09:00
Kohya S
92e7600cc2
Move freeze_blocks to sd3_train because it's only for sd3
2024-09-01 18:57:07 +09:00
青龍聖者@bdsqlsz
ef510b3cb9
Sd3 freeze x_block ( #1417 )
...
* Update sd3_train.py
* add freeze block lr
* Update train_util.py
* update
2024-09-01 18:41:01 +09:00
Nando Metzger
2a3aefb4e4
Update train_util.py, bug fix
2024-08-30 08:15:05 +02:00
Kohya S
81411a398e
speed up getting image sizes
2024-08-22 22:02:29 +09:00
kohya-ss
98c91a7625
Fix bug in FLUX multi GPU training
2024-08-22 12:37:41 +09:00
Kohya S
6ab48b09d8
feat: Support multi-resolution training with caching latents to disk
2024-08-20 21:39:43 +09:00
Kohya S
400955d3ea
add fine tuning FLUX.1 (WIP)
2024-08-17 15:36:18 +09:00
Kohya S
e45d3f8634
add merge LoRA script
2024-08-16 22:19:21 +09:00
kohya-ss
f5ce754bc2
Merge branch 'dev' into sd3
2024-08-13 21:00:44 +09:00
Kohya S
8a0f12dde8
update FLUX LoRA training
2024-08-10 23:42:05 +09:00
Kohya S
da4d0fe016
support attn mask for l+g/t5
2024-08-05 20:51:34 +09:00
Kohya S
41dee60383
Refactor caching mechanism for latents and text encoder outputs, etc.
2024-07-27 13:50:05 +09:00
sdbds
9ca7a5b6cc
instead cv2 LANCZOS4 resize to pil resize
2024-07-20 21:59:11 +08:00
sdbds
1f16b80e88
Revert "judge image size for using diff interpolation"
...
This reverts commit 87526942a6
2024-07-20 21:35:24 +08:00
Millie
2e67978ee2
Generate sample images without having CUDA (such as on Macs)
2024-07-18 11:52:58 -07:00
sdbds
87526942a6
judge image size for using diff interpolation
2024-07-12 22:56:38 +08:00
Kohya S
082f13658b
reduce peak GPU memory usage before training
2024-07-12 21:28:01 +09:00
Kohya S
3d402927ef
WIP: update new latents caching
2024-07-09 23:15:38 +09:00
Kohya S
c9de7c4e9a
WIP: new latents caching
2024-07-08 19:48:28 +09:00
Kohya S
8f2ba27869
support text_encoder_batch_size for caching
2024-06-26 20:36:22 +09:00
Kohya S
0b3e4f7ab6
show file name if error in load_image ref #1385
2024-06-25 20:03:09 +09:00
Kohya S
d53ea22b2a
sd3 training
2024-06-23 23:38:20 +09:00
Kohya S
4dbcef429b
update for corner cases
2024-06-04 21:26:55 +09:00
Kohya S
321e24d83b
Merge pull request #1353 from KohakuBlueleaf/train_resume_step
...
Resume correct step for "resume from state" feature.
2024-06-04 19:30:11 +09:00
Kohya S
e5bab69e3a
fix alpha mask without disk cache closes #1351 , ref #1339
2024-06-02 21:11:40 +09:00
Kohaku-Blueleaf
b2363f1021
Final implementation
2024-05-31 12:20:20 +08:00
Kohya S
e8cfd4ba1d
fix to work cond mask and alpha mask
2024-05-26 22:01:37 +09:00
Kohya S
da6fea3d97
simplify and update alpha mask to work with various cases
2024-05-19 21:26:18 +09:00
Kohya S
f2dd43e198
revert kwargs to explicit declaration
2024-05-19 19:23:59 +09:00
u-haru
db6752901f
画像のアルファチャンネルをlossのマスクとして使用するオプションを追加 ( #1223 )
...
* Add alpha_mask parameter and apply masked loss
* Fix type hint in trim_and_resize_if_required function
* Refactor code to use keyword arguments in train_util.py
* Fix alpha mask flipping logic
* Fix alpha mask initialization
* Fix alpha_mask transformation
* Cache alpha_mask
* Update alpha_masks to be on CPU
* Set flipped_alpha_masks to Null if option disabled
* Check if alpha_mask is None
* Set alpha_mask to None if option disabled
* Add description of alpha_mask option to docs
2024-05-19 19:07:25 +09:00
Kohya S
c68baae480
add --log_config option to enable/disable output training config
2024-05-19 17:21:04 +09:00
Kohya S
47187f7079
Merge pull request #1285 from ccharest93/main
...
Hyperparameter tracking
2024-05-19 16:31:33 +09:00
Kohya S
3701507874
raise original error if error is occured in checking latents
2024-05-12 20:56:56 +09:00
Kohya S
78020936d2
Merge pull request #1278 from Cauldrath/catch_latent_error_file
...
Display name of error latent file
2024-05-12 20:46:25 +09:00
Kohya S
1ffc0b330a
fix typo
2024-05-12 16:18:43 +09:00
Kohya S
017b82ebe3
update help message for fused_backward_pass
2024-05-06 15:05:42 +09:00
Cauldrath
040e26ff1d
Regenerate failed file
...
If a latent file fails to load, print out the path and the error, then return false to regenerate it
2024-04-21 13:46:31 -04:00
Maatra
b886d0a359
Cleaned typing to be in line with accelerate hyperparameters type resctrictions
2024-04-20 14:36:47 +01:00
Maatra
2c9db5d9f2
passing filtered hyperparameters to accelerate
2024-04-20 14:11:43 +01:00
Cauldrath
feefcf256e
Display name of error latent file
...
When trying to load stored latents, if an error occurs, this change will tell you what file failed to load
Currently it will just tell you that something failed without telling you which file
2024-04-18 23:15:36 -04:00
2kpr
4f203ce40d
Fused backward pass
2024-04-14 09:56:58 -05:00
Kohya S
bfb352bc43
change huber_schedule from exponential to snr
2024-04-07 21:07:52 +09:00
Kohya S
d30ebb205c
update readme, add metadata for network module
2024-04-07 14:58:17 +09:00
kabachuha
90b18795fc
Add option to use Scheduled Huber Loss in all training pipelines to improve resilience to data corruption ( #1228 )
...
* add huber loss and huber_c compute to train_util
* add reduction modes
* add huber_c retrieval from timestep getter
* move get timesteps and huber to own function
* add conditional loss to all training scripts
* add cond loss to train network
* add (scheduled) huber_loss to args
* fixup twice timesteps getting
* PHL-schedule should depend on noise scheduler's num timesteps
* *2 multiplier to huber loss cause of 1/2 a^2 conv.
The Taylor expansion of sqrt near zero gives 1/2 a^2, which differs from a^2 of the standard MSE loss. This change scales them better against one another
* add option for smooth l1 (huber / delta)
* unify huber scheduling
* add snr huber scheduler
---------
Co-authored-by: Kohya S <52813779+kohya-ss@users.noreply.github.com >
2024-04-07 13:54:21 +09:00
ykume
cd587ce62c
verify command line args if wandb is enabled
2024-04-05 08:23:03 +09:00
Kohya S
c86e356013
Merge branch 'dev' into dataset-cache
2024-03-26 19:43:40 +09:00
Kohya S
ab1e389347
Merge branch 'dev' into masked-loss
2024-03-26 19:39:30 +09:00
