Kohya S
65fb69f808
Merge branch 'dev' into sd3
2024-09-25 20:56:16 +09:00
sdbds
ab7b231870
init
2024-09-25 19:38:52 +08:00
recris
e1f23af1bc
make timestep sampling behave in the standard way when huber loss is used
2024-09-21 13:21:56 +01:00
Kohya S
b844c70d14
Merge branch 'dev' into sd3
2024-09-19 21:51:33 +09:00
Maru-mee
e7040669bc
Bug fix: alpha_mask load
2024-09-19 15:47:06 +09:00
Kohya S
1286e00bb0
fix to call train/eval in schedulefree #1605
2024-09-18 21:31:54 +09:00
Kohya S.
bbd160b4ca
sd3 schedule free opt ( #1605 )
...
* New ScheduleFree support for Flux (#1600 )
* init
* use no schedule
* fix typo
* update for eval()
* fix typo
* update
* Update train_util.py
* Update requirements.txt
* update sfwrapper WIP
* no need to check schedulefree optimizer
* remove debug print
* comment out schedulefree wrapper
* update readme
---------
Co-authored-by: 青龍聖者@bdsqlsz <865105819@qq.com >
2024-09-18 07:55:04 +09:00
Kohya S
96c677b459
fix to work lienar/cosine lr scheduler closes #1602 ref #1393
2024-09-16 10:42:09 +09:00
Plat
a823fd9fb8
Improve wandb logging ( #1576 )
...
* fix: wrong training steps were recorded to wandb, and no log was sent when logging_dir was not specified
* fix: checking of whether wandb is enabled
* feat: log images to wandb with their positive prompt as captions
* feat: logging sample images' caption for sd3 and flux
* fix: import wandb before use
2024-09-11 22:21:16 +09:00
Kohya S
c7c666b182
fix typo
2024-09-11 22:12:31 +09:00
cocktailpeanut
8311e88225
typo fix
2024-09-11 09:02:29 -04:00
Kohya S
eaafa5c9da
Merge branch 'dev' into sd3
2024-09-11 21:46:21 +09:00
Kohya S
6dbfd47a59
Fix to work PIECEWISE_CONSTANT, update requirement.txt and README #1393
2024-09-11 21:44:36 +09:00
青龍聖者@bdsqlsz
fd68703f37
Add New lr scheduler ( #1393 )
...
* add new lr scheduler
* fix bugs and use num_cycles / 2
* Update requirements.txt
* add num_cycles for min lr
* keep PIECEWISE_CONSTANT
* allow use float with warmup or decay ratio.
* Update train_util.py
2024-09-11 21:25:45 +09:00
Kohya S
ce144476cf
Merge branch 'dev' into sd3
2024-09-07 10:59:22 +09:00
Kohya S
62ec3e6424
Merge branch 'main' into dev
2024-09-07 10:52:49 +09:00
Kohya S
0005867ba5
update README, format code
2024-09-07 10:45:18 +09:00
Kohya S.
16bb5699ac
Merge pull request #1426 from sdbds/resize
...
Replacing CV2 resize to Pil resize
2024-09-07 10:22:52 +09:00
Kohya S.
319e4d9831
Merge pull request #1433 from millie-v/sample-image-without-cuda
...
Generate sample images without having CUDA (such as on Macs)
2024-09-07 10:19:55 +09:00
Kohya S
92e7600cc2
Move freeze_blocks to sd3_train because it's only for sd3
2024-09-01 18:57:07 +09:00
青龍聖者@bdsqlsz
ef510b3cb9
Sd3 freeze x_block ( #1417 )
...
* Update sd3_train.py
* add freeze block lr
* Update train_util.py
* update
2024-09-01 18:41:01 +09:00
Nando Metzger
2a3aefb4e4
Update train_util.py, bug fix
2024-08-30 08:15:05 +02:00
Kohya S
81411a398e
speed up getting image sizes
2024-08-22 22:02:29 +09:00
kohya-ss
98c91a7625
Fix bug in FLUX multi GPU training
2024-08-22 12:37:41 +09:00
Kohya S
6ab48b09d8
feat: Support multi-resolution training with caching latents to disk
2024-08-20 21:39:43 +09:00
Kohya S
400955d3ea
add fine tuning FLUX.1 (WIP)
2024-08-17 15:36:18 +09:00
Kohya S
e45d3f8634
add merge LoRA script
2024-08-16 22:19:21 +09:00
kohya-ss
f5ce754bc2
Merge branch 'dev' into sd3
2024-08-13 21:00:44 +09:00
Kohya S
8a0f12dde8
update FLUX LoRA training
2024-08-10 23:42:05 +09:00
Kohya S
da4d0fe016
support attn mask for l+g/t5
2024-08-05 20:51:34 +09:00
Kohya S
41dee60383
Refactor caching mechanism for latents and text encoder outputs, etc.
2024-07-27 13:50:05 +09:00
sdbds
9ca7a5b6cc
instead cv2 LANCZOS4 resize to pil resize
2024-07-20 21:59:11 +08:00
sdbds
1f16b80e88
Revert "judge image size for using diff interpolation"
...
This reverts commit 87526942a6
2024-07-20 21:35:24 +08:00
Millie
2e67978ee2
Generate sample images without having CUDA (such as on Macs)
2024-07-18 11:52:58 -07:00
sdbds
87526942a6
judge image size for using diff interpolation
2024-07-12 22:56:38 +08:00
Kohya S
082f13658b
reduce peak GPU memory usage before training
2024-07-12 21:28:01 +09:00
Kohya S
3d402927ef
WIP: update new latents caching
2024-07-09 23:15:38 +09:00
Kohya S
c9de7c4e9a
WIP: new latents caching
2024-07-08 19:48:28 +09:00
Kohya S
8f2ba27869
support text_encoder_batch_size for caching
2024-06-26 20:36:22 +09:00
Kohya S
0b3e4f7ab6
show file name if error in load_image ref #1385
2024-06-25 20:03:09 +09:00
Kohya S
d53ea22b2a
sd3 training
2024-06-23 23:38:20 +09:00
Kohya S
4dbcef429b
update for corner cases
2024-06-04 21:26:55 +09:00
Kohya S
321e24d83b
Merge pull request #1353 from KohakuBlueleaf/train_resume_step
...
Resume correct step for "resume from state" feature.
2024-06-04 19:30:11 +09:00
Kohya S
e5bab69e3a
fix alpha mask without disk cache closes #1351 , ref #1339
2024-06-02 21:11:40 +09:00
Kohaku-Blueleaf
b2363f1021
Final implementation
2024-05-31 12:20:20 +08:00
Kohya S
e8cfd4ba1d
fix to work cond mask and alpha mask
2024-05-26 22:01:37 +09:00
Kohya S
da6fea3d97
simplify and update alpha mask to work with various cases
2024-05-19 21:26:18 +09:00
Kohya S
f2dd43e198
revert kwargs to explicit declaration
2024-05-19 19:23:59 +09:00
u-haru
db6752901f
画像のアルファチャンネルをlossのマスクとして使用するオプションを追加 ( #1223 )
...
* Add alpha_mask parameter and apply masked loss
* Fix type hint in trim_and_resize_if_required function
* Refactor code to use keyword arguments in train_util.py
* Fix alpha mask flipping logic
* Fix alpha mask initialization
* Fix alpha_mask transformation
* Cache alpha_mask
* Update alpha_masks to be on CPU
* Set flipped_alpha_masks to Null if option disabled
* Check if alpha_mask is None
* Set alpha_mask to None if option disabled
* Add description of alpha_mask option to docs
2024-05-19 19:07:25 +09:00
Kohya S
c68baae480
add --log_config option to enable/disable output training config
2024-05-19 17:21:04 +09:00
