Commit Graph

790 Commits

Author SHA1 Message Date
rockerBOO
695f38962c Move get_huber_threshold_if_needed 2025-01-03 15:25:12 -05:00
rockerBOO
0522070d19 Fix training, validation split, revert to using upstream implemenation 2025-01-03 15:20:25 -05:00
rockerBOO
6604b36044 Remove duplicate assignment 2025-01-03 02:04:59 -05:00
rockerBOO
fbfc2753eb Update text for train/reg with repeats 2025-01-03 01:53:12 -05:00
rockerBOO
c8c3569df2 Cleanup order, types, print to logger 2025-01-03 01:26:45 -05:00
rockerBOO
534059dea5 Typos and lingering is_train 2025-01-03 01:18:15 -05:00
rockerBOO
d23c7322ee Merge remote-tracking branch 'hina/feature/val-loss' into validation-loss-upstream
Modified implementation for process_batch and cleanup validation
recording
2025-01-03 00:48:08 -05:00
rockerBOO
7f6e124c7c Merge branch 'gesen2egee/val' into validation-loss-upstream
Modified various implementations to restore original behavior
2025-01-02 23:04:38 -05:00
rockerBOO
449c1c5c50 Adding modified train_util and config_util 2025-01-02 15:59:20 -05:00
gesen2egee
8743532963 val 2025-01-02 15:57:12 -05:00
Hina Chen
05bb9183fa Add Validation loss for LoRA training 2024-12-27 16:47:59 +08:00
nhamanasu
8e378cf03d add RAdamScheduleFree support 2024-12-11 19:43:44 +09:00
青龍聖者@bdsqlsz
abff4b0ec7 Unify controlnet parameters name and change scripts name. (#1821)
* Update sd3_train.py

* add freeze block lr

* Update train_util.py

* update

* Revert "add freeze block lr"

This reverts commit 8b1653548f.

# Conflicts:
#	library/train_util.py
#	sd3_train.py

* use same control net model path

* use controlnet_model_name_or_path
2024-12-07 17:12:46 +09:00
Kohya S
6bee18db4f fix: resolve model corruption issue with pos_embed when using --enable_scaled_pos_embed 2024-12-07 15:12:27 +09:00
kohya-ss
e369b9a252 docs: update README with FLUX.1 ControlNet training details and improve argument help text 2024-12-02 23:38:54 +09:00
Kohya S.
09a3740f6c Merge pull request #1813 from minux302/flux-controlnet
Add Flux ControlNet
2024-12-02 23:32:16 +09:00
Kohya S.
e3fd6c52a0 Merge pull request #1812 from rockerBOO/tests
Add pytest testing
2024-12-02 21:38:43 +09:00
Kohya S
1dc873d9b4 update README and clean up code for schedulefree optimizer 2024-12-01 22:00:44 +09:00
Kohya S.
14c9ba925f Merge pull request #1811 from rockerBOO/schedule-free-prodigy
Allow unknown schedule-free optimizers to continue to module loader
2024-12-01 21:51:25 +09:00
Kohya S
1476040787 fix: update help text for huber loss parameters in train_util.py 2024-12-01 21:26:39 +09:00
Kohya S
cc11989755 fix: refactor huber-loss calculation in multiple training scripts 2024-12-01 21:20:28 +09:00
Kohya S
14f642f88b fix: huber_schedule exponential not working on sd3_train.py 2024-12-01 13:30:35 +09:00
Kohya S.
a5a27fe4c3 Merge pull request #1808 from recris/huber-loss-flux
Implement pseudo Huber loss for Flux and SD3
2024-12-01 13:15:33 +09:00
recris
7b61e9eb58 Fix issues found in review (pt 2) 2024-11-30 11:36:40 +00:00
Kohya S
9c885e549d fix: improve pos_embed handling for oversized images and update resolution_area_to_latent_size, when sample image size > train image size 2024-11-30 18:25:50 +09:00
rockerBOO
c7cadbc8c7 Add pytest testing 2024-11-29 15:52:03 -05:00
rockerBOO
928b9393da Allow unknown schedule-free optimizers to continue to module loader 2024-11-29 14:12:34 -05:00
minux302
be5860f8e2 add schnell option to load_cn 2024-11-30 00:08:21 +09:00
minux302
9dff44d785 fix device 2024-11-29 14:40:38 +00:00
recris
740ec1d526 Fix issues found in review 2024-11-28 20:38:32 +00:00
recris
420a180d93 Implement pseudo Huber loss for Flux and SD3 2024-11-27 18:37:09 +00:00
minux302
0b5229a955 save cn 2024-11-21 15:55:27 +00:00
minux302
31ca899b6b fix depth value 2024-11-18 13:03:28 +00:00
minux302
4dd4cd6ec8 work cn load and validation 2024-11-18 12:47:01 +00:00
minux302
35778f0218 fix sample_images type 2024-11-17 11:09:05 +00:00
minux302
b2660bbe74 train run 2024-11-17 10:24:57 +00:00
minux302
e358b118af fix dataloader 2024-11-16 14:49:29 +09:00
minux302
42f6edf3a8 fix for adding controlnet 2024-11-15 23:48:51 +09:00
minux302
ccfaa001e7 add flux controlnet base module 2024-11-15 20:21:28 +09:00
Kohya S
2bb0f547d7 update grad hook creation to fix TE lr in sd3 fine tuning 2024-11-14 19:33:12 +09:00
Kohya S
2cb7a6db02 feat: add block swap for FLUX.1/SD3 LoRA training 2024-11-12 21:39:13 +09:00
Kohya S
17cf249d76 Merge branch 'sd3' into faster-block-swap 2024-11-12 08:49:15 +09:00
Kohya S
cde90b8903 feat: implement block swapping for FLUX.1 LoRA (WIP) 2024-11-12 08:49:05 +09:00
Kohya S
3fe94b058a update comment 2024-11-12 08:09:07 +09:00
Kohya S
7feaae5f06 Merge branch 'sd3' into faster-block-swap 2024-11-11 21:16:01 +09:00
Kohya S
02bd76e6c7 Refactor block swapping to utilize custom offloading utilities 2024-11-11 21:15:36 +09:00
sdbds
26bd4540a6 init 2024-11-11 09:25:28 +08:00
Kohya S.
2a2042a762 Merge pull request #1770 from feffy380/fix-size-from-cache
fix: sort order when getting image size from cache file
2024-11-09 19:51:03 +09:00
feffy380
b3248a8eef fix: sort order when getting image size from cache file 2024-11-07 14:31:05 +01:00
Kohya S
186aa5b97d fix illeagal block is swapped #1764 2024-11-07 22:16:05 +09:00