Commit Graph

1926 Commits

Author SHA1 Message Date
Kohya S
1dc873d9b4 update README and clean up code for schedulefree optimizer 2024-12-01 22:00:44 +09:00
Kohya S.
14c9ba925f Merge pull request #1811 from rockerBOO/schedule-free-prodigy
Allow unknown schedule-free optimizers to continue to module loader
2024-12-01 21:51:25 +09:00
Kohya S
34e7f509c4 docs: update README for huber loss 2024-12-01 21:36:24 +09:00
Kohya S.
bdf9a8cc29 Merge pull request #1815 from kohya-ss/flux-huber-loss
Flux huber loss
2024-12-01 21:29:53 +09:00
Kohya S
1476040787 fix: update help text for huber loss parameters in train_util.py 2024-12-01 21:26:39 +09:00
Kohya S
cc11989755 fix: refactor huber-loss calculation in multiple training scripts 2024-12-01 21:20:28 +09:00
Kohya S
0fe6320f09 fix flux_train.py is not working 2024-12-01 14:13:37 +09:00
Kohya S
14f642f88b fix: huber_schedule exponential not working on sd3_train.py 2024-12-01 13:30:35 +09:00
Kohya S.
a5a27fe4c3 Merge pull request #1808 from recris/huber-loss-flux
Implement pseudo Huber loss for Flux and SD3
2024-12-01 13:15:33 +09:00
recris
7b61e9eb58 Fix issues found in review (pt 2) 2024-11-30 11:36:40 +00:00
Kohya S
9c885e549d fix: improve pos_embed handling for oversized images and update resolution_area_to_latent_size, when sample image size > train image size 2024-11-30 18:25:50 +09:00
rockerBOO
6593cfbec1 Fix d * lr step log 2024-11-29 14:16:24 -05:00
rockerBOO
87f5224e2d Support d*lr for ProdigyPlus optimizer 2024-11-29 14:16:00 -05:00
rockerBOO
928b9393da Allow unknown schedule-free optimizers to continue to module loader 2024-11-29 14:12:34 -05:00
recris
740ec1d526 Fix issues found in review 2024-11-28 20:38:32 +00:00
recris
420a180d93 Implement pseudo Huber loss for Flux and SD3 2024-11-27 18:37:09 +00:00
kohya-ss
2a61fc0784 docs: fix typo from block_to_swap to blocks_to_swap in README 2024-11-20 21:20:35 +09:00
Kohya S
2a188f07e6 Fix to work DOP with bock swap 2024-11-17 16:12:10 +09:00
Kohya S.
0047bb1fc3 Merge pull request #1779 from kohya-ss/faster-block-swap
Improve block swap speed and apply to LoRA
2024-11-14 19:47:10 +09:00
Kohya S
fd2d879ac8 docs: update README 2024-11-14 19:43:08 +09:00
Kohya S
5c5b544b91 refactor: remove unused prepare_split_model method from FluxNetworkTrainer 2024-11-14 19:35:43 +09:00
Kohya S
2bb0f547d7 update grad hook creation to fix TE lr in sd3 fine tuning 2024-11-14 19:33:12 +09:00
Kohya S
2cb7a6db02 feat: add block swap for FLUX.1/SD3 LoRA training 2024-11-12 21:39:13 +09:00
Kohya S
17cf249d76 Merge branch 'sd3' into faster-block-swap 2024-11-12 08:49:15 +09:00
Kohya S
cde90b8903 feat: implement block swapping for FLUX.1 LoRA (WIP) 2024-11-12 08:49:05 +09:00
Kohya S
3fe94b058a update comment 2024-11-12 08:09:07 +09:00
Kohya S.
92482c7a07 Merge pull request #1774 from sdbds/avif_get_imagesize
Support avif get image size
2024-11-12 08:02:16 +09:00
Kohya S
7feaae5f06 Merge branch 'sd3' into faster-block-swap 2024-11-11 21:16:01 +09:00
Kohya S
02bd76e6c7 Refactor block swapping to utilize custom offloading utilities 2024-11-11 21:15:36 +09:00
sdbds
26bd4540a6 init 2024-11-11 09:25:28 +08:00
Kohya S
8fac3c3b08 update README 2024-11-09 19:56:02 +09:00
Kohya S.
2a2042a762 Merge pull request #1770 from feffy380/fix-size-from-cache
fix: sort order when getting image size from cache file
2024-11-09 19:51:03 +09:00
feffy380
b3248a8eef fix: sort order when getting image size from cache file 2024-11-07 14:31:05 +01:00
Kohya S
186aa5b97d fix illeagal block is swapped #1764 2024-11-07 22:16:05 +09:00
Kohya S
b8d3feca77 Merge branch 'sd3' into faster-block-swap 2024-11-07 21:43:48 +09:00
Kohya S
123474d784 Merge branch 'sd3' of https://github.com/kohya-ss/sd-scripts into sd3 2024-11-07 21:43:37 +09:00
Kohya S
e877b306c8 Merge branch 'dev' into sd3 2024-11-07 21:43:04 +09:00
Kohya S
6adb69be63 Merge branch 'main' into dev 2024-11-07 21:42:44 +09:00
Kohya S.
387b40ea37 Merge pull request #1769 from Dango233/patch-1
Update README.md
2024-11-07 21:41:12 +09:00
Kohya S
e5ac095749 add about dev and sd3 branch to README v0.8.8 2024-11-07 21:39:47 +09:00
Dango233
5eb6d209d5 Update README.md 2024-11-07 20:33:31 +08:00
Kohya S.
f264f4091f Update README.md 2024-11-07 21:30:31 +09:00
Kohya S
5e86323f12 Update README and clean-up the code for SD3 timesteps 2024-11-07 21:27:12 +09:00
Kohya S.
588ea9e123 Merge pull request #1768 from Dango233/dango/timesteps_fix
Dango/timesteps fix
2024-11-07 20:56:04 +09:00
Dango233
bafd10d558 Fix typo 2024-11-07 18:21:04 +08:00
Dango233
e54462a4a9 Fix SD3 trained lora loading and merging 2024-11-07 09:54:12 +00:00
Dango233
40ed54bfc0 Simplify Timestep weighting
* Remove diffusers dependency in ts & sigma calc
* support Shift setting
* Add uniform distribution
* Default to Uniform distribution and shift 1
2024-11-07 09:53:54 +00:00
Kohya S
43849030cf Fix to work without latent cache #1758 2024-11-06 21:33:28 +09:00
Kohya S
aab943cea3 remove unused weight swapping functions from utils.py 2024-11-05 23:27:41 +09:00
Kohya S
81c0c965a2 faster block swap 2024-11-05 21:22:42 +09:00