Kohya S
589c2aa025
update README
2024-05-13 21:20:37 +09:00
Kohya S
16677da0d9
fix create_network_from_weights doesn't work
2024-05-12 22:15:07 +09:00
Kohya S
a384bf2187
Merge pull request #1313 from rockerBOO/patch-3
...
Add caption_separator to output for subset
2024-05-12 21:36:56 +09:00
Kohya S
1c296f7229
Merge pull request #1312 from rockerBOO/patch-2
...
Fix caption_separator missing in subset schema
2024-05-12 21:33:12 +09:00
Kohya S
e96a5217c3
Merge pull request #1291 from frodo821/patch-1
...
removed unnecessary `torch` import on line 115
2024-05-12 21:14:50 +09:00
Kohya S
39b82f26e5
update readme
2024-05-12 20:58:45 +09:00
Kohya S
3701507874
raise original error if error is occured in checking latents
2024-05-12 20:56:56 +09:00
Kohya S
78020936d2
Merge pull request #1278 from Cauldrath/catch_latent_error_file
...
Display name of error latent file
2024-05-12 20:46:25 +09:00
Kohya S
9ddb4d7a01
update readme and help message etc.
2024-05-12 17:55:08 +09:00
Kohya S
8d1b1acd33
Merge pull request #1266 from Zovjsra/feature/disable-mmap
...
Add "--disable_mmap_load_safetensors" parameter
2024-05-12 17:43:44 +09:00
Kohya S
02298e3c4a
Merge pull request #1331 from kohya-ss/lora-plus
...
Lora plus
2024-05-12 17:04:58 +09:00
Kohya S
44190416c6
update docs etc.
2024-05-12 17:01:20 +09:00
Kohya S
3c8193f642
revert lora+ for lora_fa
2024-05-12 17:00:51 +09:00
Kohya S
c6a437054a
Merge branch 'dev' into lora-plus
2024-05-12 16:18:57 +09:00
Kohya S
1ffc0b330a
fix typo
2024-05-12 16:18:43 +09:00
Kohya S
e01e148705
Merge branch 'dev' into lora-plus
2024-05-12 16:17:52 +09:00
Kohya S
e9f3a622f4
Merge branch 'dev' into lora-plus
2024-05-12 16:17:27 +09:00
Kohya S
7983d3db5f
Merge pull request #1319 from kohya-ss/fused-backward-pass
...
Fused backward pass
2024-05-12 15:09:39 +09:00
Kohya S
bee8cee7e8
update README for fused optimizer
2024-05-12 15:08:52 +09:00
Kohya S
f3d2cf22ff
update README for fused optimizer
2024-05-12 15:03:02 +09:00
Kohya S
6dbc23cf63
Merge branch 'dev' into fused-backward-pass
2024-05-12 14:21:56 +09:00
Kohya S
c1ba0b4356
update readme
2024-05-12 14:21:10 +09:00
Kohya S
607e041f3d
chore: Refactor optimizer group
2024-05-12 14:16:41 +09:00
AngelBottomless
793aeb94da
fix get_trainable_params in controlnet-llite training
2024-05-07 18:21:31 +09:00
Kohya S
b56d5f7801
add experimental option to fuse params to optimizer groups
2024-05-06 21:35:39 +09:00
Kohya S
017b82ebe3
update help message for fused_backward_pass
2024-05-06 15:05:42 +09:00
Kohya S
2a359e0a41
Merge pull request #1259 from 2kpr/fused_backward_pass
...
Adafactor fused backward pass and optimizer step, lowers SDXL (@ 1024 resolution) VRAM usage to BF16(10GB)/FP32(16.4GB)
2024-05-06 15:01:56 +09:00
Kohya S
3fd8cdc55d
fix dylora loraplus
2024-05-06 14:03:19 +09:00
Kohya S
7fe81502d0
update loraplus on dylora/lofa_fa
2024-05-06 11:09:32 +09:00
Kohya S
52e64c69cf
add debug log
2024-05-04 18:43:52 +09:00
Kohya S
58c2d856ae
support block dim/lr for sdxl
2024-05-03 22:18:20 +09:00
Dave Lage
8db0cadcee
Add caption_separator to output for subset
2024-05-02 18:08:28 -04:00
Dave Lage
dbb7bb288e
Fix caption_separator missing in subset schema
2024-05-02 17:39:35 -04:00
Kohya S
969f82ab47
move loraplus args from args to network_args, simplify log lr desc
2024-04-29 20:04:25 +09:00
Kohya S
834445a1d6
Merge pull request #1233 from rockerBOO/lora-plus
...
Add LoRA+ support
2024-04-29 18:05:12 +09:00
frodo821
fdbb03c360
removed unnecessary torch import on line 115
...
as per #1290
2024-04-23 14:29:05 +09:00
Cauldrath
040e26ff1d
Regenerate failed file
...
If a latent file fails to load, print out the path and the error, then return false to regenerate it
2024-04-21 13:46:31 -04:00
Kohya S
0540c33aca
pop weights if available #1247
2024-04-21 17:45:29 +09:00
Kohya S
52652cba1a
disable main process check for deepspeed #1247
2024-04-21 17:41:32 +09:00
青龍聖者@bdsqlsz
5cb145d13b
Update train_util.py
2024-04-20 21:56:24 +08:00
Maatra
b886d0a359
Cleaned typing to be in line with accelerate hyperparameters type resctrictions
2024-04-20 14:36:47 +01:00
青龍聖者@bdsqlsz
4477116a64
fix train controlnet
2024-04-20 21:26:09 +08:00
Maatra
2c9db5d9f2
passing filtered hyperparameters to accelerate
2024-04-20 14:11:43 +01:00
Cauldrath
fc374375de
Allow negative learning rate
...
This can be used to train away from a group of images you don't want
As this moves the model away from a point instead of towards it, the change in the model is unbounded
So, don't set it too low. -4e-7 seemed to work well.
2024-04-18 23:29:01 -04:00
Cauldrath
feefcf256e
Display name of error latent file
...
When trying to load stored latents, if an error occurs, this change will tell you what file failed to load
Currently it will just tell you that something failed without telling you which file
2024-04-18 23:15:36 -04:00
Zovjsra
64916a35b2
add disable_mmap to args
2024-04-16 16:40:08 +08:00
2kpr
4f203ce40d
Fused backward pass
2024-04-14 09:56:58 -05:00
rockerBOO
68467bdf4d
Fix unset or invalid LR from making a param_group
2024-04-11 17:33:19 -04:00
rockerBOO
75833e84a1
Fix default LR, Add overall LoRA+ ratio, Add log
...
`--loraplus_ratio` added for both TE and UNet
Add log for lora+
2024-04-08 19:23:02 -04:00
Kohya S
71e2c91330
Merge pull request #1230 from kohya-ss/dependabot/github_actions/crate-ci/typos-1.19.0
...
Bump crate-ci/typos from 1.17.2 to 1.19.0
2024-04-07 21:14:18 +09:00
