Kohya S
a384bf2187
Merge pull request #1313 from rockerBOO/patch-3
...
Add caption_separator to output for subset
2024-05-12 21:36:56 +09:00
Kohya S
1c296f7229
Merge pull request #1312 from rockerBOO/patch-2
...
Fix caption_separator missing in subset schema
2024-05-12 21:33:12 +09:00
Kohya S
e96a5217c3
Merge pull request #1291 from frodo821/patch-1
...
removed unnecessary `torch` import on line 115
2024-05-12 21:14:50 +09:00
Kohya S
39b82f26e5
update readme
2024-05-12 20:58:45 +09:00
Kohya S
3701507874
raise original error if error is occured in checking latents
2024-05-12 20:56:56 +09:00
Kohya S
78020936d2
Merge pull request #1278 from Cauldrath/catch_latent_error_file
...
Display name of error latent file
2024-05-12 20:46:25 +09:00
Kohya S
9ddb4d7a01
update readme and help message etc.
2024-05-12 17:55:08 +09:00
Kohya S
8d1b1acd33
Merge pull request #1266 from Zovjsra/feature/disable-mmap
...
Add "--disable_mmap_load_safetensors" parameter
2024-05-12 17:43:44 +09:00
Kohya S
02298e3c4a
Merge pull request #1331 from kohya-ss/lora-plus
...
Lora plus
2024-05-12 17:04:58 +09:00
Kohya S
44190416c6
update docs etc.
2024-05-12 17:01:20 +09:00
Kohya S
3c8193f642
revert lora+ for lora_fa
2024-05-12 17:00:51 +09:00
Kohya S
c6a437054a
Merge branch 'dev' into lora-plus
2024-05-12 16:18:57 +09:00
Kohya S
1ffc0b330a
fix typo
2024-05-12 16:18:43 +09:00
Kohya S
e01e148705
Merge branch 'dev' into lora-plus
2024-05-12 16:17:52 +09:00
Kohya S
e9f3a622f4
Merge branch 'dev' into lora-plus
2024-05-12 16:17:27 +09:00
Kohya S
7983d3db5f
Merge pull request #1319 from kohya-ss/fused-backward-pass
...
Fused backward pass
2024-05-12 15:09:39 +09:00
Kohya S
bee8cee7e8
update README for fused optimizer
2024-05-12 15:08:52 +09:00
Kohya S
f3d2cf22ff
update README for fused optimizer
2024-05-12 15:03:02 +09:00
Kohya S
6dbc23cf63
Merge branch 'dev' into fused-backward-pass
2024-05-12 14:21:56 +09:00
Kohya S
c1ba0b4356
update readme
2024-05-12 14:21:10 +09:00
Kohya S
607e041f3d
chore: Refactor optimizer group
2024-05-12 14:16:41 +09:00
Kohya S
b56d5f7801
add experimental option to fuse params to optimizer groups
2024-05-06 21:35:39 +09:00
Kohya S
017b82ebe3
update help message for fused_backward_pass
2024-05-06 15:05:42 +09:00
Kohya S
2a359e0a41
Merge pull request #1259 from 2kpr/fused_backward_pass
...
Adafactor fused backward pass and optimizer step, lowers SDXL (@ 1024 resolution) VRAM usage to BF16(10GB)/FP32(16.4GB)
2024-05-06 15:01:56 +09:00
Kohya S
3fd8cdc55d
fix dylora loraplus
2024-05-06 14:03:19 +09:00
Kohya S
7fe81502d0
update loraplus on dylora/lofa_fa
2024-05-06 11:09:32 +09:00
Kohya S
52e64c69cf
add debug log
2024-05-04 18:43:52 +09:00
Kohya S
58c2d856ae
support block dim/lr for sdxl
2024-05-03 22:18:20 +09:00
Dave Lage
8db0cadcee
Add caption_separator to output for subset
2024-05-02 18:08:28 -04:00
Dave Lage
dbb7bb288e
Fix caption_separator missing in subset schema
2024-05-02 17:39:35 -04:00
Kohya S
969f82ab47
move loraplus args from args to network_args, simplify log lr desc
2024-04-29 20:04:25 +09:00
Kohya S
834445a1d6
Merge pull request #1233 from rockerBOO/lora-plus
...
Add LoRA+ support
2024-04-29 18:05:12 +09:00
frodo821
fdbb03c360
removed unnecessary torch import on line 115
...
as per #1290
2024-04-23 14:29:05 +09:00
Cauldrath
040e26ff1d
Regenerate failed file
...
If a latent file fails to load, print out the path and the error, then return false to regenerate it
2024-04-21 13:46:31 -04:00
Kohya S
0540c33aca
pop weights if available #1247
2024-04-21 17:45:29 +09:00
Kohya S
52652cba1a
disable main process check for deepspeed #1247
2024-04-21 17:41:32 +09:00
Cauldrath
feefcf256e
Display name of error latent file
...
When trying to load stored latents, if an error occurs, this change will tell you what file failed to load
Currently it will just tell you that something failed without telling you which file
2024-04-18 23:15:36 -04:00
Zovjsra
64916a35b2
add disable_mmap to args
2024-04-16 16:40:08 +08:00
2kpr
4f203ce40d
Fused backward pass
2024-04-14 09:56:58 -05:00
rockerBOO
68467bdf4d
Fix unset or invalid LR from making a param_group
2024-04-11 17:33:19 -04:00
rockerBOO
75833e84a1
Fix default LR, Add overall LoRA+ ratio, Add log
...
`--loraplus_ratio` added for both TE and UNet
Add log for lora+
2024-04-08 19:23:02 -04:00
Kohya S
71e2c91330
Merge pull request #1230 from kohya-ss/dependabot/github_actions/crate-ci/typos-1.19.0
...
Bump crate-ci/typos from 1.17.2 to 1.19.0
2024-04-07 21:14:18 +09:00
Kohya S
bfb352bc43
change huber_schedule from exponential to snr
2024-04-07 21:07:52 +09:00
Kohya S
c973b29da4
update readme
2024-04-07 20:51:52 +09:00
Kohya S
683f3d6ab3
Merge pull request #1212 from kohya-ss/dev
...
Version 0.8.6
2024-04-07 20:42:41 +09:00
Kohya S
dfa30790a9
update readme
2024-04-07 20:34:26 +09:00
Kohya S
d30ebb205c
update readme, add metadata for network module
2024-04-07 14:58:17 +09:00
kabachuha
90b18795fc
Add option to use Scheduled Huber Loss in all training pipelines to improve resilience to data corruption ( #1228 )
...
* add huber loss and huber_c compute to train_util
* add reduction modes
* add huber_c retrieval from timestep getter
* move get timesteps and huber to own function
* add conditional loss to all training scripts
* add cond loss to train network
* add (scheduled) huber_loss to args
* fixup twice timesteps getting
* PHL-schedule should depend on noise scheduler's num timesteps
* *2 multiplier to huber loss cause of 1/2 a^2 conv.
The Taylor expansion of sqrt near zero gives 1/2 a^2, which differs from a^2 of the standard MSE loss. This change scales them better against one another
* add option for smooth l1 (huber / delta)
* unify huber scheduling
* add snr huber scheduler
---------
Co-authored-by: Kohya S <52813779+kohya-ss@users.noreply.github.com >
2024-04-07 13:54:21 +09:00
Kohya S
089727b5ee
update readme
2024-04-07 12:42:49 +09:00
Kohya S
921036dd91
Merge pull request #1240 from kohya-ss/verify-command-line-args
...
verify command line args if wandb is enabled
2024-04-07 12:27:03 +09:00
