Cauldrath
040e26ff1d
Regenerate failed file
...
If a latent file fails to load, print out the path and the error, then return false to regenerate it
2024-04-21 13:46:31 -04:00
Kohya S
0540c33aca
pop weights if available #1247
2024-04-21 17:45:29 +09:00
Kohya S
52652cba1a
disable main process check for deepspeed #1247
2024-04-21 17:41:32 +09:00
青龍聖者@bdsqlsz
5cb145d13b
Update train_util.py
2024-04-20 21:56:24 +08:00
Maatra
b886d0a359
Cleaned typing to be in line with accelerate hyperparameters type resctrictions
2024-04-20 14:36:47 +01:00
青龍聖者@bdsqlsz
4477116a64
fix train controlnet
2024-04-20 21:26:09 +08:00
Maatra
2c9db5d9f2
passing filtered hyperparameters to accelerate
2024-04-20 14:11:43 +01:00
Cauldrath
fc374375de
Allow negative learning rate
...
This can be used to train away from a group of images you don't want
As this moves the model away from a point instead of towards it, the change in the model is unbounded
So, don't set it too low. -4e-7 seemed to work well.
2024-04-18 23:29:01 -04:00
Cauldrath
feefcf256e
Display name of error latent file
...
When trying to load stored latents, if an error occurs, this change will tell you what file failed to load
Currently it will just tell you that something failed without telling you which file
2024-04-18 23:15:36 -04:00
Zovjsra
64916a35b2
add disable_mmap to args
2024-04-16 16:40:08 +08:00
2kpr
4f203ce40d
Fused backward pass
2024-04-14 09:56:58 -05:00
rockerBOO
68467bdf4d
Fix unset or invalid LR from making a param_group
2024-04-11 17:33:19 -04:00
rockerBOO
75833e84a1
Fix default LR, Add overall LoRA+ ratio, Add log
...
`--loraplus_ratio` added for both TE and UNet
Add log for lora+
2024-04-08 19:23:02 -04:00
Kohya S
71e2c91330
Merge pull request #1230 from kohya-ss/dependabot/github_actions/crate-ci/typos-1.19.0
...
Bump crate-ci/typos from 1.17.2 to 1.19.0
2024-04-07 21:14:18 +09:00
Kohya S
bfb352bc43
change huber_schedule from exponential to snr
2024-04-07 21:07:52 +09:00
Kohya S
c973b29da4
update readme
2024-04-07 20:51:52 +09:00
Kohya S
683f3d6ab3
Merge pull request #1212 from kohya-ss/dev
...
Version 0.8.6
2024-04-07 20:42:41 +09:00
Kohya S
dfa30790a9
update readme
2024-04-07 20:34:26 +09:00
Kohya S
d30ebb205c
update readme, add metadata for network module
2024-04-07 14:58:17 +09:00
kabachuha
90b18795fc
Add option to use Scheduled Huber Loss in all training pipelines to improve resilience to data corruption ( #1228 )
...
* add huber loss and huber_c compute to train_util
* add reduction modes
* add huber_c retrieval from timestep getter
* move get timesteps and huber to own function
* add conditional loss to all training scripts
* add cond loss to train network
* add (scheduled) huber_loss to args
* fixup twice timesteps getting
* PHL-schedule should depend on noise scheduler's num timesteps
* *2 multiplier to huber loss cause of 1/2 a^2 conv.
The Taylor expansion of sqrt near zero gives 1/2 a^2, which differs from a^2 of the standard MSE loss. This change scales them better against one another
* add option for smooth l1 (huber / delta)
* unify huber scheduling
* add snr huber scheduler
---------
Co-authored-by: Kohya S <52813779+kohya-ss@users.noreply.github.com >
2024-04-07 13:54:21 +09:00
Kohya S
089727b5ee
update readme
2024-04-07 12:42:49 +09:00
Kohya S
921036dd91
Merge pull request #1240 from kohya-ss/verify-command-line-args
...
verify command line args if wandb is enabled
2024-04-07 12:27:03 +09:00
ykume
cd587ce62c
verify command line args if wandb is enabled
2024-04-05 08:23:03 +09:00
rockerBOO
1933ab4b48
Fix default_lr being applied
2024-04-03 12:46:34 -04:00
Kohya S
b748b48dbb
fix attention couple+deep shink cause error in some reso
2024-04-03 12:43:08 +09:00
rockerBOO
c7691607ea
Add LoRA-FA for LoRA+
2024-04-01 15:43:04 -04:00
rockerBOO
f99fe281cb
Add LoRA+ support
2024-04-01 15:38:26 -04:00
dependabot[bot]
80e9f72234
Bump crate-ci/typos from 1.17.2 to 1.19.0
...
Bumps [crate-ci/typos](https://github.com/crate-ci/typos ) from 1.17.2 to 1.19.0.
- [Release notes](https://github.com/crate-ci/typos/releases )
- [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md )
- [Commits](https://github.com/crate-ci/typos/compare/v1.17.2...v1.19.0 )
---
updated-dependencies:
- dependency-name: crate-ci/typos
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com >
2024-04-01 01:50:22 +00:00
Kohya S
2258a1b753
add save/load hook to remove U-Net/TEs from state
2024-03-31 15:50:35 +09:00
Kohya S
059ee047f3
fix typo
2024-03-30 23:02:24 +09:00
Kohya S
2c2ca9d726
update tagger doc
2024-03-30 22:55:56 +09:00
Kohya S
f5323e3c4b
update tagger doc
2024-03-30 22:10:37 +09:00
Kohya S
cae5aa0a56
update wd14 tagger and doc
2024-03-30 21:48:22 +09:00
Kohya S
6ba84288d9
Merge pull request #1216 from Disty0/dev
...
Rating support for WD Tagger
2024-03-30 18:50:49 +09:00
Kohya S
434dc408f9
update readme
2024-03-30 17:12:36 +09:00
Kohya S
ae3f625739
Merge branch 'dev' of https://github.com/kohya-ss/sd-scripts into dev
2024-03-30 14:57:43 +09:00
Kohya S
f1f30ab418
fix to work with num_beams>1 closes #1149
2024-03-30 14:57:39 +09:00
Disty0
bc586ce190
Add --use_rating_tags and --character_tags_first for WD Tagger
2024-03-29 13:56:42 +03:00
Disty0
4012fd24f6
IPEX fix pin_memory
2024-03-28 21:08:16 +03:00
Disty0
954731d564
fix typo
2024-03-27 22:00:59 +03:00
Disty0
dd9763be31
Rating support for WD Tagger
2024-03-27 21:53:40 +03:00
Kohya S
b86af6798d
Merge pull request #1213 from Disty0/dev
...
Add OpenVINO and ROCm ONNX Runtime for WD14
2024-03-27 23:15:33 +09:00
Disty0
6f7e93d5cc
Add OpenVINO and ROCm ONNX Runtime for WD14
2024-03-27 03:21:13 +03:00
Kohya S
6c08e97e1f
update readme
2024-03-26 20:48:08 +09:00
Kohya S
78e0a7630c
Merge pull request #1206 from kohya-ss/dataset-cache
...
Add metadata caching for DreamBooth dataset
2024-03-26 19:49:23 +09:00
Kohya S
c86e356013
Merge branch 'dev' into dataset-cache
2024-03-26 19:43:40 +09:00
Kohya S
5a2afb3588
Merge pull request #1207 from kohya-ss/masked-loss
...
Add masked loss
2024-03-26 19:41:31 +09:00
Kohya S
ab1e389347
Merge branch 'dev' into masked-loss
2024-03-26 19:39:30 +09:00
Kohya S
ea05e3fd5b
Merge pull request #1139 from kohya-ss/deep-speed
...
Deep speed
2024-03-26 19:33:57 +09:00
Kohya S
a2b8531627
make each script consistent, fix to work w/o DeepSpeed
2024-03-25 22:28:46 +09:00
