Kohya S
496c3f2732
arbitrary args for diffusers lr scheduler
2023-07-30 14:36:03 +09:00
Kohya S
a296654c1b
refactor optimizer selection for bnb
2023-07-30 13:43:29 +09:00
Kohya S
e6034b7eb6
move releasing cache outside of the loop
2023-07-30 13:30:42 +09:00
青龍聖者@bdsqlsz
9ec70252d0
Add Paged/ adam8bit/lion8bit for Sdxl bitsandbytes 0.39.1 cuda118 on windows ( #623 )
...
* ADD libbitsandbytes.dll for 0.38.1
* Delete libbitsandbytes_cuda116.dll
* Delete cextension.py
* add main.py
* Update requirements.txt for bitsandbytes 0.38.1
* Update README.md for bitsandbytes-windows
* Update README-ja.md for bitsandbytes 0.38.1
* Update main.py for return cuda118
* Update train_util.py for lion8bit
* Update train_README-ja.md for lion8bit
* Update train_util.py for add DAdaptAdan and DAdaptSGD
* Update train_util.py for DAdaptadam
* Update train_network.py for dadapt
* Update train_README-ja.md for DAdapt
* Update train_util.py for DAdapt
* Update train_network.py for DAdaptAdaGrad
* Update train_db.py for DAdapt
* Update fine_tune.py for DAdapt
* Update train_textual_inversion.py for DAdapt
* Update train_textual_inversion_XTI.py for DAdapt
* Revert "Merge branch 'qinglong' into main"
This reverts commit b65c023083f6fda20caf83abc60dfa3ba4dfe0464642c52086fa6d7485acpamhome21@gmail.com >
2023-07-30 13:15:13 +09:00
Kohya S
4072f723c1
Merge branch 'main' into sdxl
2023-07-29 14:55:03 +09:00
Kohya S
50b53e183e
re-organize import
2023-07-23 13:33:02 +09:00
青龍聖者@bdsqlsz
d131bde183
Support for bitsandbytes 0.39.1 with Paged Optimizer(AdamW8bit and Lion8bit) ( #631 )
...
* ADD libbitsandbytes.dll for 0.38.1
* Delete libbitsandbytes_cuda116.dll
* Delete cextension.py
* add main.py
* Update requirements.txt for bitsandbytes 0.38.1
* Update README.md for bitsandbytes-windows
* Update README-ja.md for bitsandbytes 0.38.1
* Update main.py for return cuda118
* Update train_util.py for lion8bit
* Update train_README-ja.md for lion8bit
* Update train_util.py for add DAdaptAdan and DAdaptSGD
* Update train_util.py for DAdaptadam
* Update train_network.py for dadapt
* Update train_README-ja.md for DAdapt
* Update train_util.py for DAdapt
* Update train_network.py for DAdaptAdaGrad
* Update train_db.py for DAdapt
* Update fine_tune.py for DAdapt
* Update train_textual_inversion.py for DAdapt
* Update train_textual_inversion_XTI.py for DAdapt
* Revert "Merge branch 'qinglong' into main"
This reverts commit b65c023083f6fda20caf83abc60dfa3ba4dfe0464642c52086fa6d7485ac
2023-07-22 19:45:32 +09:00
Kohya S
73a08c0be0
Merge pull request #630 from ddPn08/sdxl
...
make tracker init_kwargs configurable
2023-07-20 22:05:55 +09:00
Kohya S
acf16c063a
make to work with PyTorch 1.12
2023-07-20 21:41:16 +09:00
Kohya S
225e871819
enable full bf16 trainint in train_network
2023-07-19 08:41:42 +09:00
Kohya S
6d2d8dfd2f
add zero_terminal_snr option
2023-07-18 23:17:23 +09:00
Kohya S
0ec7166098
make crop top/left same as stabilityai's prep
2023-07-18 21:39:36 +09:00
Kohya S
41d195715d
fix scheduler steps with gradient accumulation
2023-07-16 15:56:29 +09:00
Kohya S
516f64f4d9
add caching to disk for text encoder outputs
2023-07-16 14:53:47 +09:00
Kohya S
94c151aea3
refactor caching latents (flip in same npz, etc)
2023-07-15 18:28:33 +09:00
Kohya S
81fa54837f
fix sampling in multi GPU training
2023-07-15 11:21:14 +09:00
Kohya S
814996b14f
fix NaN in sampling image
2023-07-11 23:18:35 +09:00
ddPn08
b841dd78fe
make tracker init_kwargs configurable
2023-07-11 10:21:45 +09:00
Kohya S
b6e328ea8f
don't hold latent on memory for finetuning dataset
2023-07-10 08:46:15 +09:00
Kohya S
3579b4570f
Merge pull request #628 from KohakuBlueleaf/full_bf16
...
Full bf16 support
2023-07-09 14:22:44 +09:00
Kohaku-Blueleaf
d974959738
Update train_util.py for full_bf16 support
2023-07-09 12:47:26 +08:00
ddPn08
d599394f60
support avif
2023-07-08 15:47:56 +09:00
Kohya S
cc3d40ca44
support sdxl in prepare scipt
2023-07-07 21:16:41 +09:00
Kohya S
3060eb5baf
remove debug print
2023-07-05 21:44:46 +09:00
Kohya S
2febbfe4b0
add error message for old npz
2023-07-03 20:58:35 +09:00
Kohya S
ea182461d3
add min/max_timestep
2023-07-03 20:44:42 +09:00
Kohya S
64cf922841
add feature to sample images during sdxl training
2023-07-02 16:42:19 +09:00
Kohya S
d395bc0647
fix max_token_length not works for sdxl
2023-06-29 13:02:19 +09:00
Kohya S
2c461e4ad3
Add no_half_vae for SDXL training, add nan check
2023-06-26 20:38:09 +09:00
Kohya S
747af145ed
add sdxl fine-tuning and LoRA
2023-06-26 08:07:24 +09:00
Kohya S
9e9df2b501
update dataset to return size, refactor ctrlnet ds
2023-06-24 17:56:02 +09:00
Kohya S
92e50133f8
Merge branch 'original-u-net' into dev
2023-06-17 21:57:08 +09:00
Kohya S
19dfa24abb
Merge branch 'main' into original-u-net
2023-06-16 20:59:34 +09:00
Kohya S
5845de7d7c
common lr checking for dadaptation and prodigy
2023-06-15 21:47:37 +09:00
青龍聖者@bdsqlsz
e97d67a681
Support for Prodigy(Dadapt variety for Dylora) ( #585 )
...
* Update train_util.py for DAdaptLion
* Update train_README-zh.md for dadaptlion
* Update train_README-ja.md for DAdaptLion
* add DAdatpt V3
* Alignment
* Update train_util.py for experimental
* Update train_util.py V3
* Update train_README-zh.md
* Update train_README-ja.md
* Update train_util.py fix
* Update train_util.py
* support Prodigy
* add lower
2023-06-15 21:12:53 +09:00
Kohya S
9806b00f74
add arbitrary dataset feature to each script
2023-06-15 20:39:39 +09:00
Kohya S
449ad7502c
use original unet for HF models, don't download TE
2023-06-14 22:26:05 +09:00
Kohya S
9aee793078
support arbitrary dataset for train_network.py
2023-06-14 12:49:12 +09:00
ykume
9e1683cf2b
support sdpa
2023-06-11 21:26:15 +09:00
ykume
0315611b11
remove workaround for accelerator=0.15, fix XTI
2023-06-11 18:32:14 +09:00
ykume
035dd3a900
fix mem_eff_attn does not work
2023-06-11 17:08:21 +09:00
ykume
4e25c8f78e
fix to work with Diffusers 0.17.0
2023-06-11 16:57:17 +09:00
Kohya S
c0a7df9ee1
fix eps value, enable xformers, etc.
2023-06-03 21:29:27 +09:00
Kohya S
ec2efe52e4
scale v-pred loss like noise pred
2023-06-03 10:52:22 +09:00
ddPn08
1e3daa247b
fix bucketing
2023-06-01 21:58:45 +09:00
ddPn08
3bd00b88c2
support for controlnet in sample output
2023-06-01 20:48:30 +09:00
ddPn08
62d00b4520
add controlnet training
2023-06-01 20:48:25 +09:00
ddPn08
c8d209d36c
update diffusers to 1.16 | train_network
2023-06-01 20:39:26 +09:00
AI-Casanova
9c7237157d
Dropout and Max Norm Regularization for LoRA training ( #545 )
...
* Instantiate max_norm
* minor
* Move to end of step
* argparse
* metadata
* phrasing
* Sqrt ratio and logging
* fix logging
* Dropout test
* Dropout Args
* Dropout changed to affect LoRA only
---------
Co-authored-by: Kohya S <52813779+kohya-ss@users.noreply.github.com >
2023-06-01 14:58:38 +09:00
Kohya S
3a06968332
warn and continue if huggingface uploading failed
2023-05-31 20:48:33 +09:00
