Kohya S
8d5858826f
Merge branch 'dev' into masked-loss
2024-03-24 18:19:53 +09:00
Kohya S
025347214d
refactor metadata caching for DreamBooth dataset
2024-03-24 18:09:32 +09:00
Kohaku-Blueleaf
ae97c8bfd1
[Experimental] Add cache mechanism for dataset groups to avoid long waiting time for initilization ( #1178 )
...
* support meta cached dataset
* add cache meta scripts
* random ip_noise_gamma strength
* random noise_offset strength
* use correct settings for parser
* cache path/caption/size only
* revert mess up commit
* revert mess up commit
* Update requirements.txt
* Add arguments for meta cache.
* remove pickle implementation
* Return sizes when enable cache
---------
Co-authored-by: Kohya S <52813779+kohya-ss@users.noreply.github.com >
2024-03-24 15:40:18 +09:00
Kohya S
381c44955e
update readme and typing hint
2024-03-24 11:27:18 +09:00
Kohya S
ad97410ba5
Merge pull request #1205 from feffy380/patch-1
...
register reg images with correct subset
2024-03-24 11:14:07 +09:00
Kohya S
691f04322a
update readme
2024-03-24 11:10:26 +09:00
Kohya S
79d1c12ab0
disable sample_every_n_xxx if value less than 1 ref #1202
2024-03-24 11:06:37 +09:00
feffy380
0c7baea88c
register reg images with correct subset
2024-03-23 17:28:02 +01:00
Kohya S
f4a4c11cd3
support multiline captions ref #1155
2024-03-23 18:51:37 +09:00
Kohya S
594c7f7050
format by black
2024-03-23 16:11:31 +09:00
Kohya S
d17c0f5084
update dataset config doc
2024-03-21 08:31:29 +09:00
Kohya S
a35e7bd595
Merge pull request #1200 from BootsofLagrangian/deep-speed
...
Fix sdxl_train.py in deepspeed branch
2024-03-20 21:32:35 +09:00
BootsofLagrangian
d9456020d7
Fix most of ZeRO stage uses optimizer partitioning
...
- we have to prepare optimizer and ds_model at the same time.
- pull/1139#issuecomment-1986790007
Signed-off-by: BootsofLagrangian <hard2251@yonsei.ac.kr >
2024-03-20 20:52:59 +09:00
Kohya S
fbb98f144e
Merge branch 'dev' into deep-speed
2024-03-20 18:15:26 +09:00
Kohya S
9b6b39f204
Merge branch 'dev' into masked-loss
2024-03-20 18:14:36 +09:00
Kohya S
855add067b
update option help and readme
2024-03-20 18:14:05 +09:00
Kohya S
bf6cd4b9da
Merge pull request #1168 from gesen2egee/save_state_on_train_end
...
Save state on train end
2024-03-20 18:02:13 +09:00
Kohya S
3b0db0f17f
update readme
2024-03-20 17:45:35 +09:00
Kohya S
119cc99fb0
Merge pull request #1167 from Horizon1704/patch-1
...
Add "encoding='utf-8'" for --config_file
2024-03-20 17:39:08 +09:00
Kohya S
5f6196e4c7
update readme
2024-03-20 16:35:23 +09:00
Victor Espinoza-Guerra
46331a9e8e
English Translation of config_README-ja.md ( #1175 )
...
* Add files via upload
Creating template to work on.
* Update config_README-en.md
Total Conversion from Japanese to English.
* Update config_README-en.md
* Update config_README-en.md
* Update config_README-en.md
2024-03-20 16:31:01 +09:00
Kohya S
cf09c6aa9f
Merge pull request #1177 from KohakuBlueleaf/random-strength-noise
...
Random strength for Noise Offset and input perturbation noise
2024-03-20 16:17:16 +09:00
Kohya S
80dbbf5e48
tagger now stores model under repo_id subdir
2024-03-20 16:14:57 +09:00
Kohya S
7da41be281
Merge pull request #1192 from sdbds/main
...
Add WDV3 support
2024-03-20 15:49:55 +09:00
Kohya S
e281e867e6
Merge branch 'main' into dev
2024-03-20 15:49:08 +09:00
青龍聖者@bdsqlsz
6c51c971d1
fix typo
2024-03-20 09:35:21 +08:00
青龍聖者@bdsqlsz
a71c35ccd9
Update requirements.txt
2024-03-18 22:31:59 +08:00
青龍聖者@bdsqlsz
5410a8c79b
Update requirements.txt
2024-03-18 22:31:00 +08:00
青龍聖者@bdsqlsz
a7dff592d3
Update tag_images_by_wd14_tagger.py
...
add WDV3
2024-03-18 22:29:05 +08:00
Kohya S
f9317052ed
update readme for timestep embs bug
2024-03-18 08:53:23 +09:00
Kohya S
86e40fabbc
Merge branch 'dev' into deep-speed
2024-03-17 19:30:42 +09:00
Kohya S
3419c3de0d
common masked loss func, apply to all training script
2024-03-17 19:30:20 +09:00
Kohya S
7081a0cf0f
extension of src image could be different than target image
2024-03-17 18:09:15 +09:00
Kohya S
0ef4fe70f0
Merge branch 'dev' into masked-loss
2024-03-17 11:18:18 +09:00
Kohya S
443f02942c
fix doc
2024-03-15 21:35:14 +09:00
Kohya S
0a8ec5224e
Merge branch 'main' into dev
2024-03-15 21:33:07 +09:00
Kohya S
6b1520a46b
Merge pull request #1187 from kohya-ss/fix-timeemb
...
fix sdxl timestep embedding
2024-03-15 21:17:13 +09:00
Kohya S
f811b115ba
fix sdxl timestep embedding
2024-03-15 21:05:00 +09:00
kblueleaf
53954a1e2e
use correct settings for parser
2024-03-13 18:21:49 +08:00
kblueleaf
86399407b2
random noise_offset strength
2024-03-13 18:21:49 +08:00
kblueleaf
948029fe61
random ip_noise_gamma strength
2024-03-13 18:21:49 +08:00
Kohya S
97524f1bda
Merge branch 'dev' into deep-speed
2024-03-12 20:41:41 +09:00
Kohya S
74c266a597
Merge branch 'dev' into masked-loss
2024-03-12 20:40:57 +09:00
gesen2egee
d282c45002
Update train_network.py
2024-03-11 23:56:09 +08:00
gesen2egee
095b8035e6
save state on train end
2024-03-10 23:33:38 +08:00
Horizon1704
124ec45876
Add "encoding='utf-8'"
2024-03-10 22:53:05 +08:00
Kohya S
14c9372a38
add doc about Colab/rich issue
2024-03-03 21:47:37 +09:00
Kohya S
a9b64ffba8
support masked loss in sdxl_train ref #589
2024-02-27 21:43:55 +09:00
Kohya S
e3ccf8fbf7
make deepspeed_utils
2024-02-27 21:30:46 +09:00
Kohya S
0e4a5738df
Merge pull request #1101 from BootsofLagrangian/deepspeed
...
support deepspeed
2024-02-27 18:59:00 +09:00
