Commit Graph

394 Commits

Author SHA1 Message Date
Kohya S
b885c6f9d2 disable annoying warning in CLIP loading 2023-03-09 20:25:21 +09:00
Kohya S
ad443e172a fix samle gen failed if use templates 2023-03-09 20:24:53 +09:00
Kohya S
c4b4d1cb40 fix LoRA always expanded to Conv2d-3x3 2023-03-09 08:47:13 +09:00
Kohya S
3ce846525b set minimum metadata even with no_metadata 2023-03-08 21:19:12 +09:00
Kohya S
8929bf31d9 sample gen h/w to div by 8, fix in steps=epoch 2023-03-08 21:18:28 +09:00
Kohya S
225c533279 accept empty caption #258 2023-03-07 08:23:34 +09:00
Kohya S
8d5ba29363 free pipe and cache after sample gen #260 2023-03-07 08:06:36 +09:00
Kohya S
19386df6e9 expand LoRA to all Conv2d 2023-03-06 22:03:09 +09:00
Kohya S
5bb571ccc0 Merge branch 'main' into dev 2023-03-06 17:49:47 +09:00
Kohya S
0cacefc749 Merge pull request #261 from camenduru/main
metadata |= to metadata.update

Thank you! I forget to fix this.
2023-03-06 17:49:03 +09:00
Kohya S
573aa8b5e7 Merge branch 'dev' of https://github.com/kohya-ss/sd-scripts into dev 2023-03-06 17:48:27 +09:00
Kohya S
c2a8290965 Merge pull request #255 from Isotr0py/dev
Add network_args to metadata if using another network
2023-03-06 17:46:45 +09:00
camenduru
772ee52ef2 metadata |= to metadata.update 2023-03-06 00:31:28 +03:00
Kohya S
46aee85d2a re2-fix to support python 3.8/3.9 2023-03-05 23:27:16 +09:00
Kohya S
2ae33db83f re-fix to support python 3.8/3.9 2023-03-05 22:35:32 +09:00
Kohya S
1c00764d01 updating documents 2023-03-05 22:32:26 +09:00
Kohya S
2ba6d74af8 Merge branch 'main' into dev 2023-03-05 21:29:40 +09:00
Kohya S
dd39e5d944 hope to support python 3.8/3.9 2023-03-05 20:04:18 +09:00
Kohya S
db8c79c463 Update documentation 2023-03-05 19:51:05 +09:00
Kohya S
2b6e9d83fa Merge branch 'main' into dev 2023-03-05 09:55:41 +09:00
Kohya S
4d9292e50a add traininig (dataset preparation) doc 2023-03-04 22:07:09 +09:00
Kohya S
fe4f4446f1 Add region control for LoRA 2023-03-04 18:03:11 +09:00
Isotr0py
4396350271 Add network_args to meta if using another network 2023-03-04 13:59:22 +08:00
Kohya S
45945f698a Merge pull request #246 from kohya-ss/dev
add dataset config file, generating images in training etc.
2023-03-02 23:27:55 +09:00
Kohya S
08fcc7b31c update README 2023-03-02 23:20:45 +09:00
Kohya S
74f317abf8 update README 2023-03-02 22:16:20 +09:00
Kohya S
5602e0e5fc change dataset config option to dataset_config 2023-03-02 21:51:58 +09:00
Kohya S
2d2407410e show index in caching latents 2023-03-02 21:32:02 +09:00
Kohya S
09f575fd4d merge image_dir for metadata editor 2023-03-02 21:17:25 +09:00
Kohya S
859f8361bb minor fix in token shuffling 2023-03-02 20:31:07 +09:00
Kohya S
c3024be8bf add help for keep_tokens 2023-03-02 20:28:42 +09:00
Kohya S
7e1aa5f4d6 keep tag_frequency for each dataset 2023-03-02 20:27:22 +09:00
Kohya S
83bfb54f20 fix num_repeats not working in DB classic dataset 2023-03-02 19:01:22 +09:00
Kohya S
e9f37c4049 do not save image_dir to metadata if None 2023-03-01 23:37:17 +09:00
Kohya S
c95943b663 merge tag frequence for metadata editor 2023-03-01 22:10:43 +09:00
Kohya S
04af36e7e2 strip tag, fix tag frequency count 2023-03-01 22:10:15 +09:00
Kohya S
d1d7d432e9 print dataset index in making buckets 2023-03-01 21:30:12 +09:00
Kohya S
089a63c573 shuffle at debug_dataset 2023-03-01 21:12:33 +09:00
Kohya S
ed19a92bbe fix typos 2023-03-01 21:01:10 +09:00
fur0ut0
8abb8645ae add detail dataset config feature by extra config file (#227)
* add config file schema

* change config file specification

* refactor config utility

* unify batch_size to train_batch_size

* fix indent size

* use batch_size instead of train_batch_size

* make cache_latents configurable on subset

* rename options
* bucket_repo_range
* shuffle_keep_tokens

* update readme

* revert to min_bucket_reso & max_bucket_reso

* use subset structure in dataset

* format import lines

* split mode specific options

* use only valid subset

* change valid subsets name

* manage multiple datasets by dataset group

* update config file sanitizer

* prune redundant validation

* add comments

* update type annotation

* rename json_file_name to metadata_file

* ignore when image dir is invalid

* fix tag shuffle and dropout

* ignore duplicated subset

* add method to check latent cachability

* fix format

* fix bug

* update caption dropout default values

* update annotation

* fix bug

* add option to enable bucket shuffle across dataset

* update blueprint generate function

* use blueprint generator for dataset initialization

* delete duplicated function

* update config readme

* delete debug print

* print dataset and subset info as info

* enable bucket_shuffle_across_dataset option

* update config readme for clarification

* compensate quotes for string option example

* fix bug of bad usage of join

* conserve trained metadata backward compatibility

* enable shuffle in data loader by default

* delete resolved TODO

* add comment for image data handling

* fix reference bug

* fix undefined variable bug

* prevent raise overwriting

* assert image_dir and metadata_file validity

* add debug message for ignoring subset

* fix inconsistent import statement

* loosen too strict validation on float value

* sanitize argument parser separately

* make image_dir optional for fine tuning dataset

* fix import

* fix trailing characters in print

* parse flexible dataset config deterministically

* use relative import

* print supplementary message for parsing error

* add note about different methods

* add note of benefit of separate dataset

* add error example

* add note for english readme plan

---------

Co-authored-by: Kohya S <52813779+kohya-ss@users.noreply.github.com>
2023-03-01 20:58:08 +09:00
Kohya S
82707654ad support sample generation in TI training 2023-02-28 22:05:31 +09:00
Kohya S
57c565c402 support sample generation in TI training 2023-02-28 22:05:10 +09:00
Kohya S
dd523c94ff sample images in training (not fully tested) 2023-02-27 17:48:32 +09:00
Kohya S
a28f9ae7a3 support tokenizer caching for offline training/gen 2023-02-25 18:46:59 +09:00
Kohya S
9993792656 latents upscaling in highres fix, vae batch size 2023-02-25 18:17:18 +09:00
Kohya S
f0ae7eea95 Update README.md v0.4.4 2023-02-23 21:59:20 +09:00
Kohya S
b22b0a5c75 Merge pull request #223 from kohya-ss/control_net
support ControlNet
2023-02-23 21:53:05 +09:00
Kohya S
c7a13c89c7 Merge branch 'main' into control_net 2023-02-23 21:51:03 +09:00
Kohya S
39a70f10bd Merge pull request #222 from kohya-ss/dev
fix training instability issue, add metadata
2023-02-23 21:50:38 +09:00
Kohya S
a3c0e4cf44 update change history 2023-02-23 21:49:34 +09:00