Commit Graph

401 Commits

Author SHA1 Message Date
mio
68cd874bb6 Append sys path for import_module
This will be better if we run the scripts we do not run the training script from the current directory.  This is reasonable as some other projects will use this as a subfolder, such as https://github.com/ddPn08/kohya-sd-scripts-webui. I can not run the script without adding this.
2023-03-10 18:29:34 +08:00
Kohya S
dd05d99efd Merge pull request #272 from kohya-ss/dev
support conv2d-3x3, update documents etc
2023-03-09 21:54:41 +09:00
Kohya S
cf2bc437ec update readme 2023-03-09 21:51:22 +09:00
Kohya S
aa317d4f57 Merge branch 'main' into dev 2023-03-09 20:56:54 +09:00
Kohya S
51249b1ba0 support conv2d 3x3 LoRA 2023-03-09 20:56:33 +09:00
Kohya S
e7051d427c fix default conv alpha to 1 2023-03-09 20:26:14 +09:00
Kohya S
b885c6f9d2 disable annoying warning in CLIP loading 2023-03-09 20:25:21 +09:00
Kohya S
ad443e172a fix samle gen failed if use templates 2023-03-09 20:24:53 +09:00
Kohya S
c4b4d1cb40 fix LoRA always expanded to Conv2d-3x3 2023-03-09 08:47:13 +09:00
Kohya S
3ce846525b set minimum metadata even with no_metadata 2023-03-08 21:19:12 +09:00
Kohya S
8929bf31d9 sample gen h/w to div by 8, fix in steps=epoch 2023-03-08 21:18:28 +09:00
Kohya S
7b0af4f382 Add comment about sample generation v0.4.5 2023-03-07 12:54:33 +09:00
Kohya S
225c533279 accept empty caption #258 2023-03-07 08:23:34 +09:00
Kohya S
8d5ba29363 free pipe and cache after sample gen #260 2023-03-07 08:06:36 +09:00
Kohya S
19386df6e9 expand LoRA to all Conv2d 2023-03-06 22:03:09 +09:00
Kohya S
5bb571ccc0 Merge branch 'main' into dev 2023-03-06 17:49:47 +09:00
Kohya S
0cacefc749 Merge pull request #261 from camenduru/main
metadata |= to metadata.update

Thank you! I forget to fix this.
2023-03-06 17:49:03 +09:00
Kohya S
573aa8b5e7 Merge branch 'dev' of https://github.com/kohya-ss/sd-scripts into dev 2023-03-06 17:48:27 +09:00
Kohya S
c2a8290965 Merge pull request #255 from Isotr0py/dev
Add network_args to metadata if using another network
2023-03-06 17:46:45 +09:00
camenduru
772ee52ef2 metadata |= to metadata.update 2023-03-06 00:31:28 +03:00
Kohya S
46aee85d2a re2-fix to support python 3.8/3.9 2023-03-05 23:27:16 +09:00
Kohya S
2ae33db83f re-fix to support python 3.8/3.9 2023-03-05 22:35:32 +09:00
Kohya S
1c00764d01 updating documents 2023-03-05 22:32:26 +09:00
Kohya S
2ba6d74af8 Merge branch 'main' into dev 2023-03-05 21:29:40 +09:00
Kohya S
dd39e5d944 hope to support python 3.8/3.9 2023-03-05 20:04:18 +09:00
Kohya S
db8c79c463 Update documentation 2023-03-05 19:51:05 +09:00
Kohya S
2b6e9d83fa Merge branch 'main' into dev 2023-03-05 09:55:41 +09:00
Kohya S
4d9292e50a add traininig (dataset preparation) doc 2023-03-04 22:07:09 +09:00
Kohya S
fe4f4446f1 Add region control for LoRA 2023-03-04 18:03:11 +09:00
Isotr0py
4396350271 Add network_args to meta if using another network 2023-03-04 13:59:22 +08:00
Kohya S
45945f698a Merge pull request #246 from kohya-ss/dev
add dataset config file, generating images in training etc.
2023-03-02 23:27:55 +09:00
Kohya S
08fcc7b31c update README 2023-03-02 23:20:45 +09:00
Kohya S
74f317abf8 update README 2023-03-02 22:16:20 +09:00
Kohya S
5602e0e5fc change dataset config option to dataset_config 2023-03-02 21:51:58 +09:00
Kohya S
2d2407410e show index in caching latents 2023-03-02 21:32:02 +09:00
Kohya S
09f575fd4d merge image_dir for metadata editor 2023-03-02 21:17:25 +09:00
Kohya S
859f8361bb minor fix in token shuffling 2023-03-02 20:31:07 +09:00
Kohya S
c3024be8bf add help for keep_tokens 2023-03-02 20:28:42 +09:00
Kohya S
7e1aa5f4d6 keep tag_frequency for each dataset 2023-03-02 20:27:22 +09:00
Kohya S
83bfb54f20 fix num_repeats not working in DB classic dataset 2023-03-02 19:01:22 +09:00
Kohya S
e9f37c4049 do not save image_dir to metadata if None 2023-03-01 23:37:17 +09:00
Kohya S
c95943b663 merge tag frequence for metadata editor 2023-03-01 22:10:43 +09:00
Kohya S
04af36e7e2 strip tag, fix tag frequency count 2023-03-01 22:10:15 +09:00
Kohya S
d1d7d432e9 print dataset index in making buckets 2023-03-01 21:30:12 +09:00
Kohya S
089a63c573 shuffle at debug_dataset 2023-03-01 21:12:33 +09:00
Kohya S
ed19a92bbe fix typos 2023-03-01 21:01:10 +09:00
fur0ut0
8abb8645ae add detail dataset config feature by extra config file (#227)
* add config file schema

* change config file specification

* refactor config utility

* unify batch_size to train_batch_size

* fix indent size

* use batch_size instead of train_batch_size

* make cache_latents configurable on subset

* rename options
* bucket_repo_range
* shuffle_keep_tokens

* update readme

* revert to min_bucket_reso & max_bucket_reso

* use subset structure in dataset

* format import lines

* split mode specific options

* use only valid subset

* change valid subsets name

* manage multiple datasets by dataset group

* update config file sanitizer

* prune redundant validation

* add comments

* update type annotation

* rename json_file_name to metadata_file

* ignore when image dir is invalid

* fix tag shuffle and dropout

* ignore duplicated subset

* add method to check latent cachability

* fix format

* fix bug

* update caption dropout default values

* update annotation

* fix bug

* add option to enable bucket shuffle across dataset

* update blueprint generate function

* use blueprint generator for dataset initialization

* delete duplicated function

* update config readme

* delete debug print

* print dataset and subset info as info

* enable bucket_shuffle_across_dataset option

* update config readme for clarification

* compensate quotes for string option example

* fix bug of bad usage of join

* conserve trained metadata backward compatibility

* enable shuffle in data loader by default

* delete resolved TODO

* add comment for image data handling

* fix reference bug

* fix undefined variable bug

* prevent raise overwriting

* assert image_dir and metadata_file validity

* add debug message for ignoring subset

* fix inconsistent import statement

* loosen too strict validation on float value

* sanitize argument parser separately

* make image_dir optional for fine tuning dataset

* fix import

* fix trailing characters in print

* parse flexible dataset config deterministically

* use relative import

* print supplementary message for parsing error

* add note about different methods

* add note of benefit of separate dataset

* add error example

* add note for english readme plan

---------

Co-authored-by: Kohya S <52813779+kohya-ss@users.noreply.github.com>
2023-03-01 20:58:08 +09:00
Kohya S
82707654ad support sample generation in TI training 2023-02-28 22:05:31 +09:00
Kohya S
57c565c402 support sample generation in TI training 2023-02-28 22:05:10 +09:00
Kohya S
dd523c94ff sample images in training (not fully tested) 2023-02-27 17:48:32 +09:00