2060 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
Kohya S	3c8193f642	revert lora+ for lora_fa	2024-05-12 17:00:51 +09:00
Kohya S	c6a437054a	Merge branch 'dev' into lora-plus	2024-05-12 16:18:57 +09:00
Kohya S	1ffc0b330a	fix typo	2024-05-12 16:18:43 +09:00
Kohya S	e01e148705	Merge branch 'dev' into lora-plus	2024-05-12 16:17:52 +09:00
Kohya S	e9f3a622f4	Merge branch 'dev' into lora-plus	2024-05-12 16:17:27 +09:00
Kohya S	7983d3db5f	Merge pull request #1319 from kohya-ss/fused-backward-pass ... Fused backward pass	2024-05-12 15:09:39 +09:00
Kohya S	bee8cee7e8	update README for fused optimizer	2024-05-12 15:08:52 +09:00
Kohya S	f3d2cf22ff	update README for fused optimizer	2024-05-12 15:03:02 +09:00
Kohya S	6dbc23cf63	Merge branch 'dev' into fused-backward-pass	2024-05-12 14:21:56 +09:00
Kohya S	c1ba0b4356	update readme	2024-05-12 14:21:10 +09:00
Kohya S	607e041f3d	chore: Refactor optimizer group	2024-05-12 14:16:41 +09:00
AngelBottomless	793aeb94da	fix get_trainable_params in controlnet-llite training	2024-05-07 18:21:31 +09:00
Kohya S	b56d5f7801	add experimental option to fuse params to optimizer groups	2024-05-06 21:35:39 +09:00
Kohya S	017b82ebe3	update help message for fused_backward_pass	2024-05-06 15:05:42 +09:00
Kohya S	2a359e0a41	Merge pull request #1259 from 2kpr/fused_backward_pass ... Adafactor fused backward pass and optimizer step, lowers SDXL (@ 1024 resolution) VRAM usage to BF16(10GB)/FP32(16.4GB)	2024-05-06 15:01:56 +09:00
Kohya S	3fd8cdc55d	fix dylora loraplus	2024-05-06 14:03:19 +09:00
Kohya S	7fe81502d0	update loraplus on dylora/lofa_fa	2024-05-06 11:09:32 +09:00
Kohya S	52e64c69cf	add debug log	2024-05-04 18:43:52 +09:00
Kohya S	58c2d856ae	support block dim/lr for sdxl	2024-05-03 22:18:20 +09:00
Dave Lage	8db0cadcee	Add caption_separator to output for subset	2024-05-02 18:08:28 -04:00
Dave Lage	dbb7bb288e	Fix caption_separator missing in subset schema	2024-05-02 17:39:35 -04:00
Kohya S	969f82ab47	move loraplus args from args to network_args, simplify log lr desc	2024-04-29 20:04:25 +09:00
Kohya S	834445a1d6	Merge pull request #1233 from rockerBOO/lora-plus ... Add LoRA+ support	2024-04-29 18:05:12 +09:00
frodo821	fdbb03c360	removed unnecessary torch import on line 115 ... as per #1290	2024-04-23 14:29:05 +09:00
Cauldrath	040e26ff1d	Regenerate failed file ... If a latent file fails to load, print out the path and the error, then return false to regenerate it	2024-04-21 13:46:31 -04:00
Kohya S	0540c33aca	pop weights if available #1247	2024-04-21 17:45:29 +09:00
Kohya S	52652cba1a	disable main process check for deepspeed #1247	2024-04-21 17:41:32 +09:00
青龍聖者@bdsqlsz	5cb145d13b	Update train_util.py	2024-04-20 21:56:24 +08:00
Maatra	b886d0a359	Cleaned typing to be in line with accelerate hyperparameters type resctrictions	2024-04-20 14:36:47 +01:00
青龍聖者@bdsqlsz	4477116a64	fix train controlnet	2024-04-20 21:26:09 +08:00
Maatra	2c9db5d9f2	passing filtered hyperparameters to accelerate	2024-04-20 14:11:43 +01:00
Cauldrath	fc374375de	Allow negative learning rate ... This can be used to train away from a group of images you don't want As this moves the model away from a point instead of towards it, the change in the model is unbounded So, don't set it too low. -4e-7 seemed to work well.	2024-04-18 23:29:01 -04:00
Cauldrath	feefcf256e	Display name of error latent file ... When trying to load stored latents, if an error occurs, this change will tell you what file failed to load Currently it will just tell you that something failed without telling you which file	2024-04-18 23:15:36 -04:00
Zovjsra	64916a35b2	add disable_mmap to args	2024-04-16 16:40:08 +08:00
2kpr	4f203ce40d	Fused backward pass	2024-04-14 09:56:58 -05:00
rockerBOO	68467bdf4d	Fix unset or invalid LR from making a param_group	2024-04-11 17:33:19 -04:00
gesen2egee	fde8026c2d	Update config_util.py	2024-04-11 11:29:26 +08:00
gesen2egee	89ad69b6a0	Update train_util.py	2024-04-11 08:42:31 +08:00
gesen2egee	459b12539b	Update config_util.py	2024-04-11 01:52:14 +08:00
gesen2egee	3b251b758d	Update config_util.py	2024-04-11 01:50:32 +08:00
gesen2egee	229c5a38ef	Update train_util.py	2024-04-11 01:45:49 +08:00
gesen2egee	36d4023431	Update config_util.py	2024-04-11 01:39:17 +08:00
gesen2egee	086f6000f2	Merge branch 'main' into val	2024-04-11 01:14:46 +08:00
rockerBOO	75833e84a1	Fix default LR, Add overall LoRA+ ratio, Add log ... `--loraplus_ratio` added for both TE and UNet Add log for lora+	2024-04-08 19:23:02 -04:00
Kohya S	71e2c91330	Merge pull request #1230 from kohya-ss/dependabot/github_actions/crate-ci/typos-1.19.0 ... Bump crate-ci/typos from 1.17.2 to 1.19.0	2024-04-07 21:14:18 +09:00
Kohya S	bfb352bc43	change huber_schedule from exponential to snr v0.8.7	2024-04-07 21:07:52 +09:00
Kohya S	c973b29da4	update readme	2024-04-07 20:51:52 +09:00
Kohya S	683f3d6ab3	Merge pull request #1212 from kohya-ss/dev ... Version 0.8.6 v0.8.6	2024-04-07 20:42:41 +09:00
Kohya S	dfa30790a9	update readme	2024-04-07 20:34:26 +09:00
Kohya S	d30ebb205c	update readme, add metadata for network module	2024-04-07 14:58:17 +09:00