202 Commits

Author	SHA1	Message	Date
Kohya S	d10ff62a78	support individual LR for CLIP-L/T5XXL	2024-09-10 20:32:09 +09:00
Kohya S	90ed2dfb52	feat: Add support for merging CLIP-L and T5XXL LoRA models	2024-09-05 08:39:29 +09:00
Kohya S	56cb2fc885	support T5XXL LoRA, reduce peak memory usage #1560	2024-09-04 23:15:27 +09:00
Kohya S	b65ae9b439	T5XXL LoRA training, fp8 T5XXL support	2024-09-04 21:33:17 +09:00
Kohya S	3be712e3e0	feat: Update direct loading fp8 ckpt for LoRA training	2024-08-27 21:40:02 +09:00
Kohya S	0087a46e14	FLUX.1 LoRA supports CLIP-L	2024-08-27 19:59:40 +09:00
Kohya S	5639c2adc0	fix typo	2024-08-24 16:37:49 +09:00
Kohya S	cf689e7aa6	feat: Add option to split projection layers and apply LoRA	2024-08-24 16:35:43 +09:00
Kohya S	99744af53a	Merge branch 'dev' into sd3	2024-08-22 21:34:24 +09:00
Kohya S	afb971f9c3	fix SD1.5 LoRA extraction #1490	2024-08-22 21:33:15 +09:00
Kohya S	bf9f798985	chore: fix typos, remove debug print	2024-08-22 19:59:38 +09:00
Kohya S	b0a980844a	added a script to extract LoRA	2024-08-22 19:57:29 +09:00
Kohya S	2b07a92c8d	Fix error in applying mask in Attention and add LoRA converter script	2024-08-21 12:30:23 +09:00
Kohya S	e17c42cb0d	Add BFL/Diffusers LoRA converter #1467 #1458 #1483	2024-08-21 12:28:45 +09:00
Kohya S	dbed5126bd	chore: formatting	2024-08-20 19:33:47 +09:00
Kohya S	9381332020	revert merge function add add option to use new func	2024-08-20 19:32:26 +09:00
Kohya S	6f6faf9b5a	fix to work with ai-toolkit LoRA	2024-08-20 19:16:25 +09:00
exveria1015	7e688913ae	fix: Flux の LoRA マージ機能を修正	2024-08-18 12:38:05 +09:00
Kohya S	e45d3f8634	add merge LoRA script	2024-08-16 22:19:21 +09:00
Kohya S	56d7651f08	add experimental split mode for FLUX	2024-08-13 22:28:39 +09:00
Kohya S	8a0f12dde8	update FLUX LoRA training	2024-08-10 23:42:05 +09:00
Kohya S	358f13f2c9	fix alpha is ignored	2024-08-10 14:03:59 +09:00
Kohya S	36b2e6fc28	add FLUX.1 LoRA training	2024-08-09 22:56:48 +09:00
Kohya S	1a104dc75e	make forward/backward pathes same ref #1363	2024-06-09 19:26:36 +09:00
rockerBOO	00513b9b70	Add LoRA+ LR Ratio info message to logger	2024-05-23 22:27:12 -04:00
Kohya S	146edce693	support Diffusers' based SDXL LoRA key for inference	2024-05-18 11:05:04 +09:00
Kohya S	16677da0d9	fix create_network_from_weights doesn't work	2024-05-12 22:15:07 +09:00
Kohya S	44190416c6	update docs etc.	2024-05-12 17:01:20 +09:00
Kohya S	3c8193f642	revert lora+ for lora_fa	2024-05-12 17:00:51 +09:00
Kohya S	3fd8cdc55d	fix dylora loraplus	2024-05-06 14:03:19 +09:00
Kohya S	7fe81502d0	update loraplus on dylora/lofa_fa	2024-05-06 11:09:32 +09:00
Kohya S	58c2d856ae	support block dim/lr for sdxl	2024-05-03 22:18:20 +09:00
Kohya S	969f82ab47	move loraplus args from args to network_args, simplify log lr desc	2024-04-29 20:04:25 +09:00
Kohya S	834445a1d6	Merge pull request #1233 from rockerBOO/lora-plus ... Add LoRA+ support	2024-04-29 18:05:12 +09:00
rockerBOO	68467bdf4d	Fix unset or invalid LR from making a param_group	2024-04-11 17:33:19 -04:00
rockerBOO	75833e84a1	Fix default LR, Add overall LoRA+ ratio, Add log ... `--loraplus_ratio` added for both TE and UNet Add log for lora+	2024-04-08 19:23:02 -04:00
rockerBOO	1933ab4b48	Fix default_lr being applied	2024-04-03 12:46:34 -04:00
Kohya S	b748b48dbb	fix attention couple+deep shink cause error in some reso	2024-04-03 12:43:08 +09:00
rockerBOO	c7691607ea	Add LoRA-FA for LoRA+	2024-04-01 15:43:04 -04:00
rockerBOO	f99fe281cb	Add LoRA+ support	2024-04-01 15:38:26 -04:00
Yuta Hayashibe	5d5f39b6e6	Replaced print with logger	2024-02-25 01:24:11 +09:00
Kohya S	8b7c14246a	some log output to print	2024-02-24 20:50:00 +09:00
Kohya S	52b3799989	fix format, add new conv rank to metadata comment	2024-02-24 20:49:41 +09:00
Kohya S	0e703608f9	Merge branch 'dev' into resize_lora-add-rank-for-conv	2024-02-24 20:09:38 +09:00
Kohya S	fb9110bac1	format by black	2024-02-24 20:00:57 +09:00
Kohya S	86279c8855	Merge branch 'dev' into DyLoRA-xl	2024-02-24 19:18:36 +09:00
tamlog06	a6f1ed2e14	fix dylora create_modules error	2024-02-18 13:20:47 +00:00
Kohya S	baa0e97ced	Merge branch 'dev' into dev_device_support	2024-02-17 11:54:07 +09:00
Kohya S	cbe9c5dc06	supprt deep shink with regional lora, add prompter module	2024-02-12 14:17:27 +09:00
Kohya S	358ca205a3	Merge branch 'dev' into dev_device_support	2024-02-12 13:01:54 +09:00