155 Commits

Author	SHA1	Message	Date
Kohya S	bf3674c1db	format by black	2023-03-29 21:23:27 +09:00
mgz-dev	c9b157b536	update resize_lora.py (fix out of bounds and index) ... Fix error where index may go out of bounds when using certain dynamic parameters. Fix index and rank issue (previously some parts of code was incorrectly using python index position rather than rank, which is -1 dim).	2023-03-25 19:56:14 -05:00
Kohya S	193674e16c	fix to support dynamic rank/alpha	2023-03-21 21:59:51 +09:00
Robert Smieja	eb66e5ebac	Extract parser setup to helper function ... - Allows users who `import` the scripts to examine the parser programmatically	2023-03-20 00:06:47 -04:00
Kohya S	0b38e663fd	remove unnecessary device change	2023-03-11 08:04:28 +09:00
Kohya S	8b25929765	fix device error	2023-03-11 08:03:02 +09:00
Kohya S	b177460807	restore comment	2023-03-10 22:02:17 +09:00
Kohya S	75d1883da6	fix LoRA rank is limited to target dim	2023-03-10 21:12:15 +09:00
Kohya S	4ad8e75291	fix to work with dim>320	2023-03-10 21:10:22 +09:00
Kohya S	1932c31c66	Merge pull request #243 from mgz-dev/dynamic-dim-lora-resize ... Enable ability to resize lora dim based off sv ratios	2023-03-10 12:59:39 +09:00
Kohya S	cf2bc437ec	update readme	2023-03-09 21:51:22 +09:00
Kohya S	51249b1ba0	support conv2d 3x3 LoRA	2023-03-09 20:56:33 +09:00
Kohya S	e7051d427c	fix default conv alpha to 1	2023-03-09 20:26:14 +09:00
Kohya S	c4b4d1cb40	fix LoRA always expanded to Conv2d-3x3	2023-03-09 08:47:13 +09:00
Kohya S	19386df6e9	expand LoRA to all Conv2d	2023-03-06 22:03:09 +09:00
mgz-dev	4a4450d6b6	make new_rank limit max rank, fix zero matrices ... -new_rank arg changed to limit the max rank of any layer. -added logic to make sure zero-ed layers do not create large lora dim	2023-03-04 03:10:04 -06:00
Kohya S	fe4f4446f1	Add region control for LoRA	2023-03-04 18:03:11 +09:00
mgz-dev	214ed092f2	add support to extract lora with resnet and 2d blocks ... Modified resize script so support different types of LoRA networks (refer to Kohaku-Blueleaf module implementation for structure).	2023-03-04 02:01:10 -06:00
mgz-dev	80be6fa130	refactor and bug fix for too large sv_ratio ... - code refactor to be able to re-use same function for dynamic extract lora - remove clamp - fix issue where if sv_ratio is too high index goes out of bounds	2023-03-03 23:32:46 -06:00
mgz-dev	52ca6c515c	add options to resize based off frobenius norm or cumulative sum	2023-03-01 13:35:24 -06:00
mgz-dev	efe4c98341	Enable ability to resize lora dim based off ratios	2023-02-28 14:55:15 -06:00
Kohya S	d9184ab21c	remove LoRA-ControlNet	2023-02-23 21:01:13 +09:00
Kohya S	d94c0d70fe	support network mul from prompt	2023-02-19 18:43:35 +09:00
Kohya S	8590d5dbca	add dtype	2023-02-16 21:59:35 +09:00
Kohya S	39aa390d2b	Merge branch 'main' into control_net	2023-02-15 12:36:34 +09:00
Kohya S	3c29784825	Add ja comment	2023-02-14 20:55:20 +09:00
Kohya S	64bffe5238	remove print	2023-02-14 19:25:43 +09:00
Kohya S	cebee02698	Official weights to LoRA	2023-02-13 23:38:38 +09:00
Kohya S	bc9fc4ccee	ControlNet by LoRA	2023-02-12 22:15:23 +09:00
michaelgzhang	b35b053b8d	clean up print formatting	2023-02-11 03:14:43 -06:00
michaelgzhang	55521eece0	add verbosity option for resize_lora.py ... add --verbose flag to print additional statistics during resize_lora function correct some parameter references in resize_lora_model function	2023-02-11 02:38:13 -06:00
Kohya S	3f3502fb57	add message	2023-02-11 13:20:58 +09:00
Kohya S	9239fefa52	add lora interrogator with text encoder	2023-02-11 13:15:57 +09:00
Kohya S	8d86f58174	add merge script with svd	2023-02-10 22:55:33 +09:00
Kohya S	c7406d6b27	keep metadata when resizing	2023-02-10 22:55:00 +09:00
Kohya S	d2da3c4236	support for models with different alphas	2023-02-10 22:54:35 +09:00
Kohya S	b3020db63f	support python 3.8	2023-02-07 22:29:12 +09:00
Yuta Hayashibe	19c2752e87	Fix a typo	2023-02-04 21:18:34 +09:00
Kohya S	8cbd3f4fca	Add device option to calculate on GPU	2023-02-04 20:36:10 +09:00
michaelgzhang	b7e7ee387a	resize lora rank ... add script which can be used to convert higher rank lora to approximate lower rank lora using svd	2023-02-03 01:00:02 -06:00
Kohya S	e6bad080cb	Merge pull request #102 from space-nuko/precalculate-hashes ... Precalculate .safetensors model hashes after training	2023-01-24 19:03:45 +09:00
Kohya S	bf3a13bb4e	Fix error for loading bf16 weights	2023-01-24 18:57:21 +09:00
space-nuko	f7fbdc4b2a	Precalculate .safetensors model hashes after training	2023-01-23 17:21:04 -08:00
Kohya S	ddfe94b33b	Update for alpha value	2023-01-22 21:33:35 +09:00
Kohya S	1bfcf164f1	Merge branch 'main' into dev	2023-01-22 11:26:18 +09:00
Kohya S	6e279730cf	Fix weights checking script to use float32	2023-01-22 10:44:29 +09:00
Kohya S	b4636d4185	Add scaling alpha for LoRA	2023-01-21 20:37:34 +09:00
Kohya S	943eae1211	Add LoRA weights checking script	2023-01-19 22:04:16 +09:00
Kohya S	eba142ccb2	do not save metadata in .pt/.ckpt	2023-01-12 21:52:55 +09:00
Kohya S	9fd91d26a3	Store metadata to .ckpt as value of state dict	2023-01-12 10:54:21 +09:00