Kohya S
4ad8e75291
fix to work with dim>320
2023-03-10 21:10:22 +09:00
mgz-dev
4a4450d6b6
make new_rank limit max rank, fix zero matrices
...
-new_rank arg changed to limit the max rank of any layer.
-added logic to make sure zero-ed layers do not create large lora dim
2023-03-04 03:10:04 -06:00
mgz-dev
214ed092f2
add support to extract lora with resnet and 2d blocks
...
Modified resize script so support different types of LoRA networks (refer to Kohaku-Blueleaf module implementation for structure).
2023-03-04 02:01:10 -06:00
mgz-dev
80be6fa130
refactor and bug fix for too large sv_ratio
...
- code refactor to be able to re-use same function for dynamic extract lora
- remove clamp
- fix issue where if sv_ratio is too high index goes out of bounds
2023-03-03 23:32:46 -06:00
mgz-dev
52ca6c515c
add options to resize based off frobenius norm or cumulative sum
2023-03-01 13:35:24 -06:00
mgz-dev
efe4c98341
Enable ability to resize lora dim based off ratios
2023-02-28 14:55:15 -06:00
Kohya S
3c29784825
Add ja comment
2023-02-14 20:55:20 +09:00
michaelgzhang
b35b053b8d
clean up print formatting
2023-02-11 03:14:43 -06:00
michaelgzhang
55521eece0
add verbosity option for resize_lora.py
...
add --verbose flag to print additional statistics during resize_lora function
correct some parameter references in resize_lora_model function
2023-02-11 02:38:13 -06:00
Kohya S
c7406d6b27
keep metadata when resizing
2023-02-10 22:55:00 +09:00
Yuta Hayashibe
19c2752e87
Fix a typo
2023-02-04 21:18:34 +09:00
Kohya S
8cbd3f4fca
Add device option to calculate on GPU
2023-02-04 20:36:10 +09:00
michaelgzhang
b7e7ee387a
resize lora rank
...
add script which can be used to convert higher rank lora to approximate lower rank lora using svd
2023-02-03 01:00:02 -06:00
