jvkap
a0e05fa291
Update resize_lora.py
2023-09-11 11:41:33 -03:00
jvkap
e33c007cd0
Update resize_lora.py
2023-09-11 11:29:06 -03:00
Kohya S
af08c56ce0
remove unnecessary newline
2023-05-11 21:20:18 +09:00
Fair
b08154dc36
fix print "saving" and "epoch" in newline
2023-05-07 02:51:01 +08:00
mgz-dev
c9b157b536
update resize_lora.py (fix out of bounds and index)
...
Fix error where index may go out of bounds when using certain dynamic parameters.
Fix index and rank issue (previously some parts of code was incorrectly using python index position rather than rank, which is -1 dim).
2023-03-25 19:56:14 -05:00
Kohya S
193674e16c
fix to support dynamic rank/alpha
2023-03-21 21:59:51 +09:00
Robert Smieja
eb66e5ebac
Extract parser setup to helper function
...
- Allows users who `import` the scripts to examine the parser programmatically
2023-03-20 00:06:47 -04:00
Kohya S
b177460807
restore comment
2023-03-10 22:02:17 +09:00
Kohya S
4ad8e75291
fix to work with dim>320
2023-03-10 21:10:22 +09:00
mgz-dev
4a4450d6b6
make new_rank limit max rank, fix zero matrices
...
-new_rank arg changed to limit the max rank of any layer.
-added logic to make sure zero-ed layers do not create large lora dim
2023-03-04 03:10:04 -06:00
mgz-dev
214ed092f2
add support to extract lora with resnet and 2d blocks
...
Modified resize script so support different types of LoRA networks (refer to Kohaku-Blueleaf module implementation for structure).
2023-03-04 02:01:10 -06:00
mgz-dev
80be6fa130
refactor and bug fix for too large sv_ratio
...
- code refactor to be able to re-use same function for dynamic extract lora
- remove clamp
- fix issue where if sv_ratio is too high index goes out of bounds
2023-03-03 23:32:46 -06:00
mgz-dev
52ca6c515c
add options to resize based off frobenius norm or cumulative sum
2023-03-01 13:35:24 -06:00
mgz-dev
efe4c98341
Enable ability to resize lora dim based off ratios
2023-02-28 14:55:15 -06:00
Kohya S
3c29784825
Add ja comment
2023-02-14 20:55:20 +09:00
michaelgzhang
b35b053b8d
clean up print formatting
2023-02-11 03:14:43 -06:00
michaelgzhang
55521eece0
add verbosity option for resize_lora.py
...
add --verbose flag to print additional statistics during resize_lora function
correct some parameter references in resize_lora_model function
2023-02-11 02:38:13 -06:00
Kohya S
c7406d6b27
keep metadata when resizing
2023-02-10 22:55:00 +09:00
Yuta Hayashibe
19c2752e87
Fix a typo
2023-02-04 21:18:34 +09:00
Kohya S
8cbd3f4fca
Add device option to calculate on GPU
2023-02-04 20:36:10 +09:00
michaelgzhang
b7e7ee387a
resize lora rank
...
add script which can be used to convert higher rank lora to approximate lower rank lora using svd
2023-02-03 01:00:02 -06:00
