* feat: Add LoHa/LoKr network support for SDXL and Anima
- networks/network_base.py: shared AdditionalNetwork base class with architecture auto-detection (SDXL/Anima) and generic module injection
- networks/loha.py: LoHa (Low-rank Hadamard Product) module with HadaWeight custom autograd, training/inference classes, and factory functions
- networks/lokr.py: LoKr (Low-rank Kronecker Product) module with factorization, training/inference classes, and factory functions
- library/lora_utils.py: extend weight merge hook to detect and merge LoHa/LoKr weights alongside standard LoRA
Linear and Conv2d 1x1 layers only; Conv2d 3x3 (Tucker decomposition) support will be added separately.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: Enhance LoHa and LoKr modules with Tucker decomposition support
- Added Tucker decomposition functionality to LoHa and LoKr modules.
- Implemented new methods for weight rebuilding using Tucker decomposition.
- Updated initialization and weight handling for Conv2d 3x3+ layers.
- Modified get_diff_weight methods to accommodate Tucker and non-Tucker modes.
- Enhanced network base to include unet_conv_target_modules for architecture detection.
* fix: rank dropout handling in LoRAModule for Conv2d and Linear layers, see #2272 for details
* doc: add dtype comment for load_safetensors_with_lora_and_fp8 function
* fix: enhance architecture detection to support InferSdxlUNet2DConditionModel for gen_img.py
* doc: update model support structure to include Lumina Image 2.0, HunyuanImage-2.1, and Anima-Preview
* doc: add documentation for LoHa and LoKr fine-tuning methods
* Update networks/network_base.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update docs/loha_lokr.md
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* fix: refactor LoHa and LoKr imports for weight merging in load_safetensors_with_lora_and_fp8 function
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
