rockerBOO
ce17007e1a
Add warning throttling for CDC shape mismatches
...
- Track warned samples in global set to prevent log spam
- Each sample only warned once per training session
- Prevents thousands of duplicate warnings during training
- Add tests to verify throttling behavior
2025-10-09 18:28:50 -04:00
rockerBOO
88af20881d
Fix: Enable gradient flow through CDC noise transformation
...
- Remove @torch.no_grad() decorator from compute_sigma_t_x()
- Gradients now properly flow through CDC transformation during training
- Add comprehensive gradient flow tests for fast/slow paths and fallback
- All 25 CDC tests passing
2025-10-09 18:28:50 -04:00
rockerBOO
0d822b2f74
Refactor: Extract CDC noise transformation to separate function
...
- Create apply_cdc_noise_transformation() for better modularity
- Implement fast path for batch processing when all shapes match
- Implement slow path for per-sample processing on shape mismatch
- Clone noise tensors in fallback path for gradient consistency
2025-10-09 18:28:50 -04:00
rockerBOO
e03200bdba
Optimize: Cache CDC shapes in memory to eliminate I/O bottleneck
...
- Cache all shapes during GammaBDataset initialization
- Eliminates file I/O on every training step (9.5M accesses/sec)
- Reduces get_shape() from file operation to dict lookup
- Memory overhead: ~126 bytes/sample (~12.6 MB per 100k images)
2025-10-09 18:28:50 -04:00
rockerBOO
f552f9a3bd
Add CDC-FM (Carré du Champ Flow Matching) support
...
Implements geometry-aware noise generation for FLUX training based on
arXiv:2510.05930v1.
2025-10-09 18:28:47 -04:00
Kohya S.
5e366acda4
Merge pull request #2003 from laolongboy/sd3-dev
...
Fix missing parameters in model conversion script
2025-10-01 21:03:12 +09:00
Kohya S
5462a6bb24
Merge branch 'dev' into sd3
2025-09-29 21:02:02 +09:00
Kohya S
63711390a0
Merge branch 'main' into dev
2025-09-29 20:56:07 +09:00
Kohya S.
206adb6438
Merge pull request #2216 from kohya-ss/fix-sdxl-textual-inversion-training-disable-mmap
...
fix: disable_mmap_safetensors not defined in SDXL TI training
2025-09-29 20:55:02 +09:00
Kohya S
60bfa97b19
fix: disable_mmap_safetensors not defined in SDXL TI training
2025-09-29 20:52:48 +09:00
Kohya S.
f0c767e0f2
Merge pull request #2213 from kohya-ss/doc-hunyuan-image-training-text-encoder-cpu-note
...
docs: enhance text encoder CPU usage instructions for HunyuanImage-2.…
2025-09-28 18:32:11 +09:00
kohya-ss
a0c26a0efa
docs: enhance text encoder CPU usage instructions for HunyuanImage-2.1 training
2025-09-28 18:21:25 +09:00
Kohya S.
67d0621313
Merge pull request #2212 from kohya-ss/fix-hunyuan-image-sample-generation
...
fix: HunyuanImage-2.1 sample generation fails
2025-09-28 18:12:04 +09:00
Kohya S
6a826d21b1
feat: add new parameters for sample image inference configuration
2025-09-28 18:06:17 +09:00
Kohya S.
4c197a538b
Merge pull request #2207 from kohya-ss/fix-flux-extract-lora-metadata-failed
...
fix: update metadata construction to include model_config for flux
2025-09-24 21:19:27 +09:00
Kohya S
4b79d73504
fix: update metadata construction to include model_config for flux
2025-09-24 21:15:37 +09:00
Kohya S.
121853ca2a
Merge pull request #2198 from kohya-ss/feat-hunyuan-image-2.1-inference
...
feat: support HunyuanImage-2.1
2025-09-23 19:11:50 +09:00
Kohya S
58df9dffa4
doc: update README with HunyuanImage-2.1 LoRA training details and requirements
2025-09-23 18:59:02 +09:00
Kohya S
31f7df3b3a
doc: add --network_train_unet_only option for HunyuanImage-2.1 training
2025-09-23 18:53:36 +09:00
Kohya S.
753c794549
Update hunyuan_image_train_network.py
...
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
2025-09-21 13:30:22 +09:00
Kohya S.
e7b89826c5
Update library/custom_offloading_utils.py
...
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
2025-09-21 13:29:58 +09:00
Kohya S
806d535ef1
fix: block-wise scaling is overwritten by per-tensor scaling
2025-09-21 13:10:41 +09:00
Kohya S
3876343fad
fix: remove print statement for guidance rescale in AdaptiveProjectedGuidance
2025-09-21 13:09:38 +09:00
Kohya S
040d976597
feat: add guidance rescale options for Adaptive Projected Guidance in inference
2025-09-21 13:03:14 +09:00
Kohya S
9621d9d637
feat: add Adaptive Projected Guidance parameters and noise rescaling
2025-09-21 12:34:40 +09:00
Kohya S
e7b8e9a778
doc: add --vae_chunk_size option for training and inference
2025-09-21 11:13:26 +09:00
Kohya S
f41e9e2b58
feat: add vae_chunk_size argument for memory-efficient VAE decoding and processing
2025-09-21 11:09:37 +09:00
Kohya S
8f20c37949
feat: add --text_encoder_cpu option to reduce VRAM usage by running text encoders on CPU for training
2025-09-20 20:26:20 +09:00
Kohya S
b090d15f7d
feat: add multi backend attention and related update for HI2.1 models and scripts
2025-09-20 19:45:33 +09:00
Kohya S
f834b2e0d4
fix: --fp8_vl to work
2025-09-18 23:46:18 +09:00
Kohya S
f6b4bdc83f
feat: block-wise fp8 quantization
2025-09-18 21:20:54 +09:00
Kohya S
2ce506e187
fix: fp8 casting not working
2025-09-18 21:20:08 +09:00
Kohya S
f5b004009e
fix: correct tensor indexing in HunyuanVAE2D class for blending and encoding functions
2025-09-17 21:54:25 +09:00
Kohya S
cbe2a9da45
feat: add conversion script for LoRA models to ComfyUI format with reverse option
2025-09-16 21:48:47 +09:00
kohya-ss
f318ddaeea
docs: update HunyuanImage-2.1 training guide with model download instructions and VRAM optimization settings (by Claude)
2025-09-16 21:18:01 +09:00
kohya-ss
39458ec0e3
fix: update default values for guidance_scale, image_size, infer_steps, and flow_shift in argument parser
2025-09-16 21:17:21 +09:00
Kohya S
2732be0b29
Merge branch 'feat-hunyuan-image-2.1-inference' of https://github.com/kohya-ss/sd-scripts into feat-hunyuan-image-2.1-inference
2025-09-14 20:49:24 +09:00
Kohya S
1a73b5e8a5
feat: add script to convert LoRA format to ComfyUI format
2025-09-14 20:49:20 +09:00
kohya-ss
e04b9f0497
docs: add LoRA training guide for HunyuanImage-2.1 model (by Gemini CLI)
2025-09-13 22:06:10 +09:00
Kohya S
29b0500e70
fix: restore files section in _typos.toml for exclusion configuration
2025-09-13 21:18:50 +09:00
Kohya S
4e2a80a6ca
refactor: update imports to use safetensors_utils for memory-efficient operations
2025-09-13 21:07:11 +09:00
Kohya S
d831c88832
fix: sample generation doesn't work with block swap
2025-09-13 21:06:04 +09:00
Kohya S
bae7fa74eb
Merge branch 'sd3' into feat-hunyuan-image-2.1-inference
2025-09-13 20:13:58 +09:00
Kohya S.
f5d44fd487
Merge pull request #2200 from kohya-ss/feat-faster-safetensors-load
...
feat: Speeding up loading .safetensors files
2025-09-13 20:09:03 +09:00
Kohya S
4568631b43
docs: update README to reflect improved loading speed of .safetensors files
2025-09-13 20:05:39 +09:00
Kohya S.
e1c666e97f
Update library/safetensors_utils.py
...
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
2025-09-13 20:03:55 +09:00
Kohya S
8783f8aed3
feat: faster safetensors load and split safetensor utils
2025-09-13 19:51:38 +09:00
Kohya S
9a61d61b22
feat: avoid unet type casting when fp8_scaled
2025-09-12 22:18:29 +09:00
Kohya S
7a651efd4d
feat: add 'tak' to recognized words and update block swap method to support backward pass
2025-09-12 22:00:41 +09:00
Kohya S
aa0af24d01
Merge branch 'sd3' into feat-hunyuan-image-2.1-inference
2025-09-12 21:41:12 +09:00
