435 Commits

Author	SHA1	Message	Date
rockerBOO	d90b293792	fix batch expansion of sigmas when batch size > 1	2025-10-12 16:11:41 -04:00
Dave Lage	2379bb27e8	Merge branch 'sd3' into network-wavelet-loss	2025-08-03 02:32:07 -04:00
Kohya S	c28e7a47c3	feat: add regex-based rank and learning rate configuration for FLUX.1 LoRA	2025-07-26 19:35:42 +09:00
rockerBOO	483ad41e69	Add import hint	2025-07-15 19:55:44 -04:00
rockerBOO	38b858926d	Revert "Add import hint" ... This reverts commit 9cedf18a97.	2025-07-15 19:54:41 -04:00
rockerBOO	9cedf18a97	Add import hint	2025-07-15 19:53:54 -04:00
rockerBOO	479ec9c8a6	Use new wavelet-loss library github.com/rockerBOO/wavelet-loss	2025-07-15 19:52:33 -04:00
rockerBOO	8cc81e45f7	Fix wavelet loss on non-flow matching models (sd1.5, SDXL). Fix wavelet coorelation.	2025-07-14 21:20:49 -04:00
rockerBOO	0145efc2f2	Merge branch 'sd3' into lumina	2025-06-09 18:13:06 -04:00
rockerBOO	9629853d15	Fix wavelet loss not separating levels. Refactor loss to be spatial	2025-06-05 22:03:52 -04:00
rockerBOO	0af0302c38	Metrics, energy, loss	2025-05-19 19:15:23 -04:00
rockerBOO	346790a996	Merge branch 'sd3' into network-wavelet-loss	2025-05-19 19:10:55 -04:00
rockerBOO	d0ce867498	Fix loss/wavelet metric	2025-05-06 00:29:27 -04:00
rockerBOO	869dc000d9	Remove latents	2025-05-06 00:04:12 -04:00
rockerBOO	984472ca09	Fix metrics	2025-05-04 18:58:54 -04:00
rockerBOO	964bfcb576	Fix padding for small latents. Add DWT tests	2025-05-04 18:57:42 -04:00
rockerBOO	d5f8f7de1f	Add wavelet loss fn	2025-05-04 18:57:10 -04:00
rockerBOO	56dfdae7c5	Add QuaternionWaveletTransform. Update WaveletLoss	2025-05-04 18:55:50 -04:00
rockerBOO	40128b7dc0	Use args.wavelet_loss to activate	2025-05-04 18:39:56 -04:00
rockerBOO	19ce0ae61f	Add wavelet_loss_band_level_weights	2025-05-04 18:39:51 -04:00
rockerBOO	7b9e92a8cc	Fix band weights via toml. Add more logging	2025-05-04 18:39:48 -04:00
rockerBOO	f553b7bf31	Add wavelet loss recording	2025-05-04 18:39:36 -04:00
rockerBOO	6d42b95e2b	Refactor transforms, fix loss calculations ... - add full conditional_loss functionality to wavelet loss - Transforms are separate and abstracted - Loss now doesn't include LL except the lowest level - ll_level_threshold allows you to control the level the ll is used in the loss - band weights can now be passed in - rectified flow calculations can be bypassed for experimentation - Fixed alpha to 1.0 with new weighted bands producing lower loss	2025-05-04 18:39:32 -04:00
rockerBOO	813942a967	Add wavelet loss	2025-05-04 18:38:47 -04:00
rockerBOO	b4a89c3cdf	Fix None	2025-05-01 02:03:22 -04:00
rockerBOO	f62c68df3c	Make grad_norm and combined_grad_norm None is not recording	2025-05-01 01:37:57 -04:00
Kohya S	5a18a03ffc	Merge branch 'dev' into sd3	2025-04-07 21:55:17 +09:00
青龍聖者@bdsqlsz	9f1892cc8e	Merge branch 'sd3' into lumina	2025-04-06 16:13:43 +08:00
Kohya S	d0b5c0e5cf	chore: formatting, add TODO comment	2025-03-30 21:15:37 +09:00
Kohya S.	59d98e45a9	Merge pull request #1974 from rockerBOO/lora-ggpo ... Add LoRA-GGPO for Flux	2025-03-30 21:07:31 +09:00
Kohya S.	93a4efabb5	Merge branch 'sd3' into resize-interpolation	2025-03-30 19:30:56 +09:00
DKnight54	381303d64f	Update train_network.py	2025-03-29 02:26:18 +08:00
rockerBOO	0181b7a042	Remove progress bar avg norms	2025-03-27 03:28:33 -04:00
rockerBOO	3647d065b5	Cache weight norms estimate on initialization. Move to update norms every step	2025-03-18 14:25:09 -04:00
rockerBOO	ea53290f62	Add LoRA-GGPO for Flux	2025-03-06 00:00:38 -05:00
rockerBOO	70403f6977	fix cache text encoder outputs if not using disk. small cleanup/alignment	2025-02-26 23:33:50 -05:00
rockerBOO	7b83d50dc0	Merge branch 'sd3' into lumina	2025-02-26 22:13:56 -05:00
Kohya S	1fcac98280	Merge branch 'sd3' into val-loss-improvement	2025-02-26 21:09:10 +09:00
Kohya S.	6e90c0f86c	Merge pull request #1909 from rockerBOO/progress_bar ... Move progress bar to account for sampling image first	2025-02-24 18:57:44 +09:00
rockerBOO	48e7da2d4a	Add sample batch size for Lumina	2025-02-23 20:19:24 -05:00
rockerBOO	025cca699b	Fix samples, LoRA training. Add system prompt, use_flash_attn	2025-02-23 01:29:18 -05:00
Kohya S	efb2a128cd	fix wandb val logging	2025-02-21 22:07:35 +09:00
rockerBOO	ca1c129ffd	Fix metadata	2025-02-19 14:20:40 -05:00
rockerBOO	7729c4c8f9	Add metadata	2025-02-19 14:20:40 -05:00
Kohya S	4a36996134	modify log step calculation	2025-02-18 22:05:08 +09:00
Kohya S	dc7d5fb459	Merge branch 'sd3' into val-loss-improvement	2025-02-18 21:34:30 +09:00
rockerBOO	1aa2f00e85	Fix validation epoch loss to check epoch average	2025-02-17 12:07:23 -05:00
rockerBOO	44782dd790	Fix validation epoch divergence	2025-02-17 12:07:22 -05:00
rockerBOO	4671e23778	Fix validation epoch loss to check epoch average	2025-02-16 01:42:44 -05:00
Kohya S	63337d9fe4	Merge branch 'sd3' into val-loss-improvement	2025-02-15 21:41:07 +09:00