861 Commits

Author	SHA1	Message	Date
recris	740ec1d526	Fix issues found in review	2024-11-28 20:38:32 +00:00
recris	420a180d93	Implement pseudo Huber loss for Flux and SD3	2024-11-27 18:37:09 +00:00
minux302	0b5229a955	save cn	2024-11-21 15:55:27 +00:00
minux302	31ca899b6b	fix depth value	2024-11-18 13:03:28 +00:00
minux302	4dd4cd6ec8	work cn load and validation	2024-11-18 12:47:01 +00:00
minux302	35778f0218	fix sample_images type	2024-11-17 11:09:05 +00:00
minux302	b2660bbe74	train run	2024-11-17 10:24:57 +00:00
minux302	e358b118af	fix dataloader	2024-11-16 14:49:29 +09:00
minux302	42f6edf3a8	fix for adding controlnet	2024-11-15 23:48:51 +09:00
minux302	ccfaa001e7	add flux controlnet base module	2024-11-15 20:21:28 +09:00
Kohya S	2bb0f547d7	update grad hook creation to fix TE lr in sd3 fine tuning	2024-11-14 19:33:12 +09:00
Kohya S	2cb7a6db02	feat: add block swap for FLUX.1/SD3 LoRA training	2024-11-12 21:39:13 +09:00
Kohya S	17cf249d76	Merge branch 'sd3' into faster-block-swap	2024-11-12 08:49:15 +09:00
Kohya S	cde90b8903	feat: implement block swapping for FLUX.1 LoRA (WIP)	2024-11-12 08:49:05 +09:00
Kohya S	3fe94b058a	update comment	2024-11-12 08:09:07 +09:00
Kohya S	7feaae5f06	Merge branch 'sd3' into faster-block-swap	2024-11-11 21:16:01 +09:00
Kohya S	02bd76e6c7	Refactor block swapping to utilize custom offloading utilities	2024-11-11 21:15:36 +09:00
sdbds	26bd4540a6	init	2024-11-11 09:25:28 +08:00
Kohya S.	2a2042a762	Merge pull request #1770 from feffy380/fix-size-from-cache ... fix: sort order when getting image size from cache file	2024-11-09 19:51:03 +09:00
feffy380	b3248a8eef	fix: sort order when getting image size from cache file	2024-11-07 14:31:05 +01:00
Kohya S	186aa5b97d	fix illeagal block is swapped #1764	2024-11-07 22:16:05 +09:00
Kohya S	b8d3feca77	Merge branch 'sd3' into faster-block-swap	2024-11-07 21:43:48 +09:00
Kohya S	5e86323f12	Update README and clean-up the code for SD3 timesteps	2024-11-07 21:27:12 +09:00
Dango233	bafd10d558	Fix typo	2024-11-07 18:21:04 +08:00
Dango233	40ed54bfc0	Simplify Timestep weighting ... * Remove diffusers dependency in ts & sigma calc * support Shift setting * Add uniform distribution * Default to Uniform distribution and shift 1	2024-11-07 09:53:54 +00:00
Kohya S	aab943cea3	remove unused weight swapping functions from utils.py	2024-11-05 23:27:41 +09:00
Kohya S	81c0c965a2	faster block swap	2024-11-05 21:22:42 +09:00
Kohya S	82daa98fe8	remove duplicate resolution for scaled pos embed	2024-11-01 21:43:47 +09:00
Kohya S	9aa6f52ac3	Fix memory leak in latent caching. bmp failed to cache	2024-11-01 21:43:21 +09:00
Kohya S	830df4abcc	Fix crashing if image is too tall or wide.	2024-10-31 21:39:07 +09:00
Kohya S	1434d8506f	Support SD3.5M multi resolutional training	2024-10-31 19:58:22 +09:00
Kohya S	70a179e446	Fix to use SDPA instead of xformers	2024-10-30 14:34:19 +09:00
Kohya S	8c3c825b5f	Merge branch 'sd3_5_support' of https://github.com/kohya-ss/sd-scripts into sd3_5_support	2024-10-30 12:51:55 +09:00
Kohya S	bdddc20d68	support SD3.5M	2024-10-30 12:51:49 +09:00
kohya-ss	c9a1417157	Merge branch 'sd3' into sd3_5_support	2024-10-29 22:30:01 +09:00
Kohya S	1e2f7b0e44	Support for checkpoint files with a mysterious prefix "model.diffusion_model."	2024-10-29 22:11:04 +09:00
Kohya S	75554867ce	Fix error on saving T5XXL	2024-10-29 08:34:31 +09:00
Kohya S	af8e216035	Fix sample image gen to work with block swap	2024-10-28 22:08:57 +09:00
kohya-ss	1065dd1b56	Fix to work dropout_rate for TEs	2024-10-27 19:36:36 +09:00
kohya-ss	d4f7849592	prevent unintended cast for disk cached TE outputs	2024-10-27 19:35:56 +09:00
kohya-ss	a1255d637f	Fix SD3 LoRA training to work (WIP)	2024-10-27 17:03:36 +09:00
Kohya S	db2b4d41b9	Add dropout rate arguments for CLIP-L, CLIP-G, and T5, fix Text Encoders LoRA not trained	2024-10-27 16:42:58 +09:00
kohya-ss	014064fd81	fix sample image generation without seed failed close #1726	2024-10-26 18:59:45 +09:00
kohya-ss	56bf761164	fix errors in SD3 LoRA training with Text Encoders close #1724	2024-10-26 17:29:24 +09:00
kohya-ss	d2c549d7b2	support SD3 LoRA	2024-10-25 21:58:31 +09:00
Kohya S	f52fb66e8f	Merge branch 'sd3' into sd3_5_support	2024-10-25 19:03:58 +09:00
Kohya S	5fba6f514a	Merge branch 'dev' into sd3	2024-10-25 19:03:27 +09:00
Kohya S.	b8ae745d0c	Merge pull request #1717 from catboxanon/fix/remove-vpred-warnings ... Remove v-pred warnings	2024-10-25 18:49:40 +09:00
Kohya S	f8c5146d71	support block swap with fused_optimizer_pass	2024-10-24 22:02:05 +09:00
Kohya S	e3c43bda49	reduce memory usage in sample image generation	2024-10-24 20:35:47 +09:00