724 Commits

Author	SHA1	Message	Date
Kohya S	3be712e3e0	feat: Update direct loading fp8 ckpt for LoRA training	2024-08-27 21:40:02 +09:00
Kohya S	0087a46e14	FLUX.1 LoRA supports CLIP-L	2024-08-27 19:59:40 +09:00
Kohya S	72287d39c7	feat: Add shift option to --timestep_sampling in FLUX.1 fine-tuning and LoRA training	2024-08-25 16:01:24 +09:00
Kohya S	2e89cd2cc6	Fix issue with attention mask not being applied in single blocks	2024-08-24 12:39:54 +09:00
Kohya S	81411a398e	speed up getting image sizes	2024-08-22 22:02:29 +09:00
Kohya S	2d8fa3387a	Fix to remove zero pad for t5xxl output	2024-08-22 19:56:27 +09:00
kohya-ss	98c91a7625	Fix bug in FLUX multi GPU training	2024-08-22 12:37:41 +09:00
Kohya S	e1cd19c0c0	add stochastic rounding, fix single block	2024-08-21 21:04:10 +09:00
Kohya S	2b07a92c8d	Fix error in applying mask in Attention and add LoRA converter script	2024-08-21 12:30:23 +09:00
Kohya S	7e459c00b2	Update T5 attention mask handling in FLUX	2024-08-21 08:02:33 +09:00
Kohya S	6ab48b09d8	feat: Support multi-resolution training with caching latents to disk	2024-08-20 21:39:43 +09:00
Kohya S	486fe8f70a	feat: reduce memory usage and add memory efficient option for model saving	2024-08-19 22:30:24 +09:00
Kohya S	6e72a799c8	reduce peak VRAM usage by excluding some blocks to cuda	2024-08-19 21:55:28 +09:00
Kohya S	ef535ec6bb	add memory efficient training for FLUX.1	2024-08-18 16:54:18 +09:00
Kohya S	400955d3ea	add fine tuning FLUX.1 (WIP)	2024-08-17 15:36:18 +09:00
Kohya S	e45d3f8634	add merge LoRA script	2024-08-16 22:19:21 +09:00
Kohya S	3921a4efda	add t5xxl max token length, support schnell	2024-08-16 17:06:05 +09:00
Kohya S	7db4222119	add sample image generation during training	2024-08-14 22:15:26 +09:00
Kohya S	56d7651f08	add experimental split mode for FLUX	2024-08-13 22:28:39 +09:00
kohya-ss	f5ce754bc2	Merge branch 'dev' into sd3	2024-08-13 21:00:44 +09:00
Kohya S	d25ae361d0	fix apply_t5_attn_mask to work	2024-08-11 19:07:07 +09:00
Kohya S	8a0f12dde8	update FLUX LoRA training	2024-08-10 23:42:05 +09:00
Kohya S	808d2d1f48	fix typos	2024-08-09 23:02:51 +09:00
Kohya S	36b2e6fc28	add FLUX.1 LoRA training	2024-08-09 22:56:48 +09:00
Kohya S	da4d0fe016	support attn mask for l+g/t5	2024-08-05 20:51:34 +09:00
Kohya S	231df197dd	Fix npz path for verification	2024-08-05 20:26:30 +09:00
Kohya S	002d75179a	sample images for training	2024-07-29 23:18:34 +09:00
Kohya S	1a977e847a	fix typos	2024-07-27 13:51:50 +09:00
Kohya S	41dee60383	Refactor caching mechanism for latents and text encoder outputs, etc.	2024-07-27 13:50:05 +09:00
sdbds	9ca7a5b6cc	instead cv2 LANCZOS4 resize to pil resize	2024-07-20 21:59:11 +08:00
sdbds	1f16b80e88	Revert "judge image size for using diff interpolation" ... This reverts commit 87526942a6.	2024-07-20 21:35:24 +08:00
Millie	2e67978ee2	Generate sample images without having CUDA (such as on Macs)	2024-07-18 11:52:58 -07:00
sdbds	87526942a6	judge image size for using diff interpolation	2024-07-12 22:56:38 +08:00
Kohya S	082f13658b	reduce peak GPU memory usage before training	2024-07-12 21:28:01 +09:00
Kohya S	3d402927ef	WIP: update new latents caching	2024-07-09 23:15:38 +09:00
Kohya S	9dc7997803	fix typo	2024-07-09 20:37:00 +09:00
Kohya S	3ea4fce5e0	load models one by one	2024-07-08 22:04:43 +09:00
Kohya S	c9de7c4e9a	WIP: new latents caching	2024-07-08 19:48:28 +09:00
Kohya S	50e3d62474	fix to work T5XXL with fp16	2024-07-08 19:46:23 +09:00
Kohya S	ea18d5ba6d	Fix to work full_bf16 and full_fp16.	2024-06-29 17:45:50 +09:00
Kohya S	19086465e8	Fix fp16 mixed precision, model is in bf16 without full_bf16	2024-06-29 17:21:25 +09:00
Kohya S	381598c8bb	fix resolution in metadata for sd3	2024-06-26 21:15:02 +09:00
Kohya S	8f2ba27869	support text_encoder_batch_size for caching	2024-06-26 20:36:22 +09:00
Kohya S	0b3e4f7ab6	show file name if error in load_image ref #1385	2024-06-25 20:03:09 +09:00
Kohya S	4802e4aaec	workaround for long caption ref #1382	2024-06-24 23:13:14 +09:00
Kohya S	d53ea22b2a	sd3 training	2024-06-23 23:38:20 +09:00
Kohya S	e5268286bf	add sd3 models and inference script	2024-06-15 22:20:24 +09:00
Kohya S	4a441889d4	Merge branch 'dev' into train_resume_step	2024-06-11 19:27:37 +09:00
Yuta Hayashibe	4ecbac131a	Bump crate-ci/typos from 1.19.0 to 1.21.0, fix typos, and updated _typos.toml (Close #1307)	2024-06-05 16:31:55 +09:00
Kohya S	4dbcef429b	update for corner cases	2024-06-04 21:26:55 +09:00