684 Commits

Author	SHA1	Message	Date
Kohya S	6dbfd47a59	Fix to work PIECEWISE_CONSTANT, update requirement.txt and README #1393	2024-09-11 21:44:36 +09:00
青龍聖者@bdsqlsz	fd68703f37	Add New lr scheduler (#1393) ... * add new lr scheduler * fix bugs and use num_cycles / 2 * Update requirements.txt * add num_cycles for min lr * keep PIECEWISE_CONSTANT * allow use float with warmup or decay ratio. * Update train_util.py	2024-09-11 21:25:45 +09:00
Kohya S	ce144476cf	Merge branch 'dev' into sd3	2024-09-07 10:59:22 +09:00
Kohya S	62ec3e6424	Merge branch 'main' into dev	2024-09-07 10:52:49 +09:00
Kohya S	0005867ba5	update README, format code	2024-09-07 10:45:18 +09:00
Kohya S.	16bb5699ac	Merge pull request #1426 from sdbds/resize ... Replacing CV2 resize to Pil resize	2024-09-07 10:22:52 +09:00
Kohya S.	319e4d9831	Merge pull request #1433 from millie-v/sample-image-without-cuda ... Generate sample images without having CUDA (such as on Macs)	2024-09-07 10:19:55 +09:00
Kohya S	b65ae9b439	T5XXL LoRA training, fp8 T5XXL support	2024-09-04 21:33:17 +09:00
Kohya S	4f6d915d15	update help and README	2024-09-01 19:12:29 +09:00
sdbds	25c9040f4f	Update flux_train_utils.py	2024-08-31 19:53:59 +08:00
Nando Metzger	2a3aefb4e4	Update train_util.py, bug fix	2024-08-30 08:15:05 +02:00
Kohya S	3be712e3e0	feat: Update direct loading fp8 ckpt for LoRA training	2024-08-27 21:40:02 +09:00
Kohya S	0087a46e14	FLUX.1 LoRA supports CLIP-L	2024-08-27 19:59:40 +09:00
Kohya S	72287d39c7	feat: Add shift option to --timestep_sampling in FLUX.1 fine-tuning and LoRA training	2024-08-25 16:01:24 +09:00
Kohya S	2e89cd2cc6	Fix issue with attention mask not being applied in single blocks	2024-08-24 12:39:54 +09:00
Kohya S	2d8fa3387a	Fix to remove zero pad for t5xxl output	2024-08-22 19:56:27 +09:00
kohya-ss	98c91a7625	Fix bug in FLUX multi GPU training	2024-08-22 12:37:41 +09:00
Kohya S	e1cd19c0c0	add stochastic rounding, fix single block	2024-08-21 21:04:10 +09:00
Kohya S	2b07a92c8d	Fix error in applying mask in Attention and add LoRA converter script	2024-08-21 12:30:23 +09:00
Kohya S	7e459c00b2	Update T5 attention mask handling in FLUX	2024-08-21 08:02:33 +09:00
Kohya S	6ab48b09d8	feat: Support multi-resolution training with caching latents to disk	2024-08-20 21:39:43 +09:00
Kohya S	486fe8f70a	feat: reduce memory usage and add memory efficient option for model saving	2024-08-19 22:30:24 +09:00
Kohya S	6e72a799c8	reduce peak VRAM usage by excluding some blocks to cuda	2024-08-19 21:55:28 +09:00
Kohya S	ef535ec6bb	add memory efficient training for FLUX.1	2024-08-18 16:54:18 +09:00
Kohya S	400955d3ea	add fine tuning FLUX.1 (WIP)	2024-08-17 15:36:18 +09:00
Kohya S	e45d3f8634	add merge LoRA script	2024-08-16 22:19:21 +09:00
Kohya S	3921a4efda	add t5xxl max token length, support schnell	2024-08-16 17:06:05 +09:00
Kohya S	7db4222119	add sample image generation during training	2024-08-14 22:15:26 +09:00
Kohya S	56d7651f08	add experimental split mode for FLUX	2024-08-13 22:28:39 +09:00
kohya-ss	f5ce754bc2	Merge branch 'dev' into sd3	2024-08-13 21:00:44 +09:00
Kohya S	d25ae361d0	fix apply_t5_attn_mask to work	2024-08-11 19:07:07 +09:00
Kohya S	8a0f12dde8	update FLUX LoRA training	2024-08-10 23:42:05 +09:00
Kohya S	808d2d1f48	fix typos	2024-08-09 23:02:51 +09:00
Kohya S	36b2e6fc28	add FLUX.1 LoRA training	2024-08-09 22:56:48 +09:00
Kohya S	da4d0fe016	support attn mask for l+g/t5	2024-08-05 20:51:34 +09:00
Kohya S	231df197dd	Fix npz path for verification	2024-08-05 20:26:30 +09:00
Kohya S	002d75179a	sample images for training	2024-07-29 23:18:34 +09:00
Kohya S	1a977e847a	fix typos	2024-07-27 13:51:50 +09:00
Kohya S	41dee60383	Refactor caching mechanism for latents and text encoder outputs, etc.	2024-07-27 13:50:05 +09:00
sdbds	9ca7a5b6cc	instead cv2 LANCZOS4 resize to pil resize	2024-07-20 21:59:11 +08:00
sdbds	1f16b80e88	Revert "judge image size for using diff interpolation" ... This reverts commit 87526942a6.	2024-07-20 21:35:24 +08:00
Millie	2e67978ee2	Generate sample images without having CUDA (such as on Macs)	2024-07-18 11:52:58 -07:00
sdbds	87526942a6	judge image size for using diff interpolation	2024-07-12 22:56:38 +08:00
Kohya S	082f13658b	reduce peak GPU memory usage before training	2024-07-12 21:28:01 +09:00
Kohya S	3d402927ef	WIP: update new latents caching	2024-07-09 23:15:38 +09:00
Kohya S	9dc7997803	fix typo	2024-07-09 20:37:00 +09:00
Kohya S	3ea4fce5e0	load models one by one	2024-07-08 22:04:43 +09:00
Kohya S	c9de7c4e9a	WIP: new latents caching	2024-07-08 19:48:28 +09:00
Kohya S	50e3d62474	fix to work T5XXL with fp16	2024-07-08 19:46:23 +09:00
Kohya S	ea18d5ba6d	Fix to work full_bf16 and full_fp16.	2024-06-29 17:45:50 +09:00