20 Commits

Author	SHA1	Message	Date
Kohya S	2500f5a798	fix latents caching not working closes #1696	2024-10-15 07:16:34 +09:00
kohya-ss	c80c304779	Refactor caching in train scripts	2024-10-12 20:18:41 +09:00
Plat	a823fd9fb8	Improve wandb logging (#1576) ... * fix: wrong training steps were recorded to wandb, and no log was sent when logging_dir was not specified * fix: checking of whether wandb is enabled * feat: log images to wandb with their positive prompt as captions * feat: logging sample images' caption for sd3 and flux * fix: import wandb before use	2024-09-11 22:21:16 +09:00
Kohya S	92e7600cc2	Move freeze_blocks to sd3_train because it's only for sd3	2024-09-01 18:57:07 +09:00
青龍聖者@bdsqlsz	ef510b3cb9	Sd3 freeze x_block (#1417) ... * Update sd3_train.py * add freeze block lr * Update train_util.py * update	2024-09-01 18:41:01 +09:00
Kohya S	7367584e67	fix sd3 training to work without cachine TE outputs #1465	2024-08-17 14:38:34 +09:00
Kohya S	da4d0fe016	support attn mask for l+g/t5	2024-08-05 20:51:34 +09:00
Kohya S	002d75179a	sample images for training	2024-07-29 23:18:34 +09:00
Kohya S	41dee60383	Refactor caching mechanism for latents and text encoder outputs, etc.	2024-07-27 13:50:05 +09:00
Kohya S	082f13658b	reduce peak GPU memory usage before training	2024-07-12 21:28:01 +09:00
Kohya S	3d402927ef	WIP: update new latents caching	2024-07-09 23:15:38 +09:00
Kohya S	9dc7997803	fix typo	2024-07-09 20:37:00 +09:00
Kohya S	3ea4fce5e0	load models one by one	2024-07-08 22:04:43 +09:00
Kohya S	c9de7c4e9a	WIP: new latents caching	2024-07-08 19:48:28 +09:00
Kohya S	ea18d5ba6d	Fix to work full_bf16 and full_fp16.	2024-06-29 17:45:50 +09:00
Kohya S	19086465e8	Fix fp16 mixed precision, model is in bf16 without full_bf16	2024-06-29 17:21:25 +09:00
Kohya S	66cf435479	re-fix assertion ref #1389	2024-06-27 13:14:09 +09:00
Kohya S	828a581e29	fix assertion for experimental impl ref #1389	2024-06-26 20:43:31 +09:00
Kohya S	8f2ba27869	support text_encoder_batch_size for caching	2024-06-26 20:36:22 +09:00
Kohya S	d53ea22b2a	sd3 training	2024-06-23 23:38:20 +09:00