302 Commits

Author	SHA1	Message	Date
Kohya S	56a7bc171d	new block swap for FLUX.1 fine tuning	2024-09-26 08:26:31 +09:00
Kohya S	65fb69f808	Merge branch 'dev' into sd3	2024-09-25 20:56:16 +09:00
Kohya S	e74f58148c	update README	2024-09-25 20:55:50 +09:00
Kohya S	1286e00bb0	fix to call train/eval in schedulefree #1605	2024-09-18 21:31:54 +09:00
Kohya S	e74502117b	update README	2024-09-18 08:04:32 +09:00
Kohya S.	bbd160b4ca	sd3 schedule free opt (#1605) ... * New ScheduleFree support for Flux (#1600) * init * use no schedule * fix typo * update for eval() * fix typo * update * Update train_util.py * Update requirements.txt * update sfwrapper WIP * no need to check schedulefree optimizer * remove debug print * comment out schedulefree wrapper * update readme --------- Co-authored-by: 青龍聖者@bdsqlsz <865105819@qq.com>	2024-09-18 07:55:04 +09:00
Kohya S	d8d15f1a7e	add support for specifying blocks in FLUX.1 LoRA training	2024-09-16 23:14:09 +09:00
Kohya S	be078bdaca	fix typo	2024-09-15 13:59:17 +09:00
Kohya S	9f44ef1330	add diffusers to FLUX.1 conversion script	2024-09-15 13:52:23 +09:00
Kohya S	6445bb2bc9	update README	2024-09-14 22:37:26 +09:00
Kohya S	c9ff4de905	Add support for specifying rank for each layer in FLUX.1	2024-09-14 22:17:52 +09:00
Kohya S	0485f236a0	Merge branch 'dev' into sd3	2024-09-13 22:39:24 +09:00
Kohya S	93d9fbf607	improve OFT implementation closes #944	2024-09-13 22:37:11 +09:00
Kohya S	c15a3a1a65	Merge branch 'dev' into sd3	2024-09-13 21:30:49 +09:00
Kohya S	43ad73860d	Merge branch 'main' into dev	2024-09-13 21:29:51 +09:00
Kohya S	b755ebd0a4	add LBW support for SDXL merge LoRA	2024-09-13 21:29:31 +09:00
Kohya S	f3ce80ef8f	Merge branch 'dev' into sd3	2024-09-13 19:49:16 +09:00
Kohya S	9d2860760d	Merge branch 'main' into dev	2024-09-13 19:48:53 +09:00
Kohya S	3387dc7306	formatting, update README	2024-09-13 19:45:42 +09:00
Kohya S	237317fffd	update README	2024-09-11 22:23:43 +09:00
Kohya S	eaafa5c9da	Merge branch 'dev' into sd3	2024-09-11 21:46:21 +09:00
Kohya S	6dbfd47a59	Fix to work PIECEWISE_CONSTANT, update requirement.txt and README #1393	2024-09-11 21:44:36 +09:00
Kohya S	65b8a064f6	update README	2024-09-10 21:20:38 +09:00
Kohya S	d10ff62a78	support individual LR for CLIP-L/T5XXL	2024-09-10 20:32:09 +09:00
Kohya S	d29af146b8	add negative prompt for flux inference script	2024-09-09 23:01:15 +09:00
Kohya S	ce144476cf	Merge branch 'dev' into sd3	2024-09-07 10:59:22 +09:00
Kohya S	0005867ba5	update README, format code	2024-09-07 10:45:18 +09:00
Kohya S	2889108d85	feat: Add --cpu_offload_checkpointing option to LoRA training	2024-09-05 20:58:33 +09:00
Kohya S	90ed2dfb52	feat: Add support for merging CLIP-L and T5XXL LoRA models	2024-09-05 08:39:29 +09:00
Kohya S	b7cff0a754	update README	2024-09-04 21:35:47 +09:00
Kohya S	b65ae9b439	T5XXL LoRA training, fp8 T5XXL support	2024-09-04 21:33:17 +09:00
Kohya S	6abacf04da	update README	2024-09-02 13:05:26 +09:00
Kohya S	4f6d915d15	update help and README	2024-09-01 19:12:29 +09:00
Kohya S	92e7600cc2	Move freeze_blocks to sd3_train because it's only for sd3	2024-09-01 18:57:07 +09:00
Kohya S	8fdfd8c857	Update safetensors to version 0.4.4 in requirements.txt #1524	2024-08-29 22:26:29 +09:00
Kohya S.	daa6ad5165	Update README.md	2024-08-29 21:25:30 +09:00
Kohya S	a0cfb0894c	Cleaned up README	2024-08-29 21:20:33 +09:00
Kohya S	a61cf73a5c	update readme	2024-08-27 21:44:10 +09:00
Kohya S	3be712e3e0	feat: Update direct loading fp8 ckpt for LoRA training	2024-08-27 21:40:02 +09:00
Kohya S	0087a46e14	FLUX.1 LoRA supports CLIP-L	2024-08-27 19:59:40 +09:00
Kohya S	72287d39c7	feat: Add shift option to --timestep_sampling in FLUX.1 fine-tuning and LoRA training	2024-08-25 16:01:24 +09:00
Kohya S	ea9242653c	Merge branch 'dev' into sd3	2024-08-24 21:24:44 +09:00
Kohya S	d5c076cf90	update readme	2024-08-24 21:21:39 +09:00
Kohya S	cf689e7aa6	feat: Add option to split projection layers and apply LoRA	2024-08-24 16:35:43 +09:00
Kohya S	2e89cd2cc6	Fix issue with attention mask not being applied in single blocks	2024-08-24 12:39:54 +09:00
Kohya S	2d8fa3387a	Fix to remove zero pad for t5xxl output	2024-08-22 19:56:27 +09:00
kohya-ss	98c91a7625	Fix bug in FLUX multi GPU training	2024-08-22 12:37:41 +09:00
Kohya S	e1cd19c0c0	add stochastic rounding, fix single block	2024-08-21 21:04:10 +09:00
Kohya S	2b07a92c8d	Fix error in applying mask in Attention and add LoRA converter script	2024-08-21 12:30:23 +09:00
Kohya S	7e459c00b2	Update T5 attention mask handling in FLUX	2024-08-21 08:02:33 +09:00