836 Commits

Author	SHA1	Message	Date
rockerBOO	9647f1e324	Fix validation block swap. Add custom offloading tests	2025-02-27 20:36:36 -05:00
rockerBOO	42fe22f5a2	Enable block swap for Lumina	2025-02-27 03:21:24 -05:00
rockerBOO	0886d976f1	Add block swap	2025-02-27 02:31:50 -05:00
rockerBOO	7b83d50dc0	Merge branch 'sd3' into lumina	2025-02-26 22:13:56 -05:00
Kohya S	ec350c83eb	Merge branch 'dev' into sd3	2025-02-26 21:17:29 +09:00
Kohya S	1fcac98280	Merge branch 'sd3' into val-loss-improvement	2025-02-26 21:09:10 +09:00
sdbds	a1a5627b13	fix shift	2025-02-26 11:35:38 +08:00
sdbds	ce37c08b9a	clean code and add finetune code	2025-02-26 11:20:03 +08:00
Disty0	f68702f71c	Update IPEX libs	2025-02-25 21:27:41 +03:00
sdbds	5f9047c8cf	add truncation when > max_length	2025-02-26 01:00:35 +08:00
Kohya S	67fde015f7	Merge branch 'dev' into sd3	2025-02-24 18:56:15 +09:00
Kohya S.	386b7332c6	Merge pull request #1918 from tsukimiya/fix_vperd_warning ... Remove v-pred warning.	2025-02-24 18:55:25 +09:00
Kohya S	905f081798	Merge branch 'dev' into sd3	2025-02-24 18:54:28 +09:00
sdbds	fc772affbe	1、Implement cfg_trunc calculation directly using timesteps, without intermediate steps. ... 2、Deprecate and remove the guidance_scale parameter because it used in inference not train 3、Add inference command-line arguments --ct for cfg_trunc_ratio and --rc for renorm_cfg to control CFG truncation and renormalization during inference.	2025-02-24 14:10:24 +08:00
rockerBOO	2c94d17f05	Fix typo	2025-02-23 20:21:06 -05:00
rockerBOO	48e7da2d4a	Add sample batch size for Lumina	2025-02-23 20:19:24 -05:00
rockerBOO	ba725a84e9	Set default discrete_flow_shift to 6.0. Remove default system prompt.	2025-02-23 18:01:09 -05:00
rockerBOO	42a801514c	Fix system prompt in datasets	2025-02-23 13:48:37 -05:00
rockerBOO	6d7bec8a37	Remove non-used code	2025-02-23 01:46:47 -05:00
rockerBOO	025cca699b	Fix samples, LoRA training. Add system prompt, use_flash_attn	2025-02-23 01:29:18 -05:00
Kohya S	efb2a128cd	fix wandb val logging	2025-02-21 22:07:35 +09:00
Yidi	13df47516d	Remove position_ids for V2 ... The postions_ids cause errors for the newer version of transformer. This has already been fixed in convert_ldm_clip_checkpoint_v1() but not in v2. The new code applies the same fix to convert_ldm_clip_checkpoint_v2().	2025-02-20 04:49:51 -05:00
Kohya S	dc7d5fb459	Merge branch 'sd3' into val-loss-improvement	2025-02-18 21:34:30 +09:00
rockerBOO	bd16bd13ae	Remove unused attention, fix typo	2025-02-18 01:21:18 -05:00
rockerBOO	98efbc3bb7	Add documentation to model, use SDPA attention, sample images	2025-02-18 00:58:53 -05:00
rockerBOO	9436b41061	Fix validation split and add test	2025-02-17 14:28:41 -05:00
rockerBOO	3ed7606f88	Clear sizes for validation reg images to be consistent	2025-02-17 12:07:23 -05:00
rockerBOO	3365cfadd7	Fix sizes for validation split	2025-02-17 12:07:23 -05:00
sdbds	aa36c48685	update for always use gemma2 mask	2025-02-17 19:00:18 +08:00
rockerBOO	f3a010978c	Clear sizes for validation reg images to be consistent	2025-02-16 22:28:34 -05:00
rockerBOO	3c7496ae3f	Fix sizes for validation split	2025-02-16 22:18:14 -05:00
rockerBOO	16015635d2	Update metadata.resolution for Lumina 2	2025-02-16 01:36:29 -05:00
rockerBOO	60a76ebb72	Add caching gemma2, add gradient checkpointing, refactor lumina model code	2025-02-16 01:06:34 -05:00
rockerBOO	a00b06bc97	Lumina 2 and Gemma 2 model loading	2025-02-15 14:56:11 -05:00
sdbds	c0caf33e3f	update	2025-02-15 16:38:59 +08:00
sdbds	d154e76c45	init	2025-02-12 16:30:05 +08:00
Kohya S	a24db1d532	fix: validation timestep generation fails on SD/SDXL training	2025-02-04 22:02:42 +09:00
tsukimiya	4a71687d20	不要な警告の削除 ... (おそらく be14c06267 の修正漏れ )	2025-02-04 00:42:27 +09:00
Kohya S	58b82a576e	Fix to work with validation dataset	2025-01-26 21:21:21 +09:00
rockerBOO	c04e5dfe92	Fix loss recorder on 0. Fix validation for cached runs. Assert on validation dataset	2025-01-23 09:57:24 -05:00
rockerBOO	b489082495	Disable repeats for validation datasets	2025-01-12 16:42:04 -05:00
rockerBOO	2bbb40ce51	Fix regularization images with validation ... Adding metadata recording for validation arguments Add comments about the validation split for clarity of intention	2025-01-12 14:29:50 -05:00
rockerBOO	264167fa16	Apply is_training_dataset only to DreamBoothDataset. Add validation_split check and warning	2025-01-09 12:43:58 -05:00
rockerBOO	9fde0d7972	Handle tuple return from generate_dataset_group_by_blueprint	2025-01-08 18:38:20 -05:00
rockerBOO	556f3f1696	Fix documentation, remove unused function, fix bucket reso for sd1.5, fix multiple datasets	2025-01-08 13:41:15 -05:00
rockerBOO	1231f5114c	Remove unused train_util code, fix accelerate.log for wandb, add init_trackers library code	2025-01-07 22:31:41 -05:00
rockerBOO	695f38962c	Move get_huber_threshold_if_needed	2025-01-03 15:25:12 -05:00
rockerBOO	0522070d19	Fix training, validation split, revert to using upstream implemenation	2025-01-03 15:20:25 -05:00
rockerBOO	6604b36044	Remove duplicate assignment	2025-01-03 02:04:59 -05:00
rockerBOO	fbfc2753eb	Update text for train/reg with repeats	2025-01-03 01:53:12 -05:00