17 Commits

Author	SHA1	Message	Date
Duoong	1640e53392	Fix bug and optimization Lumina training	2026-02-12 22:52:28 +07:00
rockerBOO	d94bed645a	Add lumina tests and fix image masks	2025-06-09 21:14:51 -04:00
rockerBOO	1f22a94cfe	Update embedder_dims, add more flexible caption extension	2025-03-04 02:25:50 -05:00
青龍聖者@bdsqlsz	09c4710d1e	Merge pull request #22 from rockerBOO/sage_attn ... Add Sage Attention for Lumina	2025-03-03 10:26:02 +08:00
rockerBOO	a69884a209	Add Sage Attention for Lumina	2025-03-01 20:37:45 -05:00
rockerBOO	9647f1e324	Fix validation block swap. Add custom offloading tests	2025-02-27 20:36:36 -05:00
rockerBOO	42fe22f5a2	Enable block swap for Lumina	2025-02-27 03:21:24 -05:00
rockerBOO	0886d976f1	Add block swap	2025-02-27 02:31:50 -05:00
sdbds	fc772affbe	1、Implement cfg_trunc calculation directly using timesteps, without intermediate steps. ... 2、Deprecate and remove the guidance_scale parameter because it used in inference not train 3、Add inference command-line arguments --ct for cfg_trunc_ratio and --rc for renorm_cfg to control CFG truncation and renormalization during inference.	2025-02-24 14:10:24 +08:00
rockerBOO	48e7da2d4a	Add sample batch size for Lumina	2025-02-23 20:19:24 -05:00
rockerBOO	ba725a84e9	Set default discrete_flow_shift to 6.0. Remove default system prompt.	2025-02-23 18:01:09 -05:00
rockerBOO	025cca699b	Fix samples, LoRA training. Add system prompt, use_flash_attn	2025-02-23 01:29:18 -05:00
rockerBOO	bd16bd13ae	Remove unused attention, fix typo	2025-02-18 01:21:18 -05:00
rockerBOO	98efbc3bb7	Add documentation to model, use SDPA attention, sample images	2025-02-18 00:58:53 -05:00
rockerBOO	60a76ebb72	Add caching gemma2, add gradient checkpointing, refactor lumina model code	2025-02-16 01:06:34 -05:00
rockerBOO	a00b06bc97	Lumina 2 and Gemma 2 model loading	2025-02-15 14:56:11 -05:00
sdbds	d154e76c45	init	2025-02-12 16:30:05 +08:00