sdbds
fc772affbe
1、Implement cfg_trunc calculation directly using timesteps, without intermediate steps.
...
2、Deprecate and remove the guidance_scale parameter because it used in inference not train
3、Add inference command-line arguments --ct for cfg_trunc_ratio and --rc for renorm_cfg to control CFG truncation and renormalization during inference.
2025-02-24 14:10:24 +08:00
青龍聖者@bdsqlsz
653621de57
Merge pull request #15 from rockerBOO/samples-training
...
Fix samples, LoRA training. Add system prompt, use_flash_attn
2025-02-24 11:24:53 +08:00
rockerBOO
2c94d17f05
Fix typo
2025-02-23 20:21:06 -05:00
rockerBOO
48e7da2d4a
Add sample batch size for Lumina
2025-02-23 20:19:24 -05:00
rockerBOO
ba725a84e9
Set default discrete_flow_shift to 6.0. Remove default system prompt.
2025-02-23 18:01:09 -05:00
rockerBOO
42a801514c
Fix system prompt in datasets
2025-02-23 13:48:37 -05:00
rockerBOO
6d7bec8a37
Remove non-used code
2025-02-23 01:46:47 -05:00
rockerBOO
025cca699b
Fix samples, LoRA training. Add system prompt, use_flash_attn
2025-02-23 01:29:18 -05:00
青龍聖者@bdsqlsz
6597631b90
Merge pull request #14 from rockerBOO/samples-attention
...
Samples attention
2025-02-19 13:08:00 +08:00
rockerBOO
bd16bd13ae
Remove unused attention, fix typo
2025-02-18 01:21:18 -05:00
rockerBOO
98efbc3bb7
Add documentation to model, use SDPA attention, sample images
2025-02-18 00:58:53 -05:00
rockerBOO
1aa2f00e85
Fix validation epoch loss to check epoch average
2025-02-17 12:07:23 -05:00
rockerBOO
3ed7606f88
Clear sizes for validation reg images to be consistent
2025-02-17 12:07:23 -05:00
rockerBOO
3365cfadd7
Fix sizes for validation split
2025-02-17 12:07:23 -05:00
rockerBOO
44782dd790
Fix validation epoch divergence
2025-02-17 12:07:22 -05:00
sdbds
aa36c48685
update for always use gemma2 mask
2025-02-17 19:00:18 +08:00
青龍聖者@bdsqlsz
bb7bae5dff
Merge pull request #13 from rockerBOO/lumina-cache-checkpointing
...
Lumina cache checkpointing
2025-02-17 15:43:54 +08:00
sdbds
3ce23b7f16
Merge branch 'lumina' of https://github.com/sdbds/sd-scripts into lumina
...
# Conflicts:
# library/lumina_models.py
2025-02-17 14:53:16 +08:00
sdbds
733fdc09c6
update
2025-02-17 14:52:48 +08:00
青龍聖者@bdsqlsz
6965a0178a
Merge pull request #12 from rockerBOO/lumina-model-loading
...
Lumina 2 and Gemma 2 model loading
2025-02-17 00:47:08 +08:00
rockerBOO
16015635d2
Update metadata.resolution for Lumina 2
2025-02-16 01:36:29 -05:00
rockerBOO
60a76ebb72
Add caching gemma2, add gradient checkpointing, refactor lumina model code
2025-02-16 01:06:34 -05:00
rockerBOO
a00b06bc97
Lumina 2 and Gemma 2 model loading
2025-02-15 14:56:11 -05:00
sdbds
7323ee1b9d
update lora_lumina
2025-02-15 17:10:34 +08:00
sdbds
c0caf33e3f
update
2025-02-15 16:38:59 +08:00
sdbds
d154e76c45
init
2025-02-12 16:30:05 +08:00
Kohya S
0778dd9b1d
fix Text Encoder only LoRA training
2025-01-27 22:03:42 +09:00
Kohya S
59b3b94faf
README.md: Update limitation for validation loss support to include schedule-free optimizer
2025-01-26 21:52:58 +09:00
Kohya S.
f1ac81e07a
Merge pull request #1899 from kohya-ss/val-loss
...
Val loss
2025-01-26 21:32:29 +09:00
Kohya S
e8529613d8
README.md: Update recent updates section to include validation loss support for training scripts
2025-01-26 21:27:22 +09:00
Kohya S
58b82a576e
Fix to work with validation dataset
2025-01-26 21:21:21 +09:00
Kohya S.
b833d47afe
Merge pull request #1864 from rockerBOO/validation-loss-upstream
...
Validation loss
2025-01-26 21:08:14 +09:00
Kohya S
23ce75cf95
Merge branch 'dev' into sd3
2025-01-26 21:05:29 +09:00
Kohya S
6acdbed967
Merge branch 'dev' into sd3
2025-01-26 21:04:05 +09:00
rockerBOO
c04e5dfe92
Fix loss recorder on 0. Fix validation for cached runs. Assert on validation dataset
2025-01-23 09:57:24 -05:00
Kohya S.
6e3c1d0b58
Merge pull request #1879 from kohya-ss/dev
...
merge dev to main
2025-01-17 23:25:56 +09:00
Kohya S
345daaa986
update README for merging
2025-01-17 23:22:38 +09:00
rockerBOO
b489082495
Disable repeats for validation datasets
2025-01-12 16:42:04 -05:00
rockerBOO
25929dd0d7
Remove Validating... print to fix output layout
2025-01-12 15:38:57 -05:00
rockerBOO
ee9265cf26
Fix validate_every_n_steps for gradient accumulation
2025-01-12 14:56:35 -05:00
rockerBOO
0456858992
Fix validate_every_n_steps always running first step
2025-01-12 14:47:49 -05:00
rockerBOO
2bbb40ce51
Fix regularization images with validation
...
Adding metadata recording for validation arguments
Add comments about the validation split for clarity of intention
2025-01-12 14:29:50 -05:00
rockerBOO
4c61adc996
Add divergence to logs
...
Divergence is the difference between training and validation to
allow a clear value to indicate the difference between the two
in the logs.
2025-01-12 13:18:26 -05:00
rockerBOO
264167fa16
Apply is_training_dataset only to DreamBoothDataset. Add validation_split check and warning
2025-01-09 12:43:58 -05:00
rockerBOO
d6f158ddf6
Fix incorrect destructoring for load_abritrary_dataset
2025-01-08 18:48:05 -05:00
rockerBOO
1e61392cf2
Revert bucket_reso_steps to correct 64
2025-01-08 18:43:26 -05:00
rockerBOO
9fde0d7972
Handle tuple return from generate_dataset_group_by_blueprint
2025-01-08 18:38:20 -05:00
rockerBOO
556f3f1696
Fix documentation, remove unused function, fix bucket reso for sd1.5, fix multiple datasets
2025-01-08 13:41:15 -05:00
rockerBOO
1231f5114c
Remove unused train_util code, fix accelerate.log for wandb, add init_trackers library code
2025-01-07 22:31:41 -05:00
rockerBOO
742bee9738
Set validation steps in multiple lines for readability
2025-01-06 17:34:23 -05:00
