Commit Graph

2049 Commits

Author SHA1 Message Date
Kohya S
0778dd9b1d fix Text Encoder only LoRA training 2025-01-27 22:03:42 +09:00
Kohya S
59b3b94faf README.md: Update limitation for validation loss support to include schedule-free optimizer 2025-01-26 21:52:58 +09:00
Kohya S.
f1ac81e07a Merge pull request #1899 from kohya-ss/val-loss
Val loss
2025-01-26 21:32:29 +09:00
Kohya S
e8529613d8 README.md: Update recent updates section to include validation loss support for training scripts 2025-01-26 21:27:22 +09:00
Kohya S
58b82a576e Fix to work with validation dataset 2025-01-26 21:21:21 +09:00
Kohya S.
b833d47afe Merge pull request #1864 from rockerBOO/validation-loss-upstream
Validation loss
2025-01-26 21:08:14 +09:00
Kohya S
23ce75cf95 Merge branch 'dev' into sd3 2025-01-26 21:05:29 +09:00
Kohya S
6acdbed967 Merge branch 'dev' into sd3 2025-01-26 21:04:05 +09:00
rockerBOO
c04e5dfe92 Fix loss recorder on 0. Fix validation for cached runs. Assert on validation dataset 2025-01-23 09:57:24 -05:00
Kohya S.
6e3c1d0b58 Merge pull request #1879 from kohya-ss/dev
merge dev to main
v0.9.0
2025-01-17 23:25:56 +09:00
Kohya S
345daaa986 update README for merging 2025-01-17 23:22:38 +09:00
rockerBOO
b489082495 Disable repeats for validation datasets 2025-01-12 16:42:04 -05:00
rockerBOO
25929dd0d7 Remove Validating... print to fix output layout 2025-01-12 15:38:57 -05:00
rockerBOO
ee9265cf26 Fix validate_every_n_steps for gradient accumulation 2025-01-12 14:56:35 -05:00
rockerBOO
0456858992 Fix validate_every_n_steps always running first step 2025-01-12 14:47:49 -05:00
rockerBOO
2bbb40ce51 Fix regularization images with validation
Adding metadata recording for validation arguments
Add comments about the validation split for clarity of intention
2025-01-12 14:29:50 -05:00
rockerBOO
4c61adc996 Add divergence to logs
Divergence is the difference between training and validation to
allow a clear value to indicate the difference between the two
in the logs.
2025-01-12 13:18:26 -05:00
rockerBOO
264167fa16 Apply is_training_dataset only to DreamBoothDataset. Add validation_split check and warning 2025-01-09 12:43:58 -05:00
rockerBOO
d6f158ddf6 Fix incorrect destructoring for load_abritrary_dataset 2025-01-08 18:48:05 -05:00
rockerBOO
1e61392cf2 Revert bucket_reso_steps to correct 64 2025-01-08 18:43:26 -05:00
rockerBOO
9fde0d7972 Handle tuple return from generate_dataset_group_by_blueprint 2025-01-08 18:38:20 -05:00
rockerBOO
556f3f1696 Fix documentation, remove unused function, fix bucket reso for sd1.5, fix multiple datasets 2025-01-08 13:41:15 -05:00
rockerBOO
1231f5114c Remove unused train_util code, fix accelerate.log for wandb, add init_trackers library code 2025-01-07 22:31:41 -05:00
rockerBOO
742bee9738 Set validation steps in multiple lines for readability 2025-01-06 17:34:23 -05:00
rockerBOO
fcb2ff010c Clean up some validation help documentation 2025-01-06 11:39:32 -05:00
rockerBOO
f8850296c8 Fix validate epoch, cleanup imports 2025-01-06 11:34:10 -05:00
rockerBOO
c64d1a22fc Add validate_every_n_epochs, change name validate_every_n_steps 2025-01-06 11:30:21 -05:00
rockerBOO
1c63e7cc49 Cleanup unused code and formatting 2025-01-06 11:07:47 -05:00
rockerBOO
f4840ef29e Revert train_db.py 2025-01-06 10:52:07 -05:00
rockerBOO
bbf6bbd5ea Use self.get_noise_pred_and_target and drop fixed timesteps 2025-01-06 10:48:38 -05:00
rockerBOO
1c0ae306e5 Add missing functions for training batch 2025-01-03 15:43:02 -05:00
rockerBOO
1f9ba40b8b Add step break for validation epoch. Remove unused variable 2025-01-03 15:32:07 -05:00
rockerBOO
695f38962c Move get_huber_threshold_if_needed 2025-01-03 15:25:12 -05:00
rockerBOO
0522070d19 Fix training, validation split, revert to using upstream implemenation 2025-01-03 15:20:25 -05:00
rockerBOO
6604b36044 Remove duplicate assignment 2025-01-03 02:04:59 -05:00
rockerBOO
58bfa36d02 Add seed help clarifying info 2025-01-03 02:00:28 -05:00
rockerBOO
fbfc2753eb Update text for train/reg with repeats 2025-01-03 01:53:12 -05:00
rockerBOO
c8c3569df2 Cleanup order, types, print to logger 2025-01-03 01:26:45 -05:00
rockerBOO
534059dea5 Typos and lingering is_train 2025-01-03 01:18:15 -05:00
rockerBOO
7470173044 Remove defunct code for train_controlnet.py 2025-01-03 01:13:57 -05:00
rockerBOO
d23c7322ee Merge remote-tracking branch 'hina/feature/val-loss' into validation-loss-upstream
Modified implementation for process_batch and cleanup validation
recording
2025-01-03 00:48:08 -05:00
rockerBOO
7f6e124c7c Merge branch 'gesen2egee/val' into validation-loss-upstream
Modified various implementations to restore original behavior
2025-01-02 23:04:38 -05:00
rockerBOO
449c1c5c50 Adding modified train_util and config_util 2025-01-02 15:59:20 -05:00
gesen2egee
8743532963 val 2025-01-02 15:57:12 -05:00
Hina Chen
cb89e0284e Change val latent loss compare 2024-12-28 11:57:04 +08:00
Hina Chen
64bd5317dc Split val latents/batch and pick up val latents shape size which equal to training batch. 2024-12-28 11:42:15 +08:00
Hina Chen
62164e5792 Change val loss calculate method 2024-12-27 17:28:05 +08:00
Hina Chen
05bb9183fa Add Validation loss for LoRA training 2024-12-27 16:47:59 +08:00
Kohya S
e89653975d update requirements.txt and README to include RAdamScheduleFree optimizer support 2024-12-15 19:39:47 +09:00
Kohya S.
f2d38e6cda Merge pull request #1830 from nhamanasu/sd3
Added RAdamScheduleFree support
2024-12-15 19:28:21 +09:00