Cauldrath
feefcf256e
Display name of error latent file
...
When trying to load stored latents, if an error occurs, this change will tell you what file failed to load
Currently it will just tell you that something failed without telling you which file
2024-04-18 23:15:36 -04:00
Kohya S
bfb352bc43
change huber_schedule from exponential to snr
2024-04-07 21:07:52 +09:00
Kohya S
d30ebb205c
update readme, add metadata for network module
2024-04-07 14:58:17 +09:00
kabachuha
90b18795fc
Add option to use Scheduled Huber Loss in all training pipelines to improve resilience to data corruption ( #1228 )
...
* add huber loss and huber_c compute to train_util
* add reduction modes
* add huber_c retrieval from timestep getter
* move get timesteps and huber to own function
* add conditional loss to all training scripts
* add cond loss to train network
* add (scheduled) huber_loss to args
* fixup twice timesteps getting
* PHL-schedule should depend on noise scheduler's num timesteps
* *2 multiplier to huber loss cause of 1/2 a^2 conv.
The Taylor expansion of sqrt near zero gives 1/2 a^2, which differs from a^2 of the standard MSE loss. This change scales them better against one another
* add option for smooth l1 (huber / delta)
* unify huber scheduling
* add snr huber scheduler
---------
Co-authored-by: Kohya S <52813779+kohya-ss@users.noreply.github.com >
2024-04-07 13:54:21 +09:00
ykume
cd587ce62c
verify command line args if wandb is enabled
2024-04-05 08:23:03 +09:00
Disty0
4012fd24f6
IPEX fix pin_memory
2024-03-28 21:08:16 +03:00
Disty0
6f7e93d5cc
Add OpenVINO and ROCm ONNX Runtime for WD14
2024-03-27 03:21:13 +03:00
Kohya S
c86e356013
Merge branch 'dev' into dataset-cache
2024-03-26 19:43:40 +09:00
Kohya S
ab1e389347
Merge branch 'dev' into masked-loss
2024-03-26 19:39:30 +09:00
Kohya S
a2b8531627
make each script consistent, fix to work w/o DeepSpeed
2024-03-25 22:28:46 +09:00
Kohya S
c24422fb9d
Merge branch 'dev' into deep-speed
2024-03-25 22:11:05 +09:00
Kohya S
1648ade6da
format by black
2024-03-24 20:55:48 +09:00
Kohya S
993b2ab4c1
Merge branch 'dev' into deep-speed
2024-03-24 18:45:59 +09:00
Kohya S
8d5858826f
Merge branch 'dev' into masked-loss
2024-03-24 18:19:53 +09:00
Kohya S
025347214d
refactor metadata caching for DreamBooth dataset
2024-03-24 18:09:32 +09:00
Kohaku-Blueleaf
ae97c8bfd1
[Experimental] Add cache mechanism for dataset groups to avoid long waiting time for initilization ( #1178 )
...
* support meta cached dataset
* add cache meta scripts
* random ip_noise_gamma strength
* random noise_offset strength
* use correct settings for parser
* cache path/caption/size only
* revert mess up commit
* revert mess up commit
* Update requirements.txt
* Add arguments for meta cache.
* remove pickle implementation
* Return sizes when enable cache
---------
Co-authored-by: Kohya S <52813779+kohya-ss@users.noreply.github.com >
2024-03-24 15:40:18 +09:00
Kohya S
381c44955e
update readme and typing hint
2024-03-24 11:27:18 +09:00
Kohya S
ad97410ba5
Merge pull request #1205 from feffy380/patch-1
...
register reg images with correct subset
2024-03-24 11:14:07 +09:00
Kohya S
79d1c12ab0
disable sample_every_n_xxx if value less than 1 ref #1202
2024-03-24 11:06:37 +09:00
feffy380
0c7baea88c
register reg images with correct subset
2024-03-23 17:28:02 +01:00
Kohya S
f4a4c11cd3
support multiline captions ref #1155
2024-03-23 18:51:37 +09:00
Kohya S
fbb98f144e
Merge branch 'dev' into deep-speed
2024-03-20 18:15:26 +09:00
Kohya S
9b6b39f204
Merge branch 'dev' into masked-loss
2024-03-20 18:14:36 +09:00
Kohya S
855add067b
update option help and readme
2024-03-20 18:14:05 +09:00
Kohya S
bf6cd4b9da
Merge pull request #1168 from gesen2egee/save_state_on_train_end
...
Save state on train end
2024-03-20 18:02:13 +09:00
Kohya S
119cc99fb0
Merge pull request #1167 from Horizon1704/patch-1
...
Add "encoding='utf-8'" for --config_file
2024-03-20 17:39:08 +09:00
Kohya S
cf09c6aa9f
Merge pull request #1177 from KohakuBlueleaf/random-strength-noise
...
Random strength for Noise Offset and input perturbation noise
2024-03-20 16:17:16 +09:00
Kohya S
86e40fabbc
Merge branch 'dev' into deep-speed
2024-03-17 19:30:42 +09:00
Kohya S
3419c3de0d
common masked loss func, apply to all training script
2024-03-17 19:30:20 +09:00
Kohya S
7081a0cf0f
extension of src image could be different than target image
2024-03-17 18:09:15 +09:00
Kohya S
0ef4fe70f0
Merge branch 'dev' into masked-loss
2024-03-17 11:18:18 +09:00
Kohya S
0a8ec5224e
Merge branch 'main' into dev
2024-03-15 21:33:07 +09:00
Kohya S
f811b115ba
fix sdxl timestep embedding
2024-03-15 21:05:00 +09:00
kblueleaf
53954a1e2e
use correct settings for parser
2024-03-13 18:21:49 +08:00
kblueleaf
86399407b2
random noise_offset strength
2024-03-13 18:21:49 +08:00
kblueleaf
948029fe61
random ip_noise_gamma strength
2024-03-13 18:21:49 +08:00
Kohya S
74c266a597
Merge branch 'dev' into masked-loss
2024-03-12 20:40:57 +09:00
gesen2egee
095b8035e6
save state on train end
2024-03-10 23:33:38 +08:00
Horizon1704
124ec45876
Add "encoding='utf-8'"
2024-03-10 22:53:05 +08:00
Kohya S
e3ccf8fbf7
make deepspeed_utils
2024-02-27 21:30:46 +09:00
Kohya S
eefb3cc1e7
Merge branch 'deep-speed' into deepspeed
2024-02-27 18:57:42 +09:00
Kohya S
074d32af20
Merge branch 'main' into dev
2024-02-27 18:53:43 +09:00
Kohya S
f2c727fc8c
add minimal impl for masked loss
2024-02-26 23:19:58 +09:00
Kohya S
577e9913ca
add some new dataset settings
2024-02-26 20:01:25 +09:00
Kohya S
fccbee2727
revert logging #1137
2024-02-25 10:43:14 +09:00
Yuta Hayashibe
5d5f39b6e6
Replaced print with logger
2024-02-25 01:24:11 +09:00
BootsofLagrangian
4d5186d1cf
refactored codes, some function moved into train_utils.py
2024-02-22 16:20:53 +09:00
Kohya S
d1fb480887
format by black
2024-02-18 09:13:24 +09:00
Kohya S
baa0e97ced
Merge branch 'dev' into dev_device_support
2024-02-17 11:54:07 +09:00
Kohya S
93bed60762
fix to work --console_log_xxx options
2024-02-12 14:49:29 +09:00
