Kohya S
017b82ebe3
update help message for fused_backward_pass
2024-05-06 15:05:42 +09:00
Dave Lage
8db0cadcee
Add caption_separator to output for subset
2024-05-02 18:08:28 -04:00
Dave Lage
dbb7bb288e
Fix caption_separator missing in subset schema
2024-05-02 17:39:35 -04:00
Cauldrath
040e26ff1d
Regenerate failed file
...
If a latent file fails to load, print out the path and the error, then return false to regenerate it
2024-04-21 13:46:31 -04:00
Maatra
b886d0a359
Cleaned typing to be in line with accelerate hyperparameters type resctrictions
2024-04-20 14:36:47 +01:00
Maatra
2c9db5d9f2
passing filtered hyperparameters to accelerate
2024-04-20 14:11:43 +01:00
Cauldrath
feefcf256e
Display name of error latent file
...
When trying to load stored latents, if an error occurs, this change will tell you what file failed to load
Currently it will just tell you that something failed without telling you which file
2024-04-18 23:15:36 -04:00
Zovjsra
64916a35b2
add disable_mmap to args
2024-04-16 16:40:08 +08:00
2kpr
4f203ce40d
Fused backward pass
2024-04-14 09:56:58 -05:00
Kohya S
bfb352bc43
change huber_schedule from exponential to snr
2024-04-07 21:07:52 +09:00
Kohya S
d30ebb205c
update readme, add metadata for network module
2024-04-07 14:58:17 +09:00
kabachuha
90b18795fc
Add option to use Scheduled Huber Loss in all training pipelines to improve resilience to data corruption ( #1228 )
...
* add huber loss and huber_c compute to train_util
* add reduction modes
* add huber_c retrieval from timestep getter
* move get timesteps and huber to own function
* add conditional loss to all training scripts
* add cond loss to train network
* add (scheduled) huber_loss to args
* fixup twice timesteps getting
* PHL-schedule should depend on noise scheduler's num timesteps
* *2 multiplier to huber loss cause of 1/2 a^2 conv.
The Taylor expansion of sqrt near zero gives 1/2 a^2, which differs from a^2 of the standard MSE loss. This change scales them better against one another
* add option for smooth l1 (huber / delta)
* unify huber scheduling
* add snr huber scheduler
---------
Co-authored-by: Kohya S <52813779+kohya-ss@users.noreply.github.com >
2024-04-07 13:54:21 +09:00
ykume
cd587ce62c
verify command line args if wandb is enabled
2024-04-05 08:23:03 +09:00
Disty0
4012fd24f6
IPEX fix pin_memory
2024-03-28 21:08:16 +03:00
Disty0
6f7e93d5cc
Add OpenVINO and ROCm ONNX Runtime for WD14
2024-03-27 03:21:13 +03:00
Kohya S
c86e356013
Merge branch 'dev' into dataset-cache
2024-03-26 19:43:40 +09:00
Kohya S
ab1e389347
Merge branch 'dev' into masked-loss
2024-03-26 19:39:30 +09:00
Kohya S
a2b8531627
make each script consistent, fix to work w/o DeepSpeed
2024-03-25 22:28:46 +09:00
Kohya S
c24422fb9d
Merge branch 'dev' into deep-speed
2024-03-25 22:11:05 +09:00
Kohya S
1648ade6da
format by black
2024-03-24 20:55:48 +09:00
Kohya S
993b2ab4c1
Merge branch 'dev' into deep-speed
2024-03-24 18:45:59 +09:00
Kohya S
8d5858826f
Merge branch 'dev' into masked-loss
2024-03-24 18:19:53 +09:00
Kohya S
025347214d
refactor metadata caching for DreamBooth dataset
2024-03-24 18:09:32 +09:00
Kohaku-Blueleaf
ae97c8bfd1
[Experimental] Add cache mechanism for dataset groups to avoid long waiting time for initilization ( #1178 )
...
* support meta cached dataset
* add cache meta scripts
* random ip_noise_gamma strength
* random noise_offset strength
* use correct settings for parser
* cache path/caption/size only
* revert mess up commit
* revert mess up commit
* Update requirements.txt
* Add arguments for meta cache.
* remove pickle implementation
* Return sizes when enable cache
---------
Co-authored-by: Kohya S <52813779+kohya-ss@users.noreply.github.com >
2024-03-24 15:40:18 +09:00
Kohya S
381c44955e
update readme and typing hint
2024-03-24 11:27:18 +09:00
Kohya S
ad97410ba5
Merge pull request #1205 from feffy380/patch-1
...
register reg images with correct subset
2024-03-24 11:14:07 +09:00
Kohya S
79d1c12ab0
disable sample_every_n_xxx if value less than 1 ref #1202
2024-03-24 11:06:37 +09:00
feffy380
0c7baea88c
register reg images with correct subset
2024-03-23 17:28:02 +01:00
Kohya S
f4a4c11cd3
support multiline captions ref #1155
2024-03-23 18:51:37 +09:00
Kohya S
fbb98f144e
Merge branch 'dev' into deep-speed
2024-03-20 18:15:26 +09:00
Kohya S
9b6b39f204
Merge branch 'dev' into masked-loss
2024-03-20 18:14:36 +09:00
Kohya S
855add067b
update option help and readme
2024-03-20 18:14:05 +09:00
Kohya S
bf6cd4b9da
Merge pull request #1168 from gesen2egee/save_state_on_train_end
...
Save state on train end
2024-03-20 18:02:13 +09:00
Kohya S
119cc99fb0
Merge pull request #1167 from Horizon1704/patch-1
...
Add "encoding='utf-8'" for --config_file
2024-03-20 17:39:08 +09:00
Kohya S
cf09c6aa9f
Merge pull request #1177 from KohakuBlueleaf/random-strength-noise
...
Random strength for Noise Offset and input perturbation noise
2024-03-20 16:17:16 +09:00
Kohya S
86e40fabbc
Merge branch 'dev' into deep-speed
2024-03-17 19:30:42 +09:00
Kohya S
3419c3de0d
common masked loss func, apply to all training script
2024-03-17 19:30:20 +09:00
Kohya S
7081a0cf0f
extension of src image could be different than target image
2024-03-17 18:09:15 +09:00
Kohya S
0ef4fe70f0
Merge branch 'dev' into masked-loss
2024-03-17 11:18:18 +09:00
Kohya S
0a8ec5224e
Merge branch 'main' into dev
2024-03-15 21:33:07 +09:00
Kohya S
f811b115ba
fix sdxl timestep embedding
2024-03-15 21:05:00 +09:00
kblueleaf
53954a1e2e
use correct settings for parser
2024-03-13 18:21:49 +08:00
kblueleaf
86399407b2
random noise_offset strength
2024-03-13 18:21:49 +08:00
kblueleaf
948029fe61
random ip_noise_gamma strength
2024-03-13 18:21:49 +08:00
Kohya S
74c266a597
Merge branch 'dev' into masked-loss
2024-03-12 20:40:57 +09:00
gesen2egee
095b8035e6
save state on train end
2024-03-10 23:33:38 +08:00
Horizon1704
124ec45876
Add "encoding='utf-8'"
2024-03-10 22:53:05 +08:00
Kohya S
e3ccf8fbf7
make deepspeed_utils
2024-02-27 21:30:46 +09:00
Kohya S
eefb3cc1e7
Merge branch 'deep-speed' into deepspeed
2024-02-27 18:57:42 +09:00
Kohya S
074d32af20
Merge branch 'main' into dev
2024-02-27 18:53:43 +09:00
