Disty0
a6a2b5a867
Fix IPEX support and add XPU device to device_utils
2024-01-31 17:32:37 +03:00
Kohya S
2ca4d0c831
Merge pull request #1054 from akx/mps
...
Device support improvements (MPS)
2024-01-31 21:30:12 +09:00
Disty0
988dee02b9
IPEX torch.tensor FP64 workaround
2024-01-30 01:52:32 +03:00
Disty0
ccc3a481e7
Update IPEX Libs
2024-01-28 14:14:31 +03:00
Kohya S
c576f80639
Fix ControlNetLLLite training issue #1069
2024-01-25 18:43:07 +09:00
Aarni Koskela
478156b4f7
Refactor device determination to function; add MPS fallback
2024-01-23 14:29:03 +02:00
Aarni Koskela
afc38707d5
Refactor memory cleaning into a single function
2024-01-23 14:28:50 +02:00
Aarni Koskela
2e4bee6f24
Log accelerator device
2024-01-23 14:20:40 +02:00
Kohya S
bea4362e21
Merge pull request #1060 from akx/refactor-xpu-init
...
Deduplicate ipex initialization code
2024-01-23 20:25:37 +09:00
Kohya S
696dd7f668
Fix dtype issue in PyTorch 2.0 for generating samples in training sdxl network
2024-01-22 12:43:37 +09:00
Kohya S
fef172966f
Add network_multiplier for dataset and train LoRA
2024-01-20 16:24:43 +09:00
Kohya S
5a1ebc4c7c
format by black
2024-01-20 13:10:45 +09:00
Kohya S
1f77bb6e73
fix to work sample generation in fp8 ref #1057
2024-01-20 10:57:42 +09:00
Kohaku-Blueleaf
9cfa68c92f
[Experimental Feature] FP8 weight dtype for base model when running train_network (or sdxl_train_network) ( #1057 )
...
* Add fp8 support
* remove some debug prints
* Better implementation for te
* Fix some misunderstanding
* as same as unet, add explicit convert
* better impl for convert TE to fp8
* fp8 for not only unet
* Better cache TE and TE lr
* match arg name
* Fix with list
* Add timeout settings
* Fix arg style
* Add custom seperator
* Fix typo
* Fix typo again
* Fix dtype error
* Fix gradient problem
* Fix req grad
* fix merge
* Fix merge
* Resolve merge
* arrangement and document
* Resolve merge error
* Add assert for mixed precision
2024-01-20 09:46:53 +09:00
Aarni Koskela
6f3f701d3d
Deduplicate ipex initialization code
2024-01-19 18:07:36 +02:00
Aarni Koskela
ef50436464
Fix typo --spda (it's --sdpa)
2024-01-16 14:32:48 +02:00
Kohya S
09ef3ffa8b
Merge branch 'main' into dev
2024-01-14 21:49:25 +09:00
Kohya S
aab265e431
Fix an issue with saving as diffusers sd1/2 model close #1033
2024-01-04 21:43:50 +09:00
Kohya S
716bad188b
Update dependencies ref #1024
2024-01-04 19:53:25 +09:00
Kohya S
07bf2a21ac
Merge pull request #1024 from p1atdev/main
...
Add support for `torch.compile`
2024-01-04 10:49:52 +09:00
Nir Weingarten
ab716302e4
Added cli argument for wandb session name
2024-01-03 11:52:38 +02:00
Disty0
b9d2181192
Cleanup
2024-01-02 11:51:29 +03:00
Disty0
49148eb36e
Disable Diffusers slicing if device is not XPU
2024-01-02 11:50:08 +03:00
Disty0
479bac447e
Fix typo
2024-01-01 12:51:23 +03:00
Disty0
15d5e78ac2
Update IPEX Libs
2024-01-01 12:44:26 +03:00
Plat
62e7516537
feat: support torch.compile
2023-12-27 02:17:24 +09:00
Kohya S
2186e417ba
fix size of bucket < min_size ref #1008
2023-12-20 22:12:21 +09:00
Kohya S
1519e3067c
Merge pull request #1008 from Cauldrath/zero_height_error
...
Fix zero height buckets
2023-12-20 22:09:04 +09:00
Kohya S
35e5424255
Merge pull request #1007 from Disty0/dev
...
IPEX fix SDPA
2023-12-20 21:53:11 +09:00
Cauldrath
f8360a4831
Fix zero height buckets
...
If max_size is too large relative to max_reso, it will calculate a height of zero for some buckets.
This causes a crash later when it divides the width by the height.
This change also simplifies some math and consolidates the redundant "size" variable into "width".
2023-12-19 18:35:09 -05:00
Disty0
8556b9d7f5
IPEX fix SDPA
2023-12-19 22:59:06 +03:00
Kohya S
3efd90b2ad
fix sampling in training with mutiple gpus ref #989
2023-12-15 22:35:54 +09:00
Disty0
aff05e043f
IPEX support for Torch 2.1 and fix dtype erros
2023-12-13 19:40:38 +03:00
Kohya S
d309a27a51
change option names, add ddp kwargs if needed ref #1000
2023-12-13 21:02:26 +09:00
Kohya S
471d274803
Merge pull request #1000 from Isotr0py/dev
...
Fix multi-gpu SDXL training
2023-12-13 20:52:11 +09:00
Kohya S
35f4c9b5c7
fix an error when keep_tokens_separator is not set ref #975
2023-12-12 21:43:21 +09:00
Kohya S
034a49c69d
Merge pull request #975 from Linaqruf/dev
...
Add keep_tokens_separator as alternative for keep_tokens
2023-12-12 21:28:32 +09:00
Isotr0py
bb5ae389f7
fix DDP SDXL training
2023-12-12 19:58:44 +08:00
Kohya S
4a2cef887c
fix lllite training not working ref #913
2023-12-10 09:23:37 +09:00
Kohya S
42750f7846
fix error on pool_workaround in sdxl TE training ref #994
2023-12-10 09:18:33 +09:00
Isotr0py
db84530074
Fix gradients synchronization for multi-GPUs training ( #989 )
...
* delete DDP wrapper
* fix train_db vae and train_network
* fix train_db vae and train_network unwrap
* network grad sync
---------
Co-authored-by: Kohya S <52813779+kohya-ss@users.noreply.github.com >
2023-12-07 22:01:42 +09:00
Kohya S
72bbaac96d
Merge pull request #985 from Disty0/dev
...
Update IPEX hijacks
2023-12-07 21:39:24 +09:00
Kohya S
5713d63dc5
add temporary workaround for playground-v2
2023-12-06 23:08:02 +09:00
Disty0
dd7bb33ab6
IPEX fix torch.UntypedStorage.is_cuda
2023-12-05 22:18:47 +03:00
Disty0
a9c6182b3f
Cleanup IPEX libs
2023-12-05 19:52:31 +03:00
Disty0
3d70137d31
Disable IPEX attention if the GPU supports 64 bit
2023-12-05 19:40:16 +03:00
Disty0
bce9a081db
Update IPEX hijacks
2023-12-05 14:17:31 +03:00
Kohya S
46cf41cc93
Merge pull request #961 from rockerBOO/attention-processor
...
Add attention processor
2023-12-03 21:24:12 +09:00
Kohya S
81a440c8e8
Merge pull request #955 from xzuyn/paged_adamw
...
Add PagedAdamW
2023-12-03 21:22:38 +09:00
Kohya S
f24a3b5282
show seed in generating samples
2023-12-03 21:15:30 +09:00
