Add UniDiffuser model and pipeline#2963

Merged

sayakpaul merged 334 commits intohuggingface:mainfrom

dg845:unidiffuser-pipeline

May 26, 2023

This pull request is big! We're only showing the most recent 250 commits

Commits on May 5, 2023

Fix a bug of pano when not doing CFG (#3030)

authored and
dg845
committed
Text2video zero refinements (#3070)

19and99
authored and
dg845
committed
Release: v0.15.0

patrickvonplaten
authored and
dg845
committed
[Tests] Speed up panorama tests (#3067)

sayakpaul
authored and
dg845
committed
[Post release] v0.16.0dev (#3072)

patrickvonplaten
authored and
dg845
committed
Adds profiling flags, computes train metrics average. (#3053)

authored and
dg845
committed
[Pipelines] Make sure that None functions are correctly not saved (#3080)

patrickvonplaten
authored and
dg845
committed
doc string example remove from_pt (#3083)

yiyixuxu
authored and
dg845
committed
[Tests] parallelize (#3078)

authored and
dg845
committed
Throw deprecation warning for return_cached_folder (#3092)

patrickvonplaten
authored and
dg845
committed
Allow SD attend and excite pipeline to work with any size output images (#2835)

jcoffland
authored and
dg845
committed
[docs] Update community pipeline docs (#2989)

stevhliu
authored and
dg845
committed
Add to support Guess Mode for StableDiffusionControlnetPipleline (#2998)

takuma104
authored and
dg845
committed
fix default value for attend-and-excite (#3099)

yiyixuxu
authored and
dg845
committed
remvoe one line as requested by gc team (#3077)

yiyixuxu
authored and
dg845
committed
ddpm custom timesteps (#3007)

williamberman
authored and
dg845
committed
Fix breaking change in `pipeline_stable_diffusion_controlnet.py` (#3118)

remorses
authored and
dg845
committed
Add global pooling to controlnet (#3121)

patrickvonplaten
authored and
dg845
committed
[Bug fix] Fix img2img processor with safety checker (#3127)

patrickvonplaten
authored and
dg845
committed
[Bug fix] Make sure correct timesteps are chosen for img2img (#3128)

patrickvonplaten
authored and
dg845
committed
Improve deprecation warnings (#3131)

patrickvonplaten
authored and
dg845
committed
Fix config deprecation (#3129)

authored and
dg845
committed
feat: verfication of multi-gpu support for select examples. (#3126)

sayakpaul
authored and
dg845
committed
speed up attend-and-excite fast tests (#3079)

yiyixuxu
authored and
dg845
committed
Optimize log_validation in train_controlnet_flax (#3110)

cgarciae
authored and
dg845
committed
make style

patrickvonplaten
authored and
dg845
committed
Correct textual inversion readme (#3145)

patrickvonplaten
authored and
dg845
committed
Add unet act fn to other model components (#3136)

williamberman
authored and
dg845
committed
class labels timestep embeddings projection dtype cast (#3137)

williamberman
authored and
dg845
committed
[ckpt loader] Allow loading the Inpaint and Img2Img pipelines, while loading a ckpt model (#2705)

cmdr2
authored and
dg845
committed
add from_ckpt method as Mixin (#2318)

authored and
dg845
committed
Add TensorRT SD/txt2img Community Pipeline to diffusers along with TensorRT utils (#2974)

authored and
dg845
committed
Correct `Transformer2DModel.forward` docstring (#3074)

offchan42
authored and
dg845
committed
Update pipeline_stable_diffusion_inpaint_legacy.py (#2903)

authored and
dg845
committed
Modified altdiffusion pipline to support altdiffusion-m18 (#2993)

authored and
dg845
committed
controlnet training resize inputs to multiple of 8 (#3135)

williamberman
authored and
dg845
committed
adding custom diffusion training to diffusers examples (#3031)

authored and
dg845
committed
make style

patrickvonplaten
authored and
dg845
committed
Update custom_diffusion.mdx (#3165)

Mishig
authored and
dg845
committed
Added distillation for quantization example on textual inversion. (#2760)

XinyuYe-Intel
authored and
dg845
committed
Update Noise Autocorrelation Loss Function for Pix2PixZero Pipeline (#2942)

clarencechen
authored and
dg845
committed
[DreamBooth] add text encoder LoRA support in the DreamBooth training script (#3130)

sayakpaul
authored and
dg845
committed
Update Habana Gaudi documentation (#3169)

regisss
authored and
dg845
committed
Add model offload to x4 upscaler (#3187)

patrickvonplaten
authored and
dg845
committed
[docs] Deterministic algorithms (#3172)

stevhliu
authored and
dg845
committed
Update custom_diffusion.mdx to credit the author (#3163)

sayakpaul
authored and
dg845
committed
Fix TensorRT community pipeline device set function (#3157)

authored and
dg845
committed
make `from_flax` work for controlnet (#3161)

authored and
dg845
committed
[docs] Clarify training args (#3146)

stevhliu
authored and
dg845
committed
Multi Vector Textual Inversion (#3144)

authored and
dg845
committed
Add `Karras sigmas` to HeunDiscreteScheduler (#3160)

authored and
dg845
committed
[AudioLDM] Fix dtype of returned waveform (#3189)

sanchit-gandhi
authored and
dg845
committed
Fix bug in train_dreambooth_lora (#3183)

crywang
authored and
dg845
committed
[Community Pipelines] Update lpw_stable_diffusion pipeline (#3197)

SkyTNT
authored and
dg845
committed
Make sure VAE attention works with Torch 2_0 (#3200)

patrickvonplaten
authored and
dg845
committed
Revert "[Community Pipelines] Update lpw_stable_diffusion pipeline" (#3201)

williamberman
authored and
dg845
committed
[Bug fix] Fix batch size attention head size mismatch (#3214)

patrickvonplaten
authored and
dg845
committed
fix mixed precision training on train_dreambooth_inpaint_lora (#3138)

themrzmaster
authored and
dg845
committed
adding enable_vae_tiling and disable_vae_tiling functions (#3225)

init-22
authored and
dg845
committed
Add ControlNet v1.1 docs (#3226)

patrickvonplaten
authored and
dg845
committed
Fix issue in maybe_convert_prompt (#3188)

pdoane
authored and
dg845
committed
Sync cache version check from transformers (#3179)

ychfan
authored and
dg845
committed
Fix docs text inversion (#3166)

patrickvonplaten
authored and
dg845
committed
add model (#3230)

authored and
dg845
committed
Allow return pt x4 (#3236)

patrickvonplaten
authored and
dg845
committed
Allow fp16 attn for x4 upscaler (#3239)

patrickvonplaten
authored and
dg845
committed
fix fast test (#3241)

patrickvonplaten
authored and
dg845
committed
Adds a document on token merging (#3208)

sayakpaul
authored and
dg845
committed
[AudioLDM] Update docs to use updated ckpt (#3240)

sanchit-gandhi
authored and
dg845
committed
Release: v0.16.0

patrickvonplaten
authored and
dg845
committed
Post release for 0.16.0 (#3244)

patrickvonplaten
authored and
dg845
committed
[docs] only mention one stage (#3246)

authored and
dg845
committed
Write model card in controlnet training script (#3229)

pcuenca
authored and
dg845
committed
[2064]: Add stochastic sampler (sample_dpmpp_sde) (#3020)

authored and
dg845
committed
[Stochastic Sampler][Slow Test]: Cuda test fixes (#3257)

authored and
dg845
committed
Remove required from tracker_project_name (#3260)

pcuenca
authored and
dg845
committed
adding required parameters while calling the get_up_block and get_down_block (#3210)

authored and
dg845
committed
[docs] Update interface in repaint.mdx (#3119)

ernestchu
authored and
dg845
committed
Update IF name to XL (#3262)

authored and
dg845
committed
fix typo in score sde pipeline (#3132)

fecet
authored and
dg845
committed
Fix typo in textual inversion JAX training script (#3123)

jairtrejo
authored and
dg845
committed
AudioDiffusionPipeline - fix encode method after config changes (#3114)

teticio
authored and
dg845
committed
Revert "Revert "[Community Pipelines] Update lpw_stable_diffusion pipeline"" (#3265)

patrickvonplaten
authored and
dg845
committed
Fix community pipelines (#3266)

patrickvonplaten
authored and
dg845
committed
update notebook (#3259)

authored and
dg845
committed
[docs] add notes for stateful model changes (#3252)

authored and
dg845
committed
[LoRA] quality of life improvements in the loading semantics and docs (#3180)

authored and
dg845
committed
[Community Pipelines] EDICT pipeline implementation (#3153)

authored and
dg845
committed
[Docs]zh translated docs update (#3245)

DrDavidS
authored and
dg845
committed
Update logging.mdx (#2863)

tolgacangoz
authored and
dg845
committed
Add multiple conditions to StableDiffusionControlNetInpaintPipeline (#3125)

timegate
authored and
dg845
committed
Let's make sure that dreambooth always uploads to the Hub (#3272)

patrickvonplaten
authored and
dg845
committed
Diffedit Zero-Shot Inpainting Pipeline (#2837)

clarencechen
authored and
dg845
committed
add constant learning rate with custom rule (#3133)

jason9075
authored and
dg845
committed
Allow disabling torch 2_0 attention (#3273)

patrickvonplaten
authored and
dg845
committed
[doc] add link to training script (#3271)

authored and
dg845
committed
temp disable spectogram diffusion tests (#3278)

williamberman
authored and
dg845
committed
Changed sample[0] to images[0] (#3304)

IliaLarchenko
authored and
dg845
committed
Typo in tutorial (#3295)

IliaLarchenko
authored and
dg845
committed
Torch compile graph fix (#3286)

patrickvonplaten
authored and
dg845
committed
Postprocessing refactor img2img (#3268)

authored and
dg845
committed
[Torch 2.0 compile] Fix more torch compile breaks (#3313)

authored and
dg845
committed
fix: scale_lr and sync example readme and docs. (#3299)

sayakpaul
authored and
dg845
committed
Update stable_diffusion.mdx (#3310)

mu94-csl
authored and
dg845
committed
Fix missing variable assign in DeepFloyd-IF-II (#3315)

gitmylo
authored and
dg845
committed
Correct doc build for patch releases (#3316)

patrickvonplaten
authored and
dg845
committed
Add Stable Diffusion RePaint to community pipelines (#3320)

Markus-Pobitzer
authored and
dg845
committed
Fix multistep dpmsolver for cosine schedule (suitable for deepfloyd-if) (#3314)

authored and
dg845
committed
[docs] Improve LoRA docs (#3311)

stevhliu
authored and
dg845
committed
Added input pretubation (#3292)

isamu-isozaki
authored and
dg845
committed
Update write_own_pipeline.mdx (#3323)

csaybar
authored and
dg845
committed
update controlling generation doc with latest goodies. (#3321)

sayakpaul
authored and
dg845
committed
[Quality] Make style (#3341)

patrickvonplaten
authored and
dg845
committed
Fix config dpm (#3343)

patrickvonplaten
authored and
dg845
committed
Add the SDE variant of DPM-Solver and DPM-Solver++ (#3344)

LuChengTHU
authored and
dg845
committed
Add upsample_size to AttnUpBlock2D, AttnDownBlock2D (#3275)

will-rice
authored and
dg845
committed
Add UniDiffuser classes to __init__ files, modify transformer block to support pre- and post-LN, add fast default tests, fix some bugs.
dg845
committed
Update fast tests to use test checkpoints stored on the hub and to better match the reference UniDiffuser implementation.
dg845
committed
Fix code with make style.
dg845
committed
Revert "Fix code style with make style."
dg845
committed
Merge branch 'main' into unidiffuser-pipeline
dg845
committed
Add self.image_encoder, self.text_decoder to list of models to offload to CPU in the enable_sequential_cpu_offload(...)/enable_model_cpu_offload(...) methods to make test_cpu_offload_forward_pass p…
dg845
committed
Fix code quality with make style.
dg845
committed

Commits on May 6, 2023

Support using a data type embedding for UniDiffuser-v1.
dg845
committed

Commits on May 11, 2023

Commits on May 12, 2023

Fix shape errors for the 'joint' and 'img2text' modes.
dg845
committed

Commits on May 15, 2023

Commits on May 16, 2023

Commits on May 17, 2023

Commits on May 21, 2023

When encoding the image using the VAE, scale the image latents by the VAE's scaling factor.
dg845
committed
make style
dg845
committed
Clean up code and make slow tests pass.
dg845
committed
make fix-copies

patrickvonplaten
authored and
dg845
committed
[docs] Fix docstring (#3334)

authored and
dg845
committed
if dreambooth lora (#3360)

williamberman
authored and
dg845
committed
Postprocessing refactor all others (#3337)

authored and
dg845
committed
[docs] Improve safetensors docstring (#3368)

stevhliu
authored and
dg845
committed
add: a warning message when using xformers in a PT 2.0 env. (#3365)

authored and
dg845
committed
StableDiffusionInpaintingPipeline - resize image w.r.t height and width (#3322)

authored and
dg845
committed
make style

patrickvonplaten
authored and
dg845
committed
[docs] Adapt a model (#3326)

stevhliu
authored and
dg845
committed
[docs] Load safetensors (#3333)

authored and
dg845
committed
make style

patrickvonplaten
authored and
dg845
committed
[Docs] Fix stable_diffusion.mdx typo (#3398)

sudowind
authored and
dg845
committed
Support ControlNet v1.1 shuffle properly (#3340)

authored and
dg845
committed
[Tests] better determinism (#3374)

sayakpaul
authored and
dg845
committed
[docs] Add transformers to install (#3388)

stevhliu
authored and
dg845
committed
[deepspeed] partial ZeRO-3 support (#3076)

authored and
dg845
committed
Add omegaconf for tests (#3400)

patrickvonplaten
authored and
dg845
committed
Fix various bugs with LoRA Dreambooth and Dreambooth script (#3353)

authored and
dg845
committed
Fix docker file (#3402)

patrickvonplaten
authored and
dg845
committed
fix: deepseepd_plugin retrieval from accelerate state (#3410)

sayakpaul
authored and
dg845
committed
[Docs] Add `sigmoid` beta_scheduler to docstrings of relevant Schedulers (#3399)

authored and
dg845
committed
Don't install accelerate and transformers from source (#3415)

patrickvonplaten
authored and
dg845
committed
Don't install transformers and accelerate from source (#3414)

patrickvonplaten
authored and
dg845
committed
Improve fast tests (#3416)

patrickvonplaten
authored and
dg845
committed
attention refactor: the trilogy (#3387)

williamberman
authored and
dg845
committed
[Docs] update the PT 2.0 optimization doc with latest findings (#3370)

authored and
dg845
committed
Fix style rendering (#3433)

pcuenca
authored and
dg845
committed
unCLIP scheduler do not use note (#3417)

williamberman
authored and
dg845
committed
Replace deprecated command with environment file (#3409)

authored and
dg845
committed
fix warning message pipeline loading (#3446)

patrickvonplaten
authored and
dg845
committed
add stable diffusion tensorrt img2img pipeline (#3419)

asfiyab-nvidia
authored and
dg845
committed
Refactor controlnet and add img2img and inpaint (#3386)

patrickvonplaten
authored and
dg845
committed
[Scheduler] DPM-Solver (++) Inverse Scheduler (#3335)

authored and
dg845
committed
[Docs] Fix incomplete docstring for resnet.py (#3438)

Laurent2916
authored and
dg845
committed
fix tiled vae blend extent range (#3384)

superlabs-dev
authored and
dg845
committed
Small update to "Next steps" section (#3443)

pcuenca
authored and
dg845
committed
Allow arbitrary aspect ratio in IFSuperResolutionPipeline (#3298)

authored and
dg845
committed
Adding 'strength' parameter to StableDiffusionInpaintingPipeline (#3424)

authored and
dg845
committed
[WIP] Bugfix - Pipeline.from_pretrained is broken when the pipeline is partially downloaded (#3448)

vimarshc
authored and
dg845
committed
Fix gradient checkpointing bugs in freezing part of models (requires_grad=False) (#3404)

authored and
dg845
committed
Make dreambooth lora more robust to orig unet (#3462)

patrickvonplaten
authored and
dg845
committed
Reduce peak VRAM by releasing large attention tensors (as soon as they're unnecessary) (#3463)

cmdr2
authored and
dg845
committed
Add min snr to text2img lora training script (#3459)

wfng92
authored and
dg845
committed
Add inpaint lora scale support (#3460)

Glaceon-Hyy
authored and
dg845
committed
[From ckpt] Fix from_ckpt (#3466)

patrickvonplaten
authored and
dg845
committed
Update full dreambooth script to work with IF (#3425)

williamberman
authored and
dg845
committed
Add IF dreambooth docs (#3470)

williamberman
authored and
dg845
committed
parameterize pass single args through tuple (#3477)

williamberman
authored and
dg845
committed
attend and excite tests disable determinism on the class level (#3478)

williamberman
authored and
dg845
committed
dreambooth docs torch.compile note (#3471)

authored and
dg845
committed
add: if entry in the dreambooth training docs. (#3472)

sayakpaul
authored and
dg845
committed
[docs] Textual inversion inference (#3473)

authored and
dg845
committed
[docs] Distributed inference (#3376)

stevhliu
authored and
dg845
committed
[{Up,Down}sample1d] explicit view kernel size as number elements in flattened indices (#3479)

williamberman
authored and
dg845
committed
mps & onnx tests rework (#3449)

pcuenca
authored and
dg845
committed
[Attention processor] Better warning message when shifting to `AttnProcessor2_0` (#3457)

sayakpaul
authored and
dg845
committed
[Docs] add note on local directory path. (#3397)

authored and
dg845
committed
Refactor full determinism (#3485)

patrickvonplaten
authored and
dg845
committed
Fix DPM single (#3413)

authored and
dg845
committed
Add `use_Karras_sigmas` to DPMSolverSinglestepScheduler (#3476)

Isotr0py
authored and
dg845
committed
Adds local_files_only bool to prevent forced online connection (#3486)

w4ffl35
authored and
dg845
committed
make style

patrickvonplaten
authored and
dg845
committed
[Docs] Korean translation (optimization, training) (#3488)

authored and
dg845
committed
DataLoader respecting EXIF data in Training Images (#3465)

Ambrosiussen
authored and
dg845
committed
make style

patrickvonplaten
authored and
dg845
committed
feat: allow disk offload for diffuser models (#3285)

authored and
dg845
committed
[Community] reference only control (#3435)

okotaku
authored and
dg845
committed
Support for cross-attention bias / mask (#2634)

Birch-san
authored and
dg845
committed
do not scale the initial global step by gradient accumulation steps when loading from checkpoint (#3506)

williamberman
authored and
dg845
committed
Remove CPU latents logic for UniDiffuserPipelineFastTests.
dg845
committed
make style
dg845
committed
Revert "Clean up code and make slow tests pass."
dg845
committed
Batched load of textual inversions (#3277)

authored and
dg845
committed

Commits on May 22, 2023

Revert bad commit and clean up code.
dg845
committed

Commits on May 25, 2023