Fix panorama to support all schedulers

[Isotr0py]

• Refactor some code and support all schedulers including PNDMScheduler for panorama pipeline.

Similar to #3499, make all schedulers sync with blocks sampling. This can solve most of the issues about failed images generation in panorama pipeline.

Test code

import torch
from diffusers import StableDiffusionPanoramaPipeline, PNDMScheduler

seed = 76
model_ckpt = "stabilityai/stable-diffusion-2-base"
prompt = "a photo of the dolomites"

pipe = StableDiffusionPanoramaPipeline.from_pretrained(model_ckpt, torch_dtype=torch.float16).to("cuda")
pipe.scheduler = PNDMScheduler.from_config(pipe.scheduler.config)
pipe.to("cuda")
generator = torch.Generator(device="cuda").manual_seed(seed)
image = pipe(prompt, generator=generator, num_inference_steps=25,width=1024,height=512).images[0]
display(image)

Results

PNDMScheduler

UniPCMultistepScheduler

KDPM2DiscreteScheduler

DEISMultistepScheduler

[HuggingFaceDocBuilderDev]

The documentation is not available anymore as the PR was closed or merged.

[patrickvonplaten]

Super nice PR! Good job @Isotr0py . @sayakpaul I'll let you review this one

[sayakpaul]

Nice, this is an elegant solution to deal with stateful schedulers like PNDM!

Can we also fix this test case then?

diffusers/tests/pipelines/stable_diffusion/test_stable_diffusion_panorama.py

Line 174 in 8e69708

def test_stable_diffusion_panorama_pndm(self):

[sayakpaul]

Also Cc: @omerbt

[sayakpaul]

Awesome!