DPM-Solver Multistep Inverse Scheduler

[clarencechen]

Model/Pipeline/Scheduler description

As zero-shot image editing and interpolation techniques based on T2I diffusion models continue to proliferate, the use of deterministic inverse schedulers for deriving noise maps from user-provided images continues to grow. Implementing a higher-order inverse scheduler will help speed up the latent inversion step in image manipulation pipelines like Pix-to-Pix Zero, DiffEdit, SVDiff, and Text-to-Video Zero. I've already opened a PR at #3335 with a draft implementation.

Open source status

• The model implementation is available
• The model weights are available (Only relevant if addition is not a scheduler).

Provide useful links for the implementation

Here is an ad-hoc implementation of an inverse DPM-Solver Multistep scheduler based on the CompVis Stable Diffusion library by @Xiang-cd: https://github.com/Xiang-cd/DiffEdit-stable-diffusion

[bonlime]

@clarencechen very interesting PR and idea, i have few questions, maybe you could answer them.

1. Does using DPM++(2M) only speeds up the inversion or also makes it more stable for higher guidance scale? from my experience using cfg > 3 for DDIM inversion results in very bad reconstruction, and I wonder if using more advanced way fixes this problem?
2. Do i understand correctly that after the invert we can use any scheduler?

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.