Skip to content

[IF| add set_begin_index for all IF pipelines#7577

Merged
yiyixuxu merged 1 commit intomainfrom
fix-if
Apr 5, 2024
Merged

[IF| add set_begin_index for all IF pipelines#7577
yiyixuxu merged 1 commit intomainfrom
fix-if

Conversation

@yiyixuxu
Copy link
Copy Markdown
Collaborator

@yiyixuxu yiyixuxu commented Apr 4, 2024

IF use the "quaredcos_cap_v2" beta scheduler, it is likely to see very large k-sigmas at initial steps, which translate to the initial duplicated timesteps.

I added set_begin_index for IF pipelines since we have already implemented for this exact problem in img2img and inpainting pipelines and it is very easy to extend this to text-to-image; This should fix #7214

import torch
import gc
from diffusers import DiffusionPipeline
from diffusers import DPMSolverMultistepScheduler


pipe = DiffusionPipeline.from_pretrained(
    f"DeepFloyd/IF-I-XL-v1.0",       
    variant= "fp16",
    torch_dtype= torch.float16,
    requires_safety_checker= False,
    feature_extractor= None,
    safety_checker= None,
    watermarker= None,
)
       

pipe.scheduler = DPMSolverMultistepScheduler.from_config(pipe.scheduler.config, algorithm_type='sde-dpmsolver++', use_karras_sigmas=True)

positivePrompt="an americana painting of a storm over the colorado mountains"
negativePrompt="blurry"

positivePrompt="an americana painting of a storm over the colorado mountains"
negativePrompt="blurry"

with torch.no_grad():
    positive, negative = pipe.encode_prompt(
        prompt=positivePrompt,
        negative_prompt=negativePrompt,
    )
pipe.text_encoder = None
gc.collect()
torch.cuda.empty_cache()

pipe.enable_model_cpu_offload()

images = pipe(
    prompt_embeds=positive,
    negative_prompt_embeds=negative,
    height=64,
    width=64,
    generator=torch.manual_seed(42),
    guidance_scale=7,
    num_images_per_prompt=1,
    num_inference_steps=20,
    output_type="pt",
).images

@HuggingFaceDocBuilderDev
Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@yiyixuxu yiyixuxu merged commit 6133d98 into main Apr 5, 2024
@yiyixuxu yiyixuxu deleted the fix-if branch April 5, 2024 16:54
sayakpaul pushed a commit that referenced this pull request Dec 23, 2024
add set_begin_index for all if pipelines
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

DPMSolverMultistepScheduler using karras_sigmas doesn't work anymore

2 participants