Skip to content

[schedulers] hanlde dtype in add_noise#767

Merged
patil-suraj merged 4 commits intomainfrom
handle-dtype-add-noise
Oct 7, 2022
Merged

[schedulers] hanlde dtype in add_noise#767
patil-suraj merged 4 commits intomainfrom
handle-dtype-add-noise

Conversation

@patil-suraj
Copy link
Copy Markdown
Contributor

Make sure that the alphas/sigmas have the same dtype as the input samples.

@HuggingFaceDocBuilderDev
Copy link
Copy Markdown

HuggingFaceDocBuilderDev commented Oct 7, 2022

The documentation is not available anymore as the PR was closed or merged.

Copy link
Copy Markdown
Contributor

@patrickvonplaten patrickvonplaten left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@kashif
Copy link
Copy Markdown
Contributor

kashif commented Oct 7, 2022

@patil-suraj should i add tests for fp16 schedulers?

Copy link
Copy Markdown
Member

@NouamaneTazi NouamaneTazi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM :)

Copy link
Copy Markdown
Member

@NouamaneTazi NouamaneTazi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe that

        if self.alphas_cumprod.dtype != original_samples.dtype:
            self.alphas_cumprod = self.alphas_cumprod.to(original_samples.dtype)

is equivalent to just doing

self.alphas_cumprod = self.alphas_cumprod.to(original_samples.dtype)

Same for device. And that could cut off the lines of code quite a bit

@patil-suraj
Copy link
Copy Markdown
Contributor Author

Removed the if as suggested by @NouamaneTazi ! The vae and pipeline shouldn't have been modified in the PR, removed those from here and will address suggestions related to that in #769

@patil-suraj patil-suraj merged commit ec831b6 into main Oct 7, 2022
@patil-suraj patil-suraj deleted the handle-dtype-add-noise branch October 7, 2022 14:44
@pcuenca
Copy link
Copy Markdown
Member

pcuenca commented Oct 7, 2022

I believe that

        if self.alphas_cumprod.dtype != original_samples.dtype:
            self.alphas_cumprod = self.alphas_cumprod.to(original_samples.dtype)

is equivalent to just doing

self.alphas_cumprod = self.alphas_cumprod.to(original_samples.dtype)

Same for device. And that could cut off the lines of code quite a bit

According to this comment it may have a penalty https://github.com/huggingface/diffusers/pull/511/files#r987896215. Is there any way we can now for sure?

patrickvonplaten pushed a commit that referenced this pull request Oct 11, 2022
* handle dtype in vae and image2image pipeline

* handle dtype in add noise

* don't modify vae and pipeline

* remove the if
prathikr pushed a commit to prathikr/diffusers that referenced this pull request Oct 26, 2022
* handle dtype in vae and image2image pipeline

* handle dtype in add noise

* don't modify vae and pipeline

* remove the if
yoonseokjin pushed a commit to yoonseokjin/diffusers that referenced this pull request Dec 25, 2023
* handle dtype in vae and image2image pipeline

* handle dtype in add noise

* don't modify vae and pipeline

* remove the if
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants