Skip to content

[Tests] add a test suite to check if loading from subfolder works as expected.#8422

Closed
sayakpaul wants to merge 10 commits intomainfrom
subfolder-tests
Closed

[Tests] add a test suite to check if loading from subfolder works as expected.#8422
sayakpaul wants to merge 10 commits intomainfrom
subfolder-tests

Conversation

@sayakpaul
Copy link
Copy Markdown
Member

What does this PR do?

To prevent things like this from happening: #8416


self.assertTrue(torch.allclose(base_output[0], new_output[0], atol=1e-5))

def test_loading_from_subfolder(self):
Copy link
Copy Markdown
Collaborator

@yiyixuxu yiyixuxu Jun 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this test will catch the error in #8416 since we only have issue when it loads from the legacy model class name,
e.g. for pixart model_class = PixArtTransformer2DModel

I left a comment here https://github.com/huggingface/diffusers/pull/7647/files#r1631917736
I think we should keep the tests with legacy class name so we will be abe to catch these errors

@sayakpaul
Copy link
Copy Markdown
Member Author

@yiyixuxu do the recent changes work for you?

(Have tested them too)

Regardless, I think having a separate suite for subfolder tests wouldn't be too bad.

@sayakpaul
Copy link
Copy Markdown
Member Author

@yiyixuxu a gentle ping here.


def test_dit_legacy_class_loading_from_subfolder(self):
ckpt_id = "facebook/DiT-XL-2-512"
transformer = Transformer2DModel.from_pretrained(ckpt_id, subfolder="transformer")
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a checkpoint where the norm_type isn't one of ada_norm_zero or ada_norm_single? I think we should test that the Transformer2DModel object is returned in that case and functions normally. Perhaps another test file test_models_transformer_2d is needed.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is a different scope. Feel free to open a PR separately.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

assert not np.allclose(image_slice, no_res_bin_image_slice, atol=1e-4, rtol=1e-4)

def test_pixart_512_without_resolution_binning(self):
def test_pixart_512_without_resolution_binning_legacy_class(self):
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we are not testing the new class here?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Being tested here:

transformer = PixArtTransformer2DModel.from_pretrained(

And here:

transformer = PixArtTransformer2DModel(

I thought that would suffice?

@sayakpaul sayakpaul requested review from DN6 and yiyixuxu July 4, 2024 01:29
@github-actions
Copy link
Copy Markdown
Contributor

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

@github-actions github-actions bot added the stale Issues that haven't received updates label Sep 14, 2024
@sayakpaul sayakpaul closed this Dec 8, 2024
@sayakpaul sayakpaul deleted the subfolder-tests branch December 8, 2024 08:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

stale Issues that haven't received updates

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants