debug EMAModel.from_pretrained()

[chenguolin]

What does this PR do?

model_cls.load_config() is modified to model_cls.from_config().

When setting return_unused_kwargs=True, for model_cls.load_config(), the returned unused kwargs are referred to as unused input args, instead of unused args for model_cls initialization. So the returned ema_kwargs are always empty ({}).

model_cls.from_config() will return the real unused args for model initialization, which are the expected ema_kwargs.

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline?
• Did you read our philosophy doc (important for complex PRs)?
• Was this discussed/approved via a GitHub issue or the forum? Please add a link to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?

Who can review?

General functionalities: @sayakpaul @yiyixuxu @DN6

[sayakpaul]

Could you add a test for this here? https://github.com/huggingface/diffusers/blob/main/tests/others/test_ema.py

[chenguolin]

Could you add a test for this here? https://github.com/huggingface/diffusers/blob/main/tests/others/test_ema.py

Hi @sayakpaul , how can I add a test here? I'm not familiar with this.

[sayakpaul]

Add a test case like

diffusers/tests/others/test_ema.py

Line 146 in 9a92b81

def test_serialization(self):

that accounts for the changes you introduced in this PR?

[chenguolin]

This PR only affects training and resuming with EMAModel and also requires loading a JSON file, so this bug might not be that obvious and easy to write a test case.

Let's say you have a file named config.json:

{
    "decay": 0.9999,
    "inv_gamma": 1.0
}

You can run:

from diffusers import UNet2DConditionModel

model_cls = UNet2DConditionModel
path = "config.json"

_, ema_kwargs = model_cls.from_config(path, return_unused_kwargs=True)
print(ema_kwargs)  # Output: {"decay": 0.9999, "inv_gamma": 1.0}: ✅

_, ema_kwargs = model_cls.load_config(path, return_unused_kwargs=True)
print(ema_kwargs)  # Output: {}: ❌

[sayakpaul]

Okay, we can add this as a test case then maybe?

[chenguolin]

OK, thank you 😊. You could consider merging this PR then. I have tested it in my own training process, and this bug has confused me for quite some time.

[sayakpaul]

Sorry I think you misunderstood. I am suggesting you to add a test case to demonstrate the use case you mentioned in #9809 (comment).

[chenguolin]

OK. I tried.

But I found it's not that easy to write a test_from_pretrained() for EMAModelTests, as EMAModel.from_pretrained() requires loading both a model checkpoint and the corresponding config json file including EMA args (such as decay and optimization_step), which are not available in "hf-internal-testing/tiny-stable-diffusion-pipe".

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[SahilCarterr]

i already opened a PR for this #9779 with the test case @sayakpaul @chenguolin

[chenguolin]

i already opened a PR for this #9779 with the test case @sayakpaul @chenguolin

It looks good :)