Skip to content

Fix loading sharded checkpoints when we have variants #9061

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Aug 6, 2024
Merged

Fix loading sharded checkpoints when we have variants #9061

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
7f4b2e9
Fix loading sharded checkpoint when we have variant
SunMarc Aug 2, 2024
bb7ce33
Merge branch 'main' into shard-loading-variant
sayakpaul Aug 4, 2024
d156b8c
Merge branch 'main' into shard-loading-variant
sayakpaul Aug 6, 2024
2203525
add test
SunMarc Aug 6, 2024
6eabc33
Merge branch 'main' into shard-loading-variant
sayakpaul Aug 6, 2024
461b134
Merge branch 'shard-loading-variant' of github.com:huggingface/diffus…
SunMarc Aug 6, 2024
50bf20f
remote print
SunMarc Aug 6, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions src/diffusers/models/modeling_utils.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -773,7 +773,7 @@ def from_pretrained(cls, pretrained_model_name_or_path: Optional[Union[str, os.P
try:
accelerate.load_checkpoint_and_dispatch(
model,
model_file if not is_sharded else sharded_ckpt_cached_folder,
model_file if not is_sharded else index_file,
device_map,
max_memory=max_memory,
offload_folder=offload_folder,
Expand Down Expand Up @@ -803,7 +803,7 @@ def from_pretrained(cls, pretrained_model_name_or_path: Optional[Union[str, os.P
model._temp_convert_self_to_deprecated_attention_blocks()
accelerate.load_checkpoint_and_dispatch(
model,
model_file if not is_sharded else sharded_ckpt_cached_folder,
model_file if not is_sharded else index_file,
device_map,
max_memory=max_memory,
offload_folder=offload_folder,
Expand Down
12 changes: 12 additions & 0 deletions tests/models/unets/test_models_unet_2d_condition.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1121,6 +1121,18 @@ def test_load_sharded_checkpoint_device_map_from_hub_local_subfolder(self):
assert loaded_model
assert new_output.sample.shape == (4, 4, 16, 16)

@require_torch_gpu
def test_load_sharded_checkpoint_with_variant_from_hub(self):
_, inputs_dict = self.prepare_init_args_and_inputs_for_common()
loaded_model = self.model_class.from_pretrained(
"hf-internal-testing/unet2d-sharded-with-variant-dummy", variant="fp16"
)
loaded_model = loaded_model.to(torch_device)
new_output = loaded_model(**inputs_dict)

assert loaded_model
assert new_output.sample.shape == (4, 4, 16, 16)

@require_peft_backend
def test_lora(self):
init_dict, inputs_dict = self.prepare_init_args_and_inputs_for_common()
Expand Down
Loading