Skip to content

Fix wan T2V inference with larger batchsize than 1 #580

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from

Conversation

MoayedHajiAli
Copy link

Currently Wan2 T2V inference fails on batchsize larger than 1 due to

  1. Incompatiable shape between the time conditioning and the modulation tensor
  2. A bug in the text encoder that truncate the text to the smallest text token length available in the batch.

I introduced two small fixes but I am not sure if other inference/training behaviors are also affected (e.g I2V).

@Artiprocher
Copy link
Collaborator

@MoayedHajiAli Sorry for the late reply. I'm the main developer of this open-source project, and I just graduated not long ago, so I took an extended break. I've now returned to my regular work schedule.

We truly understand your request. However, we believe that the engineering effort required to support a batch size greater than 1 is not justified by the limited benefits it brings.

On one hand, most GPUs do not have enough memory to support training with a batch size greater than 1. On the other hand, using gradient accumulation can effectively achieve the same result.

Additionally, most developers of the Wan series models we've integrated did not implement support for batch sizes larger than 1, which would require us to conduct extensive testing to enable this feature. For these reasons, we currently have no plans to support batch sizes greater than 1.

We have updated our training and inference framework, and both multi-GPU training and inference are now supported. Feel free to try them out!

Thanks for your contribution. But we have to close this PR now because of our new training framework.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants