Skip to content

On branch ControlNet-v1-1 #5951

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
May 22, 2023
Merged

On branch ControlNet-v1-1 #5951

merged 3 commits into from
May 22, 2023

Conversation

GoooHi
Copy link
Contributor

@GoooHi GoooHi commented May 18, 2023

  1. 新增权重 control_v11e_sd15_ip2p 和 对应文件 gradio_ip2p2image.py
  2. 新增权重 control_v11e_sd15_shuffle 和 对应文件 gradio_shuffle2image.py

合并代码至https://github.com/PaddlePaddle/PaddleNLP/tree/36591ab297dfa45d89255aa6e14988940ccea299/ppdiffusers/examples/controlnet

Changes to be committed:
	new file:   ../../ppdiffusers/examples/controlnet/gradio_ip2p2image.py
	new file:   ../../ppdiffusers/examples/controlnet/gradio_mlsd2image.py
@paddle-bot
Copy link

paddle-bot bot commented May 18, 2023

Thanks for your contribution!

@CLAassistant
Copy link

CLAassistant commented May 18, 2023

CLA assistant check
All committers have signed the CLA.

@JunnYu JunnYu self-requested a review May 18, 2023 07:26
@JunnYu
Copy link
Member

JunnYu commented May 18, 2023

@Submerge-Gu 很感谢你的贡献,我这里给你留了几个comment,麻烦改一下。还有的话能不能像这里,添加一下对应的文档介绍呢?
image

@codecov
Copy link

codecov bot commented May 18, 2023

Codecov Report

Merging #5951 (ae9dba9) into develop (dd11b56) will decrease coverage by 0.01%.
The diff coverage is n/a.

@@             Coverage Diff             @@
##           develop    #5951      +/-   ##
===========================================
- Coverage    62.36%   62.35%   -0.01%     
===========================================
  Files          491      491              
  Lines        69268    69280      +12     
===========================================
+ Hits         43197    43201       +4     
- Misses       26071    26079       +8     

see 3 files with indirect coverage changes

	modified:   ppdiffusers/examples/controlnet/README.md
	modified:   ppdiffusers/examples/controlnet/gradio_ip2p2image.py
	modified:   ppdiffusers/examples/controlnet/gradio_mlsd2image.py
@GoooHi
Copy link
Contributor Author

GoooHi commented May 18, 2023

@JunnYu 您好,请问我这个失败是由于啥原因呢?

	modified:   ppdiffusers/examples/controlnet/README.md
	new file:   ppdiffusers/examples/controlnet/annotator/shuffle/__init__.py
	modified:   ppdiffusers/examples/controlnet/annotator/util.py
	renamed:    ppdiffusers/examples/controlnet/gradio_mlsd2image.py -> ppdiffusers/examples/controlnet/gradio_shuffle2image.py
@JunnYu
Copy link
Member

JunnYu commented May 19, 2023

model_name_list = [
    # "lllyasviel/control_v11u_sd15_tile", 存在问题
    "https://huggingface.co/lllyasviel/control_v11e_sd15_ip2p",
    "https://huggingface.co/lllyasviel/control_v11f1e_sd15_tile",
    "https://huggingface.co/lllyasviel/control_v11e_sd15_shuffle",
    "https://huggingface.co/lllyasviel/control_v11p_sd15_softedge",
    "https://huggingface.co/lllyasviel/control_v11p_sd15_scribble",
    "https://huggingface.co/lllyasviel/control_v11p_sd15s2_lineart_anime",
    "https://huggingface.co/lllyasviel/control_v11p_sd15_lineart",
    "https://huggingface.co/lllyasviel/control_v11p_sd15_seg",
    "https://huggingface.co/lllyasviel/control_v11p_sd15_normalbae",
    "https://huggingface.co/lllyasviel/control_v11f1p_sd15_depth",
    "https://huggingface.co/lllyasviel/control_v11p_sd15_mlsd",
    "https://huggingface.co/lllyasviel/control_v11p_sd15_inpaint",
    "https://huggingface.co/lllyasviel/control_v11p_sd15_canny",
]

当前我已经转换了这么多权重,你看看有没有什么新的 anatator 部分也可以加进来?

@GoooHi
Copy link
Contributor Author

GoooHi commented May 19, 2023

model_name_list = [
    # "lllyasviel/control_v11u_sd15_tile", 存在问题
    "https://huggingface.co/lllyasviel/control_v11e_sd15_ip2p",
    "https://huggingface.co/lllyasviel/control_v11f1e_sd15_tile",
    "https://huggingface.co/lllyasviel/control_v11e_sd15_shuffle",
    "https://huggingface.co/lllyasviel/control_v11p_sd15_softedge",
    "https://huggingface.co/lllyasviel/control_v11p_sd15_scribble",
    "https://huggingface.co/lllyasviel/control_v11p_sd15s2_lineart_anime",
    "https://huggingface.co/lllyasviel/control_v11p_sd15_lineart",
    "https://huggingface.co/lllyasviel/control_v11p_sd15_seg",
    "https://huggingface.co/lllyasviel/control_v11p_sd15_normalbae",
    "https://huggingface.co/lllyasviel/control_v11f1p_sd15_depth",
    "https://huggingface.co/lllyasviel/control_v11p_sd15_mlsd",
    "https://huggingface.co/lllyasviel/control_v11p_sd15_inpaint",
    "https://huggingface.co/lllyasviel/control_v11p_sd15_canny",
]

当前我已经转换了这么多权重,你看看有没有什么新的 anatator 部分也可以加进来?

@JunnYu anatator中需要转换的权重有:
annotator/lineart中的sk_model.pth和sk_model2.pth;
annotator/lineart_anime中的netG.pth;
annotator/pidinet中的table5_pidinet.pth;
我对这几个模型不是很了解,还在摸索怎么尝试转换。您看下呢?

@JunnYu
Copy link
Member

JunnYu commented May 19, 2023

@GoooHi
Copy link
Contributor Author

GoooHi commented May 19, 2023

我看这个模型很简单的,你可以试着转成Paddle?https://github.com/lllyasviel/ControlNet-v1-1-nightly/blob/main/annotator/lineart/__init__.py 这个也可以。 https://github.com/lllyasviel/ControlNet-v1-1-nightly/blob/main/annotator/lineart_anime/__init__.py

@JunnYu 我现在正在尝试转成Paddle。之前没有类似的开发经验,目前现有的可以参考的资料也比较少。

Copy link
Member

@JunnYu JunnYu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@JunnYu
Copy link
Member

JunnYu commented May 22, 2023

你好,当前的PR我就先合入了,之后的新功能你可以重新开启一个新PR~

@JunnYu JunnYu merged commit 3a9f5fd into PaddlePaddle:develop May 22, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants