Description
Describe the bug
Hi,
For any input image and any prompt, I am continually getting green backgrounds.
I am using the CompVis/stable-diffusion-v1-4 model to generate an image to image with a prompt similar to:
Abstract pixel art of gramophone with blue background
With many different variations of strength and image sizes, the result is the same, green backgrounds.
The code is as follows:
if torch.cuda.is_available():
print("Using CUDA")
self._pipe_line = StableDiffusionImg2ImgPipeline.from_pretrained(self._model_id,cache_dir=self._cache_dir,revision="fp16", torch_dtype=torch.float16, use_auth_token=True).to(ImageSystem.DEVICE)
self._pipe_line.safety_checker = None
#self._pipe_line.enable_attention_slicing()
else:
print("Using CPU")
self._pipe_line = StableDiffusionImg2ImgPipeline.from_pretrained(self._model_id,cache_dir=self._cache_dir).to(ImageSystem.DEVICE)
generator = torch.Generator(device=ImageSystem.DEVICE).manual_seed(self._seed)
with autocast("cuda"):
result_image = self._pipe_line(
instruction,
image = image, # The starting image
strength = self._strength, # 0 for no change, 1.0 for max strength
#num_inference_steps=30
generator=generator
).images[0]
The machine has a V100 GPU with 32GB of memory. The CUDA version is:
NVIDIA-SMI 525.116.03 Driver Version: 525.116.03 CUDA Version: 12.0
Reproduction
if torch.cuda.is_available():
print("Using CUDA")
self._pipe_line = StableDiffusionImg2ImgPipeline.from_pretrained(self._model_id,cache_dir=self._cache_dir,revision="fp16", torch_dtype=torch.float16, use_auth_token=True).to(ImageSystem.DEVICE)
self._pipe_line.safety_checker = None
#self._pipe_line.enable_attention_slicing()
else:
print("Using CPU")
self._pipe_line = StableDiffusionImg2ImgPipeline.from_pretrained(self._model_id,cache_dir=self._cache_dir).to(ImageSystem.DEVICE)
generator = torch.Generator(device=ImageSystem.DEVICE).manual_seed(self._seed)
with autocast("cuda"):
result_image = self._pipe_line(
instruction,
image = image, # The starting image
strength = self._strength, # 0 for no change, 1.0 for max strength
#num_inference_steps=30
generator=generator
).images[0]
Logs
No response
System Info
The machine has a V100 GPU with 32GB of memory. The CUDA version is:
NVIDIA-SMI 525.116.03 Driver Version: 525.116.03 CUDA Version: 12.0
The OS is:
Linux version 4.18.0-425.19.2.el8_7.x86_64 (mockbuild@x86-vm-09.build.eng.bos.redhat.com) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-16) (GCC)) #1 SMP Fri Mar 17 01:52:38 EDT 2023
Who can help?
No response