The finetuned SDXL shows bad performance on Pokeman

[fiona-lxd]

Hello, I just git clone the repo and conduct the training along the guidance here https://github.com/huggingface/diffusers/blob/main/examples/text_to_image/README_sdxl.md where I finetuned the SDXL on Pokeman dataset with script train_text_to_image_sdxl.py. However, the generated images are bad-looking. For example,
A beautiful girl.

A pokemon with green eyes and red legs.

The results are really bad-looking.

[bghira]

what parameters?

[fiona-lxd]

what parameters?

The default one here https://github.com/huggingface/diffusers/blob/main/examples/text_to_image/README_sdxl.md . I changed nothing. Is there any parameters that I should be cautious about?

[bghira]

let's review.

--pretrained_vae_model_name_or_path=$VAE_NAME \

I am hoping you used madebyollin's VAE.

--enable_xformers_memory_efficient_attention \

This is harmless.

--resolution=512 --center_crop --random_flip \

Training at a resolution of 512x512 can be destructive. I had some significant damage when training at 256x256 using the default parameters of the Pix2Pix training script. However, training at 768x768 seems to be really beneficial.

--proportion_empty_prompts=0.2 \

20% caption dropout is really high, @sayakpaul was that intentional? .1 is the value in use by SAI.

in fact, it might be best to use zero dropout for the pokemon example, maybe?

--train_batch_size=1
--gradient_accumulation_steps=4 --gradient_checkpointing \

A batch size of 1 is not great with AdamW8Bit. It might be all that you can manage with your VRAM pool. But, could you try higher values?

--max_train_steps=10000 \

This seems really really high for that small of a training set.

--use_8bit_adam \

This is fine.

--learning_rate=1e-06 --lr_scheduler="constant" --lr_warmup_steps=0 \

@sayakpaul are you sure you wanted 1e-06 for this? that seems very high.

I would test with 4e-7 especially on a batch size of 1.

--mixed_precision="fp16" \

This could be a bad idea. bf16 is a safer choice for numeric stability, especially given that we use OpenCLIP G here.

--validation_prompt="a cute Sundar Pichai creature" --validation_epochs 5 \

Try validating every epoch.

--checkpointing_steps=5000 \

Checkpoint more frequently.

I don't see --use_ema in here, which increases VRAM use but majorly improves training results.

[fiona-lxd]

Thank you a lot for the reply and the suggestion. I have started the training again with the hyper-parameters below (vae is madebyollin's VAE):

--enable_xformers_memory_efficient_attention
--resolution=768 \ # it was 512 before
--center_crop
--random_flip
--proportion_empty_prompts=0. \ # it was 0.2 before
--train_batch_size=1
--gradient_accumulation_steps=1 \ # I used 4 A100, therefore, this can be reduced from 4 to 1
--gradient_checkpointing
--max_train_steps=2500 \ # Since we use 4 A100, which means batch-size=4, therefore, the train-step is reduced to 1/4. Actually, I have viewed all the generated images every 500 step, but they are all bad-looking.
--learning_rate=4e-07 \ # Changed from 1e-6
--lr_scheduler=constant
--lr_warmup_steps=0
--mixed_precision=bf16 \ # changed from fp16
--use_ema \ # Added new here
--validation_prompt=a_cute_Sundar_Pichai_creature
--validation_epochs 1
--checkpointing_steps=500
I will update this result once the experiment is done.

[bghira]

you want to use as high of a batch size as you can. for your 768px training you should be able to achieve a batch size of 10. using gradient accumulation is good too. leave that as 4.

with each GPU at effective batch size of 10 * 4, with 4 of them, you have effective bsz of 160.

[bghira]

FWIW, within 500 steps at 1e-6 without EMA the model is obliterated. this is why you had bad results.

[fiona-lxd]

Thank you for the reminder. I'm still waiting for the computational resources. I will update this one I got the results. BTW, do you have any suggestions on finetuning the SDXL on 10 0000 images? Is the hyper-parameter same with the one trained for Pokeman?

[fiona-lxd]

OK, here is the result along the training steps. (0, 100, 500, 1000, 1500, 2000 ... , the validation prompt is a_cute_Sundar_Pichai_creature). The batch size is 1 (I cannot install the 8bit adam successfully) and the gradient accumulation is 4. I used 4 GPUs. Therefore, the effective batchsize is 16. I used the fp16 instead of bf16. I found this code has error when using bf16. Specifically, in accelerator.backward(loss), the error "RuntimeError: Expected tensor for argument #1 'grad_output' to have the same type as tensor for argument #2 'weight'; but type CUDABFloat16Type does not equal torch.cuda.HalfTensor (while checking arguments for cudnn_convolution_backward_input)" occurred. Though I have put both the unet and all the inputs to weight_dtype, this cannot resolve the issue.
Anyway, based on the above settings, the results become better than before. But it seems that the model didn't learn to draw the pokeman style. I don't think the model is learning what we designed. Should I degrade the learning rate?

[bghira]

is that with zero caption dropout? in that case it seems that the value is needed to be non-zero.

--proportion_empty_prompts=0. \ # it was 0.2 before

this should probably be put back around 20% to ensure style transfer.

[fiona-lxd]

Ok, I will try proportion_empty_prompts=0.2.

[fiona-lxd]

OK, here are the results. The left is SD1.5 finetuned on Pokeman under the guidance of https://github.com/huggingface/diffusers/blob/main/examples/text_to_image/README.md. The model has learned to generate images in Pokeman's style. In contrast, the middle (proportion_empty_prompts=0.0) the right one (proportion_empty_prompts0.2) are generated by finetuning the SDXL. They all failed.

[George0726]

I confront the same issue. It seems that SDXL is really hard to fine-tune.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.