Add support for Classifier Free Guidance in Network

[tanayag]

The paper says, "We follow classifier-free guidance and train our models with conditioning dropout: conditional inputs are set to 0 for 10% of training time."

This means during 10% of the training time, only Person UNet is to be trained, without any cross-attention, or self attention or anything in conditional inputs.

Maybe I am not familiar with the concept but how would it work without RGB-agnostic images, or how 6 channels would be passed? Do we make values 0 for RGB agnostic images? Any comments are welcome.

[rsamf]

It sounds like they use Dropout where the elements inside the conditioning input tensors are set to zero for all tensors $I_a, J_p, I_c, J_g$. We may be able to simply use the pytorch droupout for each of these tensors

[rsamf]

Maybe something like: https://github.com/kailashahirwar/tryondiffusion/compare/main...heddleai:tryondiffusion:rsamf-cond-dropout?expand=1

[tanayag]

Yeah, this might be correct, but I am not sure. I am keeping this issue on hold, let's do it without classifier free guidance first and then we'll check the results, and integrate classifier free guidance accordingly.

[rsamf]

Oh, we are already doing it with classifier free guidance (CFG) which is a fancy term for saying we are guiding the model output without an image classifier and, instead, with conditional inputs (examples: text, depth img, canny img, clothing-agnostic img, pose embeddings). In this case, we have the 4 conditions, clothing-agnostic img, segmented garment img, person pose embeddings, and garment pose embeddings.

[tanayag]

You are correct, I read it and it makes sense. We can do this.

Maybe something like: https://github.com/kailashahirwar/tryondiffusion/compare/main...heddleai:tryondiffusion:rsamf-cond-dropout?expand=1

[rsamf]

You are correct, I read it and it makes sense. We can do this.

Maybe something like: https://github.com/kailashahirwar/tryondiffusion/compare/main...heddleai:tryondiffusion:rsamf-cond-dropout?expand=1

Just checking back on existing issues. Should I make a pull request for the dropout?