2
$\begingroup$

My neural network takes an image as an input and outputs another image. It's the generator of a cycleGAN.

I would like to add (to the discriminator loss, the cycle consistency loss and the identity loss) a colour consistency loss i.e. i want the output image to globally have the same colours than the input image.

Why? My problem is that the network "tints" my images too much in my opinion (a standard example is the following: my dataset has women that wear pink clothes more than men do and when i transform men into women, the GAN also transforms the clothes, and sort of adds a pink tint to the whole image). I know that adding this loss will encourage the network to "not change anything" though as any change would be a change in colours. Unless my loss looks at the averages of red, blue and green instead of looking at them pixel by pixel, which is what I'd like to go for.

Not the main question but any thoughts on that are appreciated: any idea about how to implement it in Pytorch, or if it's already been implemented? Searches for colour consistency on GANs just give me GANs that take black and white photos and add colour to them.

Improve this question
$\endgroup$
5
  • $\begingroup$ Can you please put your specific question in the title? "Colour consistency loss neural networks" is not a question. If you're asking for code, that would be off-topic here. $\endgroup$
    – nbro
    Apr 4, 2022 at 9:10
  • $\begingroup$ I'm asking if it makes sense and if there are some references of people that did it :) I edited my post thanks, sorry for not posting it correctly! $\endgroup$
    – IHopeItWontBeAStupidQuestion
    Apr 4, 2022 at 9:23
  • $\begingroup$ Ok, so you should edit your post to clarify that "Any idea about how to implement it in Pytorch, or if it's already been implemented?" is not your main question. $\endgroup$
    – nbro
    Apr 4, 2022 at 9:27
  • $\begingroup$ Done! In the case that I'd want help on that specifically, stackoverflow is the stackexchange I should go to? $\endgroup$
    – IHopeItWontBeAStupidQuestion
    Apr 4, 2022 at 13:57
  • $\begingroup$ Yes, I think Stack Overflow would be a better option for programming questions. $\endgroup$
    – nbro
    Apr 4, 2022 at 14:22

1 Answer 1

Reset to default
2
$\begingroup$

The identity loss should already fulfill for what you're asking for, which means that if the problem is still there even with a strong weight for it then you should consider improving/cleaning up your data to remove the bias you observe.

If you really don't like this idea, something close to what you're asking for are histogram based losses, but be aware that they are mostly used in neural style transfer, so you will probably have to remove the identity loss to avoid conflicting losses and I wouldn't expect them to work as you think in preventing color biases, since they come from the data.

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .