1
$\begingroup$

I was thinking about training a neural network to colourize images. The input would be the luminosity/value for each pixel, and the output would be a hue and/or saturation. Training data would be easily obtained just by selecting the luminosity/value channel from a full colour image.

Suppose all channels are scaled to 0.0-1.0, there is a problem with pixels whose hue is nearly 0.0 or nearly 1.0.

  • The input data may have sharp discontinuities in hue which are not visible to the human eye. This is an unstable, illusory boundary which seems like it would destablilize the training.
  • Also if the network outputs a value of 1.001 instead of 0.001 then this should NOT be penalised since.

Possible workarounds might be to preprocess the image to remap e.g. 0.99 to -0.01 if that pixel is near a region dominated by near-0 hues, or similarly to to remap e.g. 0.01 to 1.01 if that pixel is near a region dominated by near-1 hues. This has its own problems. Similarly, outputs could be wrapped to the range 0-1 before being scored.

But is there a better way to encode cyclic values such as hue so that they will naturally be continuous?

One solution I thought of would be to treat (hue,saturation) as a (theta,r) polar coordinate and translate this to Cartesian (x,y) and have that be the training target, but I don't know how this change of colour space will affect things (it might be find, I haven't tried it yet).

Are there alternative colour representations which are better suited to machine learning?

$\endgroup$
1
$\begingroup$

Your solution is pretty much on spot. It corresponds to the YUV scheme used in television and designed to match human perception characteristics. As you already noticed, such an encoding wouldn't suffer from discontinuities.

$\endgroup$
1
  • $\begingroup$ And its cousin YCbCr. Which apparently in some contexts means the same thing. $\endgroup$ – NikoNyrh May 27 '20 at 21:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.