2
$\begingroup$

During image preprocessing pipeline, should one rescale each pixel value to [0, 1] by dividing 255 first, and then perform data transformation such as color distortion, gaussian blur? or vice versa?

I believe for correctness, it may depend on the particular image transformation algorithm, or if you use some libraries. Is there any general advice? If anyone has experience trying both before or after, please share, particularly if you use external library or framework that may make implicit assumption to the value range of the pixel.

$\endgroup$
1
$\begingroup$

So far, it seems this is more a software "integration" issue. One great tip from http://karpathy.github.io/2019/04/25/recipe/ is to visualize everything as often as you can during development. For data augmentation, try to visualize the image right before it enters your convnet.

What I found is a bug can happen if your particular image transform library try to detect if you have rescaled or not by just checking the data type. i.e. if it sees float, it will assume [0, 1] or if an integer, it will assume [0, 255].

This can end badly if you resize (without rescaling at the same time) using tf.image.resize(...). Resizing an image (e.g. to 224x224x3) has the "side effect" of converting the pixel value to a float, while still having an (approx.) range of 0.0-255.0. This is such that a downstream library will see a float and make the wrong assumption of [0, 1] range and totally mess up the transform. If you use any color related transforms, definitely be careful, they are usually dependent on pixel value range assumption.

Because of this, if you use tf.image, you may find it safer to do resize and rescale at the same time to ensure that float => [0, 1] such that your downstream color augmentation library may most likely work. Or you just have to manage the type carefully and do tf.cast(...) wherever necessary to satisfy any library assumption.

At the end, a human eye will still have to verify them.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.