1
$\begingroup$

In object detection, we can resize images by keeping the ratio the same as the original image, which is often known as "letterbox" resize.

My questions are

  1. Why do we need to resize images? If we resize images to have all the same dimensions, given that some original images are too long vertically or horizontally, we will lose a lot of features in those images.

  2. If the "letterbox" method is better than "normal resize" (i.e. without keeping the aspect ratio, e.g. the result of the application of OpenCV's resize function with the parameter interpolation set to cv2.INTER_AREA), why don't people apply it in the classification task?

Improve this question
$\endgroup$
0
2
$\begingroup$

There are different questions and even different lines of thought here. Let's go through them

On resizing

  • Why do we need to resize? To fit the network input which is fixed when nets are no Fully Convolutional Networks (FCN)
  • What if my net is FCN? Still makes sense to resize to bound the dimension of the input features you want to detect (a person on a small image VS big image). Take into account that the kernel sizes do not vary although the image size does.

On keeping aspect ratio (or letterbox as some people like to say)

  • Why to keep aspect ratio? This is more of a philosophical question. It is believed that keeping the aspect ratio helps the nets to learn the natural variability in object sizes (say a person bounding box can not be super hight and super thin, because that would be a street light).

  • Why not to keep aspect ratio? If you resize without keeping the aspect ratio and the aspect ratio distortion is not mega super very huge, the networks will still learn. In other words, if your input images don't have crazy aspect ratios, then there is no difference between adding or not a bit of distortion. In fact, sometimes it will even act as a regularization or augmentation.

Conlusion

As long as your application is not too specific and your input images aspect ratios are bounded (this is, if you train with images from any regular camera), you should not worry too much about this.

When to worry about this? When you train with huge vertical or horizontal images or if you train with images taken from some very specific devices like geophisical, radio, or optical sensors. In these cases you should pay special attention on how to resize or split an image. For example, with a recording of a radio sensor, if you resize with aspect ratio deformation, a wave from a specific frequency would transform to another because of the sine wave warp)

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thank you for your answer, I agree with you at the conclusion, I am trying something like that right now. However, I am not sure when you mention "philosophical" in a technical problem, I think it should be explained by mathematics or at least, logical way. Let see this answer, I will give you a vote, but not the correct answers. $\endgroup$ – CuCaRot Nov 7 '20 at 4:23
  • 2
    $\begingroup$ Thanks and no prob man, I write for the community. But just for the record, I work in applied research on AI and most of the research and product we build are first based on some philosophical assumptions, then we model those assumptions in math, and then we run the math in python. AI is still a very open field, that is why I love it, so there is not always a closed answer for everything. $\endgroup$ – JVGD Nov 7 '20 at 10:20
  • $\begingroup$ I agree, all of the deep learning ideas that I knew came from the philosophical assumptions, you are doing an amazing job, I will try solving in that direction in the next project. $\endgroup$ – CuCaRot Nov 7 '20 at 12:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.