0
$\begingroup$

I am looking to create a model that is able to perform binary segmentation of images with varying resolutions. For model should be able to classify tree or not tree regardless of the resolution of the input image. I have trouble finding existing work for that on the Internet. Maybe "varying resolution" is the wrong term?

Would a U-Net (or variations) be expected to cope with that given enough training data and epochs? Would you recommend something like to DINOv2 or SAM as basis?

$\endgroup$

1 Answer 1

1
$\begingroup$

Usually the solution would be just to add padding, and use a model that is trained to handle padding.
In other words, fix a resolution, and then downscale the image you are handling to fit in that area , and pad the rest (fill with zero), save the coordinate of the images, segment it, and then remove the unnecessary part (using the coordinates you saved) by removing the pad, and upscale to go back to the original resolution.

However, you can also opt to use an architecture that does not rely on the dimension, by using layers such as batch-normalization and convolutions, so that you can handle any dimension... however, this works only if also during training you do this, which causes some problem as the data cannot fit in a tensor (due to different dimensions) so the training won't be as fast

So, the first solution with the padding is faster, but maybe less accurate, the second one is slower (in training) but possibly more accurate.

Depends on your use case what tradeoff is better

$\endgroup$
5
  • $\begingroup$ I am trying to avoid any kind of downsampling. I would like to take advantage of the higher resolution if it is available. What about giving a Unet fixed sized images which represent different areas because of their varying resolution? $\endgroup$
    – cmosig
    Sep 6, 2023 at 22:37
  • $\begingroup$ @cmosig you can do that, but you have 2 problems... (1) data distribution, which you can easily fix with data augmentation, (2) you loose information... think about a square image with a circle, and divide it in a 2x2, now if you take only the top left hand piece, you cannot know if that is a arch of a circle or of something else $\endgroup$
    – Alberto
    Sep 7, 2023 at 0:29
  • $\begingroup$ @cmosig my example is somewhat extreme, but it's to show my concern... it might work, just bare in mind that might happen that along the edges you get some bad prediction since you don't have neighbors for those pixels $\endgroup$
    – Alberto
    Sep 7, 2023 at 0:30
  • $\begingroup$ good points, thanks! Regarding (1), what do you mean by data distribution specifically? (2) I know the data and the resolution will never be so bad that this case happens afaik. $\endgroup$
    – cmosig
    Sep 7, 2023 at 9:06
  • 1
    $\begingroup$ @cmosig that if the network is trained to see from 30px tall to 100px tall trees, it's not going to recognize 700px tall trees, because they are out of distribution, so you have no guarantee... thus during training, consider using random zooms on the images so that you have a wider range of sizes recognized $\endgroup$
    – Alberto
    Sep 7, 2023 at 11:17

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .