2
$\begingroup$

I'm trying to train an image segmentation model on satellite images. There are two main issues: first, not all of the images are the same size. My understanding is that by using a fully convolutional neural network, I can feed in images of any size, so this should not be an issue. The second problem is that the images are not rectangular. They are stored as rectangular arrays, but there are nodata values within the arrays to fill in the gaps. For example, one of the images looks like this: A non-rectangular satellite image.

Ideally, I'd like the model to ignore any nodata pixels, producing an image with the exact same shape as the input image. I have two ideas on how to accomplish this, but I feel like there is probably a better way:

  1. Include a "nodata" class in the training labels and teach the model to output nodata pixels if there are nodata pixels in the input. I don't like this because it requires the model to learn another class and will affect the training.
  2. Feed the satellite image to the model in windows, skipping any windows that contain nodata pixels. This isn't ideal either because it would mean information is lost at the edges of the windows and would probably require a lot more memory.

I see that Keras has a Masking layer, but it appears that this is intended for LSTM type models, not images. Would this work with a CNN? Or, is there a better way to solve this problem that I've missed so far?

$\endgroup$
1
  • $\begingroup$ How does it handle the edge of a square image? $\endgroup$
    – user253751
    Apr 25, 2023 at 20:20

1 Answer 1

1
$\begingroup$

I'm running into the same issue. Zero padding is common in the machine learning world to deal with conforming to the architecture dimensions, so I suppose changing nodata values to 0's and feeding that into the model could be a potential solution, though I'm concerned the model would learn to just ignore that part of the image in every image it sees. Not a problem if that pattern of nodata is consistent with all your images, but that's rarely the case with remotely sensed data. Have you figured out another solution since posting this?

$\endgroup$
2
  • $\begingroup$ Hi, welcome to the forum! This seems to better fit as a comment on original question, rather than an answer. $\endgroup$ Sep 4, 2023 at 0:23
  • $\begingroup$ I agree, but I'm not allowed to comment yet. ¯_(ツ)_/¯ $\endgroup$ Sep 5, 2023 at 1:02

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .