How to add some data input in a CNN?

Question

There is this problem I have encountered, I was trying to classify the pixels from input image into classes, sort of like segmentation, using a encoder-decoder CNN. The “interested” pixels usually locate in the top right corner of the input image, but the input images are too big, which I have to slice them in patches, by doing this, each input patch loses its “which region of the whole picture it’s from” information.

I'm using pytorch, I thought of manually add this patch location info into the input, but then it will be convoluted, which does make sense to me since it's not a part of an image.

I'm new to this, not sure if I'm thinking the whole thing right, how should I manually add this info into the input correctly or if there is some keywords I can do some researches, in order to let the CNN taking position into account? Thank you.

Answer 1

If your interest is positional information, encode it!

This could include learning an embedding for each position and leveraging that in your model. You could also use an approach to hard-encode rather than learn it (kinda like adding sinusoids in the transformer paper Attention is All You Need

an example of a paper that encodes the 2D positional info: Attention Augmented Convolutional Networks