2
$\begingroup$

There is this problem I have encountered, I was trying to classify the pixels from input image into classes, sort of like segmentation, using a encoder-decoder CNN. The “interested” pixels usually locate in the top right corner of the input image, but the input images are too big, which I have to slice them in patches, by doing this, each input patch loses its “which region of the whole picture it’s from” information.

I'm using pytorch, I thought of manually add this patch location info into the input, but then it will be convoluted, which does make sense to me since it's not a part of an image.

I'm new to this, not sure if I'm thinking the whole thing right, how should I manually add this info into the input correctly or if there is some keywords I can do some researches, in order to let the CNN taking position into account? Thank you.

|improve this question
$\endgroup$
  • $\begingroup$ Is your problem that the tiling scheme is losing positional information (where it is in the image) or losing contextual information (whats in the image that wasn't in the tile)? $\endgroup$ – mshlis May 28 at 1:28
  • $\begingroup$ @mshlis The first one, where it is in the image, sorry I didn't express it clearly. $\endgroup$ – kdlsw May 29 at 8:36
1
$\begingroup$

If your interest is positional information, encode it!

This could include learning an embedding for each position and leveraging that in your model. You could also use an approach to hard-encode rather than learn it (kinda like adding sinusoids in the transformer paper Attention is All You Need

an example of a paper that encodes the 2D positional info: Attention Augmented Convolutional Networks

|improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.