2
$\begingroup$

There is this problem I have encountered, I was trying to classify the pixels from input image into classes, sort of like segmentation, using a encoder-decoder CNN. The “interested” pixels usually locate in the top right corner of the input image, but the input images are too big, which I have to slice them in patches, by doing this, each input patch loses its “which region of the whole picture it’s from” information.

I'm using pytorch, I thought of manually add this patch location info into the input, but then it will be convoluted, which does make sense to me since it's not a part of an image.

I'm new to this, not sure if I'm thinking the whole thing right, how should I manually add this info into the input correctly or if there is some keywords I can do some researches, in order to let the CNN taking position into account? Thank you.

$\endgroup$
2
  • $\begingroup$ Is your problem that the tiling scheme is losing positional information (where it is in the image) or losing contextual information (whats in the image that wasn't in the tile)? $\endgroup$
    – mshlis
    Commented May 28, 2019 at 1:28
  • $\begingroup$ @mshlis The first one, where it is in the image, sorry I didn't express it clearly. $\endgroup$
    – kdlsw
    Commented May 29, 2019 at 8:36

1 Answer 1

1
$\begingroup$

If your interest is positional information, encode it!

This could include learning an embedding for each position and leveraging that in your model. You could also use an approach to hard-encode rather than learn it (kinda like adding sinusoids in the transformer paper Attention is All You Need

an example of a paper that encodes the 2D positional info: Attention Augmented Convolutional Networks

$\endgroup$
0

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .