I'm trying to perform a segmentation task on images of multiple sizes using fully convolutional neural networks.
Currently, I'm using efficientnet as a feature extractor, and adding a deconvolution/backwards convolution/transposed convolution layer as described in the original Fully Convolutional Networks for Semantic Segmentation paper.
But, this transposed convolution layer doesn't return an filter of size equivalent to the original image for images of varying sizes.
Lets assume the original image is 100x100, and the last layer contains filters of size 50x50. To get a filter of the same size as the original, you would need a transposed convolution layer of size 51x51.
Now assume you passed in an image of size 200x200. The last layer would contain filters of size 100x100. That same transposed convolutional filter of size 51x51 would result in an output of size 150x150.
Is there any way to make it so that a fully convolutional network always returns an image of the same size as the original?