Convolutional Neural Networks for different-sized Source and Target

Question

CNNs are often used in one of the following scenarios:

1. A known-sized image is encoded to an intermediate format for later use
2. An intermediate or precursor format is decoded into a known-sized image
3. An image is converted into a same-size image

(Usually 3 is done by sticking together 1 and 2.)

Are there any papers dealing with convolutional techniques where the image sizes vary? Not only would the size of input X differ from input Y, but also input X may differ from output Y. The total amount of variation can probably be constrained by the statistics of the dataset, but knowledge of input X does not grant a priori knowledge of the size of output Y

(Masking is an obvious solution, but I am hoping for something more elegant if research already exists. The problem domain need not be images.)

Answer 1

Yes actually. There have been quite a few different adaptations to convolutional neural networks to do precisely what you are describing.

Here is an earlier one. See section 3.2.5

Here He et al. create a method known as Spatial Pyramid Pooling(SPP). In this method, you are able construct a fixed length representation regardless of input size by pooling them into "spatial bins" which are proportional to the input size, and thusly do not need to modify input dimensions.

There are a few newer methods that improve on this in various capacities, as usual, your solution will be dependent on the problem and other situation-dependent constraints. I suggest you dig deeper into the literature to find the optimal solution in your case.