UNets with a pretrained network (like VGG16 or InceptionV3 or ResNet, or …) as the encoder portion of U-Net are common. However I'm struggling to understand how the 1D encoded second-to-last layer is transformed back to a 2D layer for the up-convs of the decoder part of the UNet.
In a normal UNet, everything is constantly 2D (+ the batch_size and the channels obviously) but the second-to-last layer of those pre-trained network encoders are usually after some flattening and fully connected layers and "lose" the notion of 2D.
Example of architecture of an encoder that makes me wonder how is that plugged in with the decoder part of the UNet:
Do we just add another FC and resize (batch_size, fc_out) to (batch_size, some_channels, width, height) with fc_out = some_channels * width * height and do the upconvs, hoping this new FC having learning parameters will do the job?
