I am trying to replicate the fully convolutional networks (FCN) concept described here for semantic segmentation. It seems people have successfully trained such models by removing fully connected layers from the popular pre-trained models such as VGG and adding upsampling and unpooling layers.
I understand that transpose convolution and unpooling in upsampling layers provide counterparts of convolution and max (or average) pooling in earlier downsampling layers respectively, but what are the counterparts for non-linearities such as ReLU? What about dropout? There seems to be no discussion of this in the video.