I have implemented a CNN for image classification. I have not used fully connected layers, but only a softmax. Still, I am getting results.
Must I use fully-connected layers in a CNN?
Artificial Intelligence Stack Exchange is a question and answer site for people interested in conceptual questions about life and challenges in a world where "cognitive" functions can be mimicked in purely digital environment. It only takes a minute to sign up.Sign up to join this community
The reasons people use the FC after convolutional layer is that CNN preserves spatial information. You said you use softmax, so you probably make some classification task. If you don't use FC layer, then you probably evaluate first class by first position of the first kernel, not by whole image with all kernels, second class by second position of the kernel and so on.
The dense net combine the info from all the kernels in all positions.
That said, you technically can convert FC to Conv net, as described here, so then you can said you "skipped" FC layer.
In theory, you do not need fully-connected (FC) layers. FC layers are used to introduce scope for updating weights during back-propagation, due to its ability to introduce more connectivity possibilities, as every neuron of the FC is connected every neuron of the further layers.
Are fully connected layers necessary in a CNN?
No. In fact, you can simulate a fully connected layer with convolutions. A convolutional neural network (CNN) that does not have fully connected layers is called a fully convolutional network (FCN). See this answer for more info.
An example of an FCN is the u-net, which does not use any fully connected layers, but only convolution, downsampling (i.e. pooling), upsampling (deconvolution), and copy and crop operations. Nevertheless, u-net is used to classify pixels (more precisely, semantic segmentation).
Moreover, you can use CNNs only for the purpose of feature extraction, and then feed these extracted features in another classifier (e.g. an SVM). In fact, transfer learning is based on the idea that CNNs extract reusable features.