I have implemented a CNN for image classification. I have not used fully connected layers, but only a softmax. Still, I am getting results.
Must I use fully-connected layers in a CNN?
Artificial Intelligence Stack Exchange is a question and answer site for people interested in conceptual questions about life and challenges in a world where "cognitive" functions can be mimicked in purely digital environment. It only takes a minute to sign up.Sign up to join this community
The reasons people use the FC after convolutional layer is that CNN preserves spatial information. You said you use softmax, so you probably make some classification task. If you don't use FC layer, then you probably evaluate first class by first position of the first kernel, not by whole image with all kernels, second class by second position of the kernel and so on.
The dense net combine the info from all the kernels in all positions.
That said, you technically can convert FC to Conv net, as described here, so then you can said you "skipped" FC layer.
In theory, you do not need fully-connected (FC) layers. FC layers are used to introduce scope for updating weights during back-propagation, due to its ability to introduce more connectivity possibilities, as every neuron of the FC is connected every neuron of the further layers.