I have been reading up on CNNs. One of the different confusing things has been that people always talk of normalization layers. A common normalization layer is a ReLU layer. But I never encountered an explanation of why all of a sudden, activation functions become their own layers in CNNs, while they are only parts of a fully connected layer in MLPs.
What is the reason for having dedicated activation layers in CNNs rather than applying the activation to the output volume of a convolutional layer as part of the convolutional layer, as it is the case for dense layers in MLPs?
I guess, in the end, there is no functional difference. We could just as well have separate activation layers in MLPs rather than activation functions in their fully connected layers. But this difference in the convention is irritating still. Well, assuming it only is an artifact of the convention.