1
$\begingroup$

I have a neural network model defined as below. How many layers exist there? Not sure which ones to count when we are asked about the number.

def create_model():
    channels = 3
    model = Sequential()
    model.add(Conv2D(32, kernel_size = (5, 5), activation='relu', input_shape=(IMAGE_SIZE, IMAGE_SIZE, channels)))
    model.add(MaxPooling2D(pool_size=(2,2)))
    model.add(BatchNormalization())
    model.add(Conv2D(64, kernel_size=(3,3), activation='relu'))
    model.add(MaxPooling2D(pool_size=(2,2)))
    model.add(BatchNormalization())
    model.add(Conv2D(128, kernel_size=(3,3), activation='relu'))
    model.add(MaxPooling2D(pool_size=(2,2)))
    model.add(BatchNormalization())
    model.add(Conv2D(256, kernel_size=(3,3), activation='relu'))
    model.add(MaxPooling2D(pool_size=(2,2)))
    model.add(BatchNormalization())

    model.add(Conv2D(64, kernel_size=(3,3), activation='relu'))
    model.add(MaxPooling2D(pool_size=(2,2)))
    model.add(BatchNormalization())
    model.add(Dropout(0.2))
    model.add(Flatten())
    model.add(Dense(256, activation='relu'))
    model.add(Dropout(0.2))
    model.add(Dense(128, activation='relu'))
    model.add(Dense(2, activation = 'softmax'))

    return model
$\endgroup$
  • 1
    $\begingroup$ I think the answer to this question is indeed subjective because some people may or not consider the activations, dropout and batch normalization separate layers. However, convolutions, pooling and dense are almost always considered standalone layers. You can also use model.layers to get the layers of the network, according to Keras. $\endgroup$ – nbro Jan 27 at 21:38
  • $\begingroup$ Exactly. My impression was I should just count CONV layers! $\endgroup$ – Tina J Jan 27 at 22:11
  • $\begingroup$ If this is a homework question, I would provide some comments regarding this issue. By the way, you may also say that it may depend on the implementation. For example, in Keras, the activations may or not always be implemented as a separate layer (note that I actually don't know if that's the case or not)! $\endgroup$ – nbro Jan 27 at 22:31
  • $\begingroup$ If its a subjective question, how about you list all the layers i.e, 5 Convolutional Layers, 2 Dense Layers and so on. Another widely used practice is to define the network with blocks where a Convolutional Block will include Conv2D, MaxPooling, BatchNormalizarion and Flatten. However, if you want a number according to how the network/graph will be structured, then number of layers only includes Conv and Dense. $\endgroup$ – Sharan Jan 28 at 5:27
3
$\begingroup$

tl;dr I'd say your model has 8 layers (5 conv, 3 dense), however a lot of people count layers in other ways. From what I've seen this is by far the most conventional way for counting layers.


Justification

This is an interesting question because its quite subjective. In most cases only the convolutional and dense layers would count from your network. Bach norm, dropout and flatten are usually considered as operations to other layers rather than layers of their own (much like activation functions).

Note: It is debatable whether or not pooling layers are considered to be layers (as they have no trainable parameters) but in most cases they are not considered to be so.

Note 2: Batch norm, on the other hand, isn't usually considered a layer even though it has trainable parameters. Clearly the authors didn't introduce it as a layer, but as a way to normalize, shift and scale the inputs of a layer. This is apparent in some of the examples below which don't count batch norm as an actual layer.

Note 3: Conventionally all networks are considered to have [at least] one input layer but it doesn't count as a layer.


Examples

Some examples that follow this reasoning when counting layers are the following. I'll also write the pooling layers in each, but they clearly aren't considered as layers be the authors. When available I'll also write the number of layers that keras registers from their official implementations:

The ResNet-50 architecture has 50 layers (49 conv, 2 pool, 1 dense), however keras registers it as 177 layers. ResNets also use batch normalization after each convolution (so 49 batch norms in total), but clearly don't count them as layers.

The Resnet-34 has 34 layers (33 conv, 2 pool, 1 dense). Like the previous, this also uses batch norm but doesn't count it.

VGG-19 has 19 layers (16 conv, 5 pool 3 dense). Keras registers this as 26 layers.

AlexNet is considered to have 8 layers (5 conv, 3 pool, 3 dense).

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.