2
$\begingroup$

How can we compute number of weights considering a convolutional neural network that is used to classify images into two classes :

  • INPUT: 100x100 gray-scale images.
  • LAYER 1: Convolutional layer with 60 7x7 convolutional filters (stride=1, valid padding).
  • LAYER 2: Convolutional layer with 100 5x5 convolutional filters (stride=1, valid padding).
  • LAYER 3: A max pooling layer that down-samples Layer 2 by a factor of 4 (e.g., from 500x500 to 250x250)
  • LAYER 4: Dense layer with 250 units
  • LAYER 5: Dense layer with 200 units
  • LAYER 6: Single output unit

Assume the existence of biases in each layer. Moreover, pooling layer has a weight (similar to AlexNet)

How many weights does this network have?

Keras Code

import keras
from keras.models import Sequential
from keras.layers import Dense
from keras.layers import Conv2D, MaxPooling2D

model = Sequential()

# Layer 1
model.add(Conv2D(60, (7, 7), input_shape = (100, 100, 1), padding="same", activation="relu"))

# Layer 2
model.add(Conv2D(100, (5, 5), padding="same", activation="relu"))

# Layer 3
model.add(MaxPooling2D(pool_size=(2, 2)))

# Layer 4
model.add(Dense(250))

# Layer 5
model.add(Dense(200))

model.summary()
$\endgroup$
  • $\begingroup$ see stackoverflow.com/a/35827171/4886927 $\endgroup$ – pasaba por aqui Dec 14 '19 at 19:44
  • $\begingroup$ @pasabaporaqui I am facing problems with lasgane , theano so i prefer solve this question theoretically rather than with code , if fix this would be nice . AMD GPU's aren't supported I think . $\endgroup$ – estamos Dec 14 '19 at 21:03
2
$\begingroup$

Calculating parameter number in a CNN is very straightforward.

CNN is composed of different filters which is essentially a 3d tensor. CNN weights are shared meaning they are used multiple times, and reused in different locations. Each layer have n tensors each with dimension w * h * c where w = width, h = height, c = channels (the input channel size), therefore getting param_no = w * h * c * n. There is also a bias for each output channel, so param_no_bias = n. At the end the parameter number is calculated with: n * w * h * c + n. See more about this in hear: Article

Pooling layer does not have a weight, it only have hyperparameters. You may have confused the two. There are hyperparameters for thje stride, the factor and etc. These are predefined and not trainable.

For a keras API, @pasabaporaqui have mentioned https://stackoverflow.com/a/35827171/4886927 which definitely works.

Hope I can help you.

| improve this answer | |
$\endgroup$
  • $\begingroup$ That's exactly what an exercise in my assignment says . I also thought that a pooling layer can't have a weight . I wrote some Keras code , if you find any error you are welcome to refer it . $\endgroup$ – estamos Dec 15 '19 at 15:19
  • $\begingroup$ I don't think there is any error, to my knowledge at least. Though this network seems to have too few layers and also the filters are way to big. You should start with a well-known backbone first like ResNet and adjust the output size to 2, and also with the advantage of having pretrained imageNet weights. ResNet-50 is a good starting point as it is not that computational intensive yet providing a great performance. For kerad see this for details. keras.io/applications . Hope I can help you @estamos . $\endgroup$ – Clement Hui Dec 16 '19 at 8:22
  • $\begingroup$ Btw you can accept an answer by clicking the tick button, you can also accept answers on your other post. $\endgroup$ – Clement Hui Dec 16 '19 at 8:23
  • $\begingroup$ I implemented this Keras code only to solve one problem of my assignment , so the poor performance it's not a problem since number of weights are computed right . $\endgroup$ – estamos Dec 16 '19 at 8:51
  • 1
    $\begingroup$ Oh I see. Then the code should work fine. $\endgroup$ – Clement Hui Dec 17 '19 at 9:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.