1
$\begingroup$

I am a bit confused about how the number of parameters are calculated in Dense model for the Kera/Tensorflow.

For example, in the figure below I thought that both the statements were the same, but I found a different number of parameters for both. In particular, I am talking about model.add(Dense(...)) command.

The models in consideration

$\endgroup$
1
  • $\begingroup$ Rather than writing "Confusion in the...", could you please just put your specific question in the title? $\endgroup$ – nbro Apr 13 at 10:54
1
$\begingroup$

Check the documentation for Dense layer:

Note: If the input to the layer has a rank greater than 2, then Dense computes the dot product between the inputs and the kernel along the last axis of the inputs and axis 1 of the kernel (using tf.tensordot). For example, if input has dimensions (batch_size, d0, d1), then we create a kernel with shape (d1, units), and the kernel operates along axis 2 of the input, on every sub-tensor of shape (1, 1, d1) (there are batch_size * d0 such sub-tensors). The output in this case will have shape (batch_size, d0, units).

That is what happening in your first case - for input dimensions (4,1) you've got d0=4 and d1=1. So it creates a kernel of shape (1,32) that gets applied along the axis of dimension 4. That's why your output shape is (4,32) and you've got 32 weights + 32 biases = 64 parameters.

In second case you've got a "standard" 32 * 4 fully-connected weight matrix + 32 biases = 160.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.