1
$\begingroup$

I am a bit confused about how the number of parameters are calculated in Dense model for the Kera/Tensorflow.

For example, in the figure below I thought that both the statements were the same, but I found a different number of parameters for both. In particular, I am talking about model.add(Dense(...)) command.

The models in consideration

$\endgroup$
0
1
$\begingroup$

Check the documentation for Dense layer:

Note: If the input to the layer has a rank greater than 2, then Dense computes the dot product between the inputs and the kernel along the last axis of the inputs and axis 1 of the kernel (using tf.tensordot). For example, if input has dimensions (batch_size, d0, d1), then we create a kernel with shape (d1, units), and the kernel operates along axis 2 of the input, on every sub-tensor of shape (1, 1, d1) (there are batch_size * d0 such sub-tensors). The output in this case will have shape (batch_size, d0, units).

That is what happening in your first case - for input dimensions (4,1) you've got d0=4 and d1=1. So it creates a kernel of shape (1,32) that gets applied along the axis of dimension 4. That's why your output shape is (4,32) and you've got 32 weights + 32 biases = 64 parameters.

In second case you've got a "standard" 32 * 4 fully-connected weight matrix + 32 biases = 160.

$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.