Check the documentation for
Note: If the input to the layer has a rank greater than 2, then Dense computes the dot product between the inputs and the kernel along the last axis of the inputs and axis 1 of the kernel (using tf.tensordot). For example, if input has dimensions (batch_size, d0, d1), then we create a kernel with shape (d1, units), and the kernel operates along axis 2 of the input, on every sub-tensor of shape (1, 1, d1) (there are batch_size * d0 such sub-tensors). The output in this case will have shape (batch_size, d0, units).
That is what happening in your first case - for input dimensions
(4,1) you've got
d1=1. So it creates a kernel of shape
(1,32) that gets applied along the axis of dimension 4. That's why your output shape is
(4,32) and you've got 32 weights + 32 biases = 64 parameters.
In second case you've got a "standard" 32 * 4 fully-connected weight matrix + 32 biases = 160.