How to Represent a Multi-Feature Input Vector in a Neural Net

Question

I'm finding myself confused everytime when I want to visualize a Neural net with input vectors and weights. Given a input like this:

Input 1	Input 2	Input 3	Output
1	2	3	1
3	4	6	0
6	8	1	1

Assuming that I have 1 hidden layer with 3 neurons and 2 output neurons, I would like to know if my understanding of this simple neural net in a Matrix notation is correct?

X = \begin{bmatrix}1&2&3\\3&4&6\\6&8&1\end{bmatrix}

My weights Matrix would then look like:

W = \begin{bmatrix}W_{11}&W_{12}&W_{13}\\W_{21}&W_{22}&W_{23}\end{bmatrix}

Assuming the bias to be 0, my multi-variate summation equation for Neuron 1 in the hidden layer will look like (in a Matrix dot product representation):

First I guess I cannot multiply both the Matrices as it does not satisfy the rule where number or columns from Matrix 1 (in my case Matrix X) should equal the number of rows from Matrix 2 (my weights Matrix w). Correct?

So my questions are now:

1. Should I then choose my hidden layer to have 3 neurons?

2. Is my overall understanding of the whole concept correct?

Answer 1

1. You can choose the number of neurons you want for your hidden layer. You have to try in order to see which number of neurons makes the neural network work better.

2. It seems you missed something:

You will need two weights matrices, one for the hidden layer and one for the output layer.

Without the bias, the hidden layer weights matrix, will have 3 rows because there are 3 input features, and 3 columns because there are 3 hidden neurons. So you can multiply it with the input matrix you named X.

It will result in another 3x3 matrix, we can name X_hidden.

The output layer weights matrix, will have 3 rows, as there are also 3 inputs for this layer: the 3 outputs of the hidden layer. And it will have only 2 columns, because there are 2 output neurons.

So you also can multiply it with the X_hidden matrix. It will result in a 3 rows, and 2 columns matrix.

About neuron 1 in the hidden layer, its matrix will have 1 column because we consider only one neuron, and 3 rows because of the 3 input features (without bias). So you can multiply it with the X matrix that have 3 columns.

It will result in a 3 rows, and 1 column matrix.