0
$\begingroup$

I'm creating a neural network with 3 layers and no bias.

On internet I saw that the expression for the derivative of the weights between the hidden layer and the output layer was:

$$\Delta W_{j,k} = (o_k - t_k) \cdot f'\left[\sum_j (W_{j,k} \ \cdot o_j)\right] \cdot o_j,$$

where $t$ is the target output, $o$ is the activated output layer and $f'$ the derivative of the activation function.

But the shape of these weights is $\text{output nodes}\times\text{hidden nodes}$, and $\text{hidden nodes}$ can be bigger than $\text{output nodes}$, so the formula is wrong because of I'm taking $o_k$ and $o$ has length $\text{output nodes}$.

  1. In simple terms, what is the right formula for updating these weights?

  2. Also, what is the right formula for updating the weights between the input layer and the hidden layer?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
0
$\begingroup$
  1. The correct formula for updating the weights between the hidden layer and the output layer is: $$\Delta W_{j,k} = h_k \ \cdot \ o'_{j} \ \cdot \ (o_j - t_j),$$ where $h$ is the activated hidden layer and $o'$ is the derivative of output layer.
    I found this formula in the book Artificial Intelligence: A Guide to Intelligent Systems by Michael Negnevitsky.
Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .