0
$\begingroup$

I'm creating a neural network with 3 layers and no bias.

On internet I saw that the expression for the derivative of the weights between the hidden layer and the output layer was:

$$\Delta W_{j,k} = (o_k - t_k) \cdot f'\left[\sum_j (W_{j,k} \ \cdot o_j)\right] \cdot o_j,$$

where $t$ is the target output, $o$ is the activated output layer and $f'$ the derivative of the activation function.

But the shape of these weights is $\text{output nodes}\times\text{hidden nodes}$, and $\text{hidden nodes}$ can be bigger than $\text{output nodes}$, so the formula is wrong because of I'm taking $o_k$ and $o$ has length $\text{output nodes}$.

  1. In simple terms, what is the right formula for updating these weights?

  2. Also, what is the right formula for updating the weights between the input layer and the hidden layer?

$\endgroup$

1 Answer 1

1
$\begingroup$
  1. The correct formula for updating the weights between the hidden layer and the output layer is: $$\Delta W_{j,k} = h_k \ \cdot \ o'_{j} \ \cdot \ (o_j - t_j),$$ where $h$ is the activated hidden layer and $o'$ is the derivative of output layer.
    I found this formula in the book Artificial Intelligence: A Guide to Intelligent Systems by Michael Negnevitsky.
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .