1
$\begingroup$

I'm creating a neural network with 3 layers and no bias.

On internet I saw that the expression for the derivative of the weights between the hidden layer and the output layer was:

$$\Delta W_{j,k} = (o_k - t_k) \cdot f'\left[\sum_j (W_{j,k} \ \cdot o_j)\right] \cdot o_j,$$

where $t$ is the target output, $o$ is the activated output layer and $f'$ the derivative of the activation function.

But the shape of these weights is $\text{output nodes}\times\text{hidden nodes}$, and $\text{hidden nodes}$ can be bigger than $\text{output nodes}$, so the formula is wrong because of I'm taking $o_k$ and $o$ has length $\text{output nodes}$.

  1. In simple terms, what is the right formula for updating these weights?

  2. Also, what is the right formula for updating the weights between the input layer and the hidden layer?

$\endgroup$
0
$\begingroup$
  1. The correct formula for updating the weights between the hidden layer and the output layer is: $$\Delta W_{j,k} = h_k \ \cdot \ o'_{j} \ \cdot \ (o_j - t_j),$$ where $h$ is the activated hidden layer and $o'$ is the derivative of output layer.
    I found this formula in the book Artificial Intelligence: A Guide to Intelligent Systems by Michael Negnevitsky.
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.