I just wabt to ask do i understand the usage and updating biasses while backpropagating and what values comes where. Let us see following network : with bach size 6, input size 2, hidden size 4 and output size 3. The biases are same to having a "ones" staying at the end of each layer and having the values staying at the bottom of weights, like that you will have the summ over the weights column plus bias. In the result matrix, each bias value is added in each cell over the column. enter image description here

So far it is understandable perfectly fine. calculating the delta weights is like : enter image description here

So if you calculate for bias cells, it will be ones X column, means over all items in batch for position where bias stays?! It is also understandable. The question begins here : enter image description here

Calculating the delta for not last layer (called hidden in this case) we need do: output delta X output weights transposed. After that, calculating hidden weights delta you will see that position of biasses, marker here as questions, are sort of useless. From the realisation of neuronal network i v seen it seems like they are not concidered at all???


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.