I just wabt to ask do i understand the usage and updating biasses while backpropagating and what values comes where. Let us see following network : with bach size 6, input size 2, hidden size 4 and output size 3. The biases are same to having a "ones" staying at the end of each layer and having the values staying at the bottom of weights, like that you will have the summ over the weights column plus bias. In the result matrix, each bias value is added in each cell over the column.
Calculating the delta for not last layer (called hidden in this case) we need do: output delta X output weights transposed. After that, calculating hidden weights delta you will see that position of biasses, marker here as questions, are sort of useless. From the realisation of neuronal network i v seen it seems like they are not concidered at all???