I am attempting to create a fully decoupled feed-forward neural network by using decoupled neural interfaces as explained in the paper (https://arxiv.org/abs/1608.05343). As in the paper, the DNI is able to produce a synthetic error gradient that reflects the error with respect the the output:
I can then use this to update the current layer's parameters by multiplying by the parameters to get the loss with respect to the parameters:
In the paper, the layer's model is then updated based on the next layer sending the true error backwards.
My question is, given that I am able to calculate the error with respect to the current output, how do I use this to calculate the Loss with respect to the previous layer's output?