When training the three-layer autoencoder, do they compute dL/dW (where L is equation 1) independently in each layer or perform backpropagation from one layer to another?
I believe the gradients in the three layers are computed independently.
They mention minimizing each layer only with respect to the weights used in that layer. If they performed some sort of backpropagation between layers, they would be minimizing the term for one layer with respect to the weights for another layer. Since the explicitly state that equation 1 (the loss for each layer) is simply minimized for all three layers simultaneously, they clearly imply that only the weights for that layer are considered in the implementation.
But this only confuses me further as to why they train all three layers simultaneously, as asked in my second question.