0
$\begingroup$

Le et al. 2013's non-weight sharing CNN has inspired me to ask two questions on this site previously.

When training the three-layer autoencoder, do they compute dL/dW (where L is equation 1) independently in each layer or perform backpropagation from one layer to another?

$\endgroup$

1 Answer 1

0
$\begingroup$

I believe the gradients in the three layers are computed independently.

They mention minimizing each layer only with respect to the weights used in that layer. If they performed some sort of backpropagation between layers, they would be minimizing the term for one layer with respect to the weights for another layer. Since the explicitly state that equation 1 (the loss for each layer) is simply minimized for all three layers simultaneously, they clearly imply that only the weights for that layer are considered in the implementation.

But this only confuses me further as to why they train all three layers simultaneously, as asked in my second question.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .