What is the time complexity for training a single-hidden layer auto-encoder, for 1 epoch?

You can assume that there are $n$ training examples, $m$ features, and $k$ neurons in the hidden layer, and that we use gradient descent and back-propagation to train the auto-encoder.

  • $\begingroup$ I removed the "stacked" part because I don't think that's necessary. What did you mean by "stack auto-encoder", btw? $\endgroup$
    – nbro
    Sep 22 '20 at 12:10
  • $\begingroup$ When decoder weights matrices (except bias term) are just the transpose of encoder weight matrices. $\endgroup$ Sep 22 '20 at 12:16
  • $\begingroup$ Are you sure that's the definition of stacked AE? Where did you take that definition from? $\endgroup$
    – nbro
    Sep 22 '20 at 12:28
  • $\begingroup$ @nbro It might be this: medium.com/@venkatakrishna.jonnalagadda/…. A stacked autoencoder is a neural network consist several layers of sparse autoencoders where output of each hidden layer is connected to the input of the successive hidden layer. By the way, it's not a single layer stacked AE! $\endgroup$
    – OmG
    Sep 22 '20 at 16:10
  • $\begingroup$ @OmG The OP is saying that the decoder has the same weights as the encoder, but only transposed weights. This definition doesn't seem correct, according to your definition or other definitions I've seen so far. That's why I was asking for a reference/source of that definition. $\endgroup$
    – nbro
    Sep 22 '20 at 16:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.