Is there any way and any reason why one would introduce a sparsity constraint on a deep autoencoder?

In particular, in deep autoencoders the first layer often has more units than the dimensionality of the input. Is there any case in the literature where a penalty is explicitly imposed for non-sparsity on this layer rather than relying solely on back-propagation and maybe weight decay as in a normal multilayer network?

I read this tutorial on sparse autoencoders and searched a bit online but did not find any case where such a sparsity constraint is used in any other case than when only a single layer is used.

  • $\begingroup$ If you can make a single layer autoencoder with a sparcity constraint then you can take a few of those to make a stacked autoencoder. In keras it is a pain to use with the sequential model API, $\endgroup$ Oct 5 '18 at 17:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.