What is the difference between stacked restricted Boltzmann machine (RBM) and deep belief network (DBN)?

Are they the same entity? If so, why? Is the latter some specific type of the former? If so, how to tell if stacked RBM is a DBN?


For what it's worth, Wikipedia says that

deep belief networks can be formed by "stacking" RBMs.

Hinton writes in Scholarpedia:

A deep belief net can be viewed as a composition of simple learning modules each of which is a restricted type of Boltzmann machine.

So, a deep belief network is definitely a stacked RBM.

I have never heard of different stacked RBMs, but it is easy to imagine something like convolutional stacked RBMs, where some RBMs are used as filters that slide over the input data or something. Whether that would still be called a deep belief network, is probably up to the guy who publishes it first.

  • $\begingroup$ RBM is undirected. Shoud not DBN have directed units (umhh.. classifiers?) as top layers? Can be simple stacked RBM which is basically a feature detector, can be studied as a DBN? $\endgroup$ – Diligent Key Presser Jan 3 '17 at 11:35
  • $\begingroup$ What do you mean RBMs are undirected? They have an input layer and the input gets propagated to a hidden layer (and back during training). That looks pretty directed to me. I think BMs are undirected. If you want to do classification you'll put a classifier on top, that's correct. $\endgroup$ – BlindKungFuMaster Jan 3 '17 at 12:13
  • $\begingroup$ By "undirected" i mean a possibility of input reconstruction from known feature vector (like with an autoencoder), which is not so straightforward for classifiers. Anyway, things became a bit clearer. Thank you:) $\endgroup$ – Diligent Key Presser Jan 3 '17 at 13:05
  • $\begingroup$ The possibility of input reconstruction from the hidden layer activation is how you train, but once you've learned your features the backward pass isn't used anymore. So then you just have an ordinary feedforward network. $\endgroup$ – BlindKungFuMaster Jan 3 '17 at 15:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.