1
$\begingroup$

What is the difference between stacked restricted Boltzmann machine (RBM) and deep belief network (DBN)?

Are they the same entity? If so, why? Is the latter some specific type of the former? If so, how to tell if stacked RBM is a DBN?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

For what it's worth, Wikipedia says that

deep belief networks can be formed by "stacking" RBMs.

Hinton writes in Scholarpedia:

A deep belief net can be viewed as a composition of simple learning modules each of which is a restricted type of Boltzmann machine.

So, a deep belief network is definitely a stacked RBM.

I have never heard of different stacked RBMs, but it is easy to imagine something like convolutional stacked RBMs, where some RBMs are used as filters that slide over the input data or something. Whether that would still be called a deep belief network, is probably up to the guy who publishes it first.

Improve this answer
$\endgroup$
4
  • $\begingroup$ RBM is undirected. Shoud not DBN have directed units (umhh.. classifiers?) as top layers? Can be simple stacked RBM which is basically a feature detector, can be studied as a DBN? $\endgroup$
    – Diligent Key Presser
    Jan 3, 2017 at 11:35
  • $\begingroup$ What do you mean RBMs are undirected? They have an input layer and the input gets propagated to a hidden layer (and back during training). That looks pretty directed to me. I think BMs are undirected. If you want to do classification you'll put a classifier on top, that's correct. $\endgroup$
    – BlindKungFuMaster
    Jan 3, 2017 at 12:13
  • $\begingroup$ By "undirected" i mean a possibility of input reconstruction from known feature vector (like with an autoencoder), which is not so straightforward for classifiers. Anyway, things became a bit clearer. Thank you:) $\endgroup$
    – Diligent Key Presser
    Jan 3, 2017 at 13:05
  • $\begingroup$ The possibility of input reconstruction from the hidden layer activation is how you train, but once you've learned your features the backward pass isn't used anymore. So then you just have an ordinary feedforward network. $\endgroup$
    – BlindKungFuMaster
    Jan 3, 2017 at 15:54

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .