16
$\begingroup$

This is from a closed beta for AI, with this question being posted by user number 47. All credit to them.


According to Wikipedia,

Boltzmann machines can be seen as the stochastic, generative counterpart of Hopfield nets.

Both are recurrent neural networks that can be trained to learn of bit patterns. Then when presented with a partial pattern, the net will retrieve the full complete pattern.

Hopfield networks have been proven to have a capacity of 0.138 (e.g. approximately 138 bit vectors can be recalled from storage for every 1000 nodes, Hertz 1991).

As a Boltzmann machine is stochastic, my understanding is that it would not necessarily always show the same pattern when the energy difference between one stored pattern and another is similar. But because of this stochasticity, maybe it allows for denser pattern storage but without the guarantee that you'll always get the "closest" pattern in terms of energy difference. Would this be true? Or would a Hopfield net be able to store more patterns?

$\endgroup$
2
$\begingroup$

Indeed you're intuition is correct, a Boltzmann machine is able to hold more than a Hopfield network in its memory because of its stochastic nature as explored in this paper. In the paper they note that the capacity is around 0.6. After this ratio it starts to break down and adds much more noise to the recalled patterns.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.