Could a Boltzmann machine store more patterns than a Hopfield net?

Question

_{This is from a closed beta for AI, with this question being posted by user number 47. All credit to them.}

---

According to Wikipedia,

Boltzmann machines can be seen as the stochastic, generative counterpart of Hopfield nets.

Both are recurrent neural networks that can be trained to learn of bit patterns. Then when presented with a partial pattern, the net will retrieve the full complete pattern.

Hopfield networks have been proven to have a capacity of 0.138 (e.g. approximately 138 bit vectors can be recalled from storage for every 1000 nodes, Hertz 1991).

As a Boltzmann machine is stochastic, my understanding is that it would not necessarily always show the same pattern when the energy difference between one stored pattern and another is similar. But because of this stochasticity, maybe it allows for denser pattern storage but without the guarantee that you'll always get the "closest" pattern in terms of energy difference. Would this be true? Or would a Hopfield net be able to store more patterns?

Answer 1

Indeed you're intuition is correct, a Boltzmann machine is able to hold more than a Hopfield network in its memory because of its stochastic nature as explored in this paper. In the paper they note that the capacity is around 0.6. After this ratio it starts to break down and adds much more noise to the recalled patterns.