3
$\begingroup$

I am looking for a book or paper which clearly explains the relationship between Ising models and deep neural networks.

Can anyone provide any references?

New contributor
david is a new contributor to this site. Take care in asking for clarification, commenting, and answering. Check out our Code of Conduct.
$\endgroup$
  • $\begingroup$ Because Energy E of network in Hopfield network: ${\displaystyle E=-{\frac {1}{2}}\sum _{i,j}{w_{ij}{s_{i}}{s_{j}}}+\sum _{i}{\theta _{i}}{s_{i}}}$ is just the energy for Ising model. en.wikipedia.org/wiki/Hopfield_network $\endgroup$ – david Nov 8 at 22:23
4
$\begingroup$

The following articles

may help you understand the "relationship" between Ising models and DNN, assuming you know what the Ising model is and what a DNN is, the similarity should be fairly intuitive to you.

The Ising model is a sort of floating soup of ferromagnetic particles each generating their own small magnetic field either working against or with their neighbor. When many of the particles aline, they create an aligned field we refer to as a dipole moment in magnetism, while in a DNN we refer to the joined effort of a few entities working to cause a larger effect in another entity an 'activation function'. In a fully connected DNN, where the Euclidean distance weights the connections and the nodes are initialized with a certain magnetic polarity in relation to the axis of the magnetic field it generates, the network would be an almost exact representation of the reality of what the Ising model seeks to simplify.

New contributor
Michael Hearn is a new contributor to this site. Take care in asking for clarification, commenting, and answering. Check out our Code of Conduct.
$\endgroup$
  • $\begingroup$ Thank you ! In Ising model, when its partition function has singularity, which means there is a phase transition for the spin system, lots of interesting things can be said about the system. For DNN, what is the corresponding properties when its partition function is zero or singular ? Any papers on this ? Thank you again. $\endgroup$ – david 10 hours ago
  • $\begingroup$ Very little research goes into examine DNN like this. Phase changes would be similar to decision boundaries of the activation functions surounding a location, similar to how a small disturbance in water causes a change in pressure at a location and cause the first ice crystal to form which can cause the chain reaction of other ice crystals forming. I'll edit in some papers at the end to check if it adds to the answer. $\endgroup$ – Michael Hearn 6 hours ago

Your Answer

david is a new contributor. Be nice, and check out our Code of Conduct.

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.