3
$\begingroup$

I am looking for a book or paper which clearly explains the relationship between Ising models and deep neural networks.

Can anyone provide any references?

$\endgroup$
4
$\begingroup$

The following articles

may help you understand the "relationship" between Ising models and DNN, assuming you know what the Ising model is and what a DNN is, the similarity should be fairly intuitive to you.

The Ising model is a sort of floating soup of ferromagnetic particles each generating their own small magnetic field either working against or with their neighbor. When many of the particles aline, they create an aligned field we refer to as a dipole moment in magnetism, while in a DNN we refer to the joined effort of a few entities working to cause a larger effect in another entity an 'activation function'. In a fully connected DNN, where the Euclidean distance weights the connections and the nodes are initialized with a certain magnetic polarity in relation to the axis of the magnetic field it generates, the network would be an almost exact representation of the reality of what the Ising model seeks to simplify.

| improve this answer | |
$\endgroup$
  • $\begingroup$ Thank you ! In Ising model, when its partition function has singularity, which means there is a phase transition for the spin system, lots of interesting things can be said about the system. For DNN, what is the corresponding properties when its partition function is zero or singular ? Any papers on this ? Thank you again. $\endgroup$ – david Nov 13 '19 at 14:30
  • $\begingroup$ Very little research goes into examine DNN like this. Phase changes would be similar to decision boundaries of the activation functions surounding a location, similar to how a small disturbance in water causes a change in pressure at a location and cause the first ice crystal to form which can cause the chain reaction of other ice crystals forming. I'll edit in some papers at the end to check if it adds to the answer. $\endgroup$ – Michael Hearn Nov 13 '19 at 17:43
  • $\begingroup$ Thank again. Let's read those papers you listed. $\endgroup$ – david Nov 14 '19 at 4:59
  • $\begingroup$ I am from physics background. I am interested in two types of questions. One is, how to use existing physics results of Ising model to DNN. Another question I am interested is, how to determine the interaction coefficient of spin model if we know its partition function. Do you know, any DNN research on the 2nd type of question ? Thank you. $\endgroup$ – david Nov 15 '19 at 19:29
  • $\begingroup$ My intuition is that if you have the partition function and existing Ising data there would be an algorithms for seeding a Deep Neural network in an ideal way so as to allow for learning the interaction coefficients in the network much faster. Knowing the function a DNN is attempting to learn/emulate allows for the resources of a NN to be used in a more optimal, by elimination of unnecessary variables/features. I'll keep my eye out for such research. $\endgroup$ – Michael Hearn Nov 15 '19 at 20:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.