I am looking for a book or paper which clearly explains the relationship between Ising models and deep neural networks.
Can anyone provide any references?
$\begingroup$
$\endgroup$
0
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
4
The following articles
Ising models for networks of real neurons (2006) by Gasper Tkacik et al.
Deep neural networks for direct, featureless learning through observation: The case of two-dimensional spin models (2018) by Kyle Mills et al.
Inverse Ising inference by combining Ornstein-Zernike theory with deep learning (2017) by Soma Turi, Alpha A. Lee et al.
Neural-network based general method for statistical mechanics on sparse systems (2019) by Feng Pan, Hai-Jun Zhou, Pan Zhang et al.
may help you understand the "relationship" between Ising models and DNN, assuming you know what the Ising model is and what a DNN is, the similarity should be fairly intuitive to you.
The Ising model is a sort of floating soup of ferromagnetic particles each generating their own small magnetic field either working against or with their neighbor. When many of the particles aline, they create an aligned field we refer to as a dipole moment in magnetism, while in a DNN we refer to the joined effort of a few entities working to cause a larger effect in another entity an 'activation function'. In a fully connected DNN, where the Euclidean distance weights the connections and the nodes are initialized with a certain magnetic polarity in relation to the axis of the magnetic field it generates, the network would be an almost exact representation of the reality of what the Ising model seeks to simplify.
-
$\begingroup$ Thank you ! In Ising model, when its partition function has singularity, which means there is a phase transition for the spin system, lots of interesting things can be said about the system. For DNN, what is the corresponding properties when its partition function is zero or singular ? Any papers on this ? Thank you again. $\endgroup$– davidCommented Nov 13, 2019 at 14:30
-
$\begingroup$ Very little research goes into examine DNN like this. Phase changes would be similar to decision boundaries of the activation functions surounding a location, similar to how a small disturbance in water causes a change in pressure at a location and cause the first ice crystal to form which can cause the chain reaction of other ice crystals forming. I'll edit in some papers at the end to check if it adds to the answer. $\endgroup$ Commented Nov 13, 2019 at 17:43
-
$\begingroup$ I am from physics background. I am interested in two types of questions. One is, how to use existing physics results of Ising model to DNN. Another question I am interested is, how to determine the interaction coefficient of spin model if we know its partition function. Do you know, any DNN research on the 2nd type of question ? Thank you. $\endgroup$– davidCommented Nov 15, 2019 at 19:29
-
$\begingroup$ My intuition is that if you have the partition function and existing Ising data there would be an algorithms for seeding a Deep Neural network in an ideal way so as to allow for learning the interaction coefficients in the network much faster. Knowing the function a DNN is attempting to learn/emulate allows for the resources of a NN to be used in a more optimal, by elimination of unnecessary variables/features. I'll keep my eye out for such research. $\endgroup$ Commented Nov 15, 2019 at 20:48