I am trying to read https://arxiv.org/abs/1701.01727 about generalisation of Hopfield neural networks and I like the clear ideas that physics and Hamiltoanian framework can be used for modeling such networks and for deducing lot of properties. My question is - how Hopfield networks are connected to the standard networks?
I see at least 3 differences:
- Hopfield neurons has binary values on/off (+1, -1) but machine learning neurons have real (are at least approximately (within machine limits) real) values.
- threshold function is the only activation function used in Hopfield neurons, machine learning has lot of more complex, real-valued functions
- Connectivity patterns in machine learning networks (LSTM, GRU cells) are far more richer.
So - maybe Hopfield neural networks can be generalized up to the level of machine learning networks and at the same time preserving the use of Hamiltonian framework. Is it possible, is there any work in this direction?