I am familiar with the currently popular neural network models that have weights and are trained with backpropagation and gradient descent.
However, I came across a different type of neural network popular in the 1980s and 1990s. The Hopfield network is one of the most classic. These neural network models different in the following ways:
- They do not have parameter weights and bias to train or to learn from data.
- They used a circuit diagram to present the model.
- The model can be simplified as an ODE system and has a Lyapunov function as objective.
Please take a look at these two papers in the 1980s:
Neural Networks for Non-linear Programming (M.P Kennedy & L.O Chua)