In a video lecture on the development of neural networks and the history of deep learning (you can start from minute 13), the lecturer (Yann LeCunn) said that the development of neural networks stopped until the 80s because people were using the wrong neurons (which were binary so discontinuous) and that is due to the slowness of multiplying floating point numbers which made the use of backpropagation really difficult.
He said, I quote, "If you have continuous neurons, you need to multiply the activation of a neuron by a weight to get a contribution to the weighted sum."
But the statement stays true even with binary (or any discontinuous activation function) neurons. Am I wrong? (at least, as long as you're in the hidden layer, the output of your neuron will be multiplied by a weight I guess). The same professor said that the perceptron, ADALINE relied on weighted sums so they were computing multiplications anyways.
I don't know what I miss here and I hope someone will enlighten me.