New answers tagged

3

In the early days of neural networks the theorists and practitioners were educated in mathematics, psychology, neurophysiology, electrical engineering, and neurobiology. Computer science was still in its infancy. The first neural networks were modeled as electrical circuits. There is evidence of this in the 1943 paper by Warren McCulloch and Walter Pitts [1],...


2

Well, the goal of any paper is to allow the reader to understand what the author is trying to describe. A lot of people have a lot of experience looking at circuit diagrams and figuring out those circuits will do. For these people, a circuit diagram may be the clearest and easiest way for them to understand how a particular thing works. So, it makes sense ...


0

2035, 2056? Those predictions are hilarious :) 2019 - 1,6Billion parameter model (GPT-2) 2020 -175Billion parameter model (GPT-3) more than 100x jump in a year 2021(April) - "Microsoft's ZeRO-Infinity can now run a model with over a trillion parameters on a single NVIDIA DGX-2 node and over 30 trillion parameters on 32 nodes (512 GPUs). With a hundred ...


Top 50 recent answers are included