I listened to a talk by a panel consisting of two influential Chinese scientists: Wang Gang and Yu Kai and others.
When being asked about the biggest bottleneck of the development of artificial intelligence in the near future (3 to 5 years), Yu Kai, who has a background in the hardware industry, said that hardware would be the essential problem and we should pay most of our attention to that. He gave us two examples:
- In the early development of the computer, we compare our machines by their chips;
- ML/DL which is very popular these years would be almost impossible if not empowered by Nvidia's GPU.
The fundamental algorithms existed already in the 1980s and 1990s, but AI went through 3 AI winters and was not empirical until we can train models with GPU boosted mega servers.
Then Dr. Wang commented to his opinions that we should also develop software systems because we cannot build an automatic car even if we have combined all GPUs and computation in the world together.
Then, as usual, my mind wandered off and I started thinking that what if those who can operate supercomputers in the 1980s and 1990s utilized the then-existing neural network algorithms and train them with tons of scientific data? Some people at that time can obviously attempt to build the AI systems we are building now.
But why did AI/ML/DL become a hot topic and become empirical until decades later? Is it only a matter of hardware, software, and data?