The question about analog computing is important. Digital circuitry gained popularity as a replacement for analog circuitry during the four decades between 1975 to 2015 due to three compelling qualities that led to digital standards, central processing units, and the architecture of a general purpose computing.
- Greater noise immunity
- Greater drift immunity (accuracy)
- No leakage of stored values
Regarding quantum computing, there have been some interesting proposals to pack digital gates into much smaller volumes, but the notion that a computer can be made of transistors the size of electrons might be a bit fantastic. That's what the term quantum computing implies. That degree of miniaturization defies some of the fairly strong notions of particle physics, including Heisenberg's uncertainty principle.
All computing involves quanta, but statistically. For a transistor in a digital circuit to be statistically stable, there must be a sufficient number of Si atoms with the appropriate doping element atoms in each junction region. Otherwise the transistor will not switch cleanly.
There needs to be at least one dopant atom per about a thousand silicon atoms for a PN junction to function, the lithographic limit of VLSI for most mass produced chips is currently 7 nm, and nucleus to nucleus in crystalline Si is about .2 nm, so the miniaturization of a stable transistor is near its limit. Exceeding that limit by a considerable amount will necessarily destabilize the digital circuitry. That's a quantum physics limitation, not a lithographic limitation.
Moore's law was simply an approximate model for the chip industry during the period between the invention of the integrated circuit to the physical limitation of the atomic composition of transistors. Field effect transistors (FETs) can take the miniaturization only slightly further than the mechanics of PN junctions. 3D circuitry has theoretical promise, but no repeatable nanotechnology mechanisms have yet been developed to form the circuitry in the third dimension.
Placing the magical idea that quantum computing will somehow completely revolutionize circuitry, we have a question that is more clearly feasible and can be reasonably answered.
Can analog an computer implement real-valued neural networks and hence do artificial network computation better?
It definitely takes fewer transistors to create the feed forward part of an artificial network using an analog approximation than a digital one. They are both approximations because analog circuitry has noise and drift and digital circuitry has rounding error. Beyond rounding, digital multiplication is much more complex in terms of circuitry than analog, and multiplication is used quite a bit in artificial networks.
The idea from the tail end of the title of the book this question referenced, "Beyond the Turing Limit," is also a little fantastic. The thought experiment of Alan Turing leading to the Turing machine and the associated computability theory (including Turing completeness) was not developed to be a limit. It was an answer to Gödel's incompleteness theory, to remove a limit that Gödel seemed to have imposed on the vision of using machines to automatically expand human knowledge.
The theory limiting what computers can do is not related to how the numbers are related. It is a limitation of information mechanics. Analog computing, miniaturization, parallel computing, and how algorithms can be defined in programming languages have little to do with the limitation. Defining what a super-Turing capability might be would be a dismissal or a dismantling of what mathematicians consider to be well constructed theory.
Dismantling or shifting the contextual frame of some computability theory is plausible. Dismissing the work that has been done would be naive and counterproductive. The compelling idea contained in the question is the reference to continuity, physicality, and the real valued nature of parameters that acquire a learned state during the training of artificial networks.
To multiply a vector of digital signals by a digital parameter matrix requires many transistors and can require a significant number of clock cycles, even when dedicated hardware is used. Only a few transistors per signal path are required to perform the analog equivalent, and the throughput potential is very high.
To say that real values cannot be represented in digital circuits is inaccurate. The IEEE standards for floating point numbers do a good job at representing real valued signals. Analog circuits suffer from noise and drift as stated, and they only appear to carry real values. When dealing in milli-Amps (mA), current seems to be a continuous phenomenon, but when dealing with nano-Amps (nA), the quantum nature of electric current begins to appear. Real valued numbers can only be represented in analog circuits through the flow of discrete electrons. The key to the advantage of an analog forward feed in artificial networks is solely that the transistor count is considerably more thrifty.
This statement is not precisely scientific, even though it may point to something true.
It is quite possible that quantum computers will be analogue.
This statement is phrased in a way that is more consistent with scientific fact as well as being factual.
It is possible that computers dealing with signals at a near quantum level of miniaturization will be analogue.
This question and its answers have many links to papers and research work regarding analog computing: https://ai.stackexchange.com/questions/7328/if-digital-values-are-mere-estimates-why-not-return-to-analog-for-ai