The question about analog computing is important. Digital circuitry gained popularity as a replacement for analog circuitry during the four decades between 1975 to 2015 due to three compelling qualities that led to digital standards, central processing units, and the architecture of a general purpose computing.

Regarding quantum computing, there have been some interesting proposals to pack digital gates into much smaller volumes, but the notion that a computer can be made of transistors the size of electrons might be a bit fantastic. That's what the term quantum computing implies. That degree of miniaturization defies some of the fairly strong notions of particle physics, including Heisenberg's uncertainty principle.

All computing involves quanta, but statistically. For a transistor in a digital circuit to be statistically stable, there must be a sufficient number of Si atoms with the appropriate doping element atoms in each junction region. Otherwise the transistor will not switch cleanly.

There needs to be at least one dopant atom per about a thousand silicon atoms for a PN junction to function, the lithographic limit of VLSI for most mass produced chips is currently 7 nm, and nucleus to nucleus in crystalline Si is about .2 nm, so the miniaturization of a stable transistor is near its limit. Exceeding that limit by a considerable amount will necessarily destabilize the digital circuitry. That's a quantum physics limitation, not a lithographic limitation.

Moore's law was simply an approximate model for the chip industry during the period between the invention of the integrated circuit to the physical limitation of the atomic composition of transistors. Field effect transistors (FETs) can take the miniaturization only slightly further than the mechanics of PN junctions. 3D circuitry has theoretical promise, but no repeatable nanotechnology mechanisms have yet been developed to form the circuitry in the third dimension.

Placing the magical idea that quantum computing will somehow completely revolutionize circuitry, we have a question that is more clearly feasible and can be reasonably answered.

Can analog an computer implement real-valued neural networks and hence do artificial network computation better?

It definitely takes fewer transistors to create the feed forward part of an artificial network using an analog approximation than a digital one. They are both approximations because analog circuitry has noise and drift and digital circuitry has rounding error. Beyond rounding, digital multiplication is much more complex in terms of circuitry than analog, and multiplication is used quite a bit in artificial networks.

The idea from the tail end of the title of the book this question referenced, "Beyond the Turing Limit," is also a little fantastic. The thought experiment of Alan Turing leading to the Turing machine and the associated computability theory (including Turing completeness) was not developed to be a limit. It was an answer to Gödel's incompleteness theory, to remove a limit that Gödel seemed to have imposed on the vision of using machines to automatically expand human knowledge.

The theory limiting what computers can do is not related to how the numbers are related. It is a limitation of information mechanics. Analog computing, miniaturization, parallel computing, and how algorithms can be defined in programming languages have little to do with the limitation. Defining what a super-Turing capability might be would be a dismissal or a dismantling of what mathematicians consider to be well constructed theory.

  Dismantling or shifting the contextual frame of some computability theory is plausible. Dismissing the work that has been done would be naive and counterproductive. The compelling idea contained in the question is the reference to continuity, physicality, and the real valued nature of parameters that acquire a learned state during the training of artificial networks.

To say that real values cannot be represented in digital circuits is inaccurate. The IEEE standards for floating point numbers do a good job at representing real valued signals. Analog circuits suffer from noise and drift as stated, and they only appear to carry real values. When dealing in milli-Amps (mA), current seems to be a continuous phenomenon, but when dealing with nano-Amps (nA), the quantum nature of electric current begins to appear. Real valued numbers can only be represented in analog circuits through the flow of discrete electrons. The key to the advantage of an analog forward feed in artificial networks is solely that the transistor count is considerably more thrifty.

This statement is not precisely scientific, even though it may point to something true.

This statement is phrased in a way that is more consistent with scientific fact as well as being factual.

It is possible that computers dealing with signals at a near quantum level of miniaturization will be analogue.

This question and its answers have many links to papers and research work regarding analog computing: https://ai.stackexchange.com/questions/7328/if-digital-values-are-mere-estimates-why-not-return-to-analog-for-ai

Digital and Analog

The question about analog computing is important. 

Digital circuitry gained popularity as a replacement for analog circuitry during the four decades between 1975 to 2015 due to three compelling qualities.

This quickly led to digital signaling standards, architecture of a general purpose computing, and central processing units on a chip. The later, combined with an array of registers to perform elementary operations is the meaning of the word microprocessor.

Quanta and Computing

Regarding quantum computing, there have been some interesting proposals to pack digital gates into much smaller volumes, but the notion that a computer can be made of transistors the size of electrons might be a bit fantastic. That's what the term quantum computing implies. That degree of miniaturization would have to defy principles of particle physics that are very strongly supported by amassed empirical evidence. Among them is Heisenberg's uncertainty principle.

All computing involves quanta, but statistically. For a transistor in a digital circuit to be statistically stable, there must be a sufficient number of Si atoms with at least 0.1% molar concentration of the atoms used to dope the Si to create a PN junction. Otherwise the transistor will not switch reliably.

The lithographic limit of most mass produced VLSI chips is around 7 nm as of this writing. Crystalline Si, nucleus to nucleus, is about .2 nm, so the miniaturization of a stable transistor is near its quantum limit already. Exceeding that limit by a considerable amount destabilizes the digital circuitry. That's a quantum physics limitation, not a lithographic limitation.

Projections, Models, and Techniques to Push Limits

Moore's law was simply an approximate model for the chip industry during the period between the invention of the integrated circuit to the physical limitation of the atomic composition of transistors, which we are now approaching. 

Field effect transistors (FETs) can take the miniaturization only slightly further than the mechanics of PN junctions. 3-D circuitry has theoretical promise, but no repeatable nanotechnology mechanisms have yet been developed to form complex circuitry in the third dimension.

Returning to the Primary Question

Placing aside the magical idea that quantum computing will somehow completely revolutionize circuitry, we have a question that is both feasible and predictable.

Can an analog computer implement real-valued neural networks and hence do artificial network computation better?

If we define better in this context as cheaper and faster, while maintaining reliability and accuracy, the answer is straightforward.

It definitely takes fewer transistors to create the feed forward part of an artificial network using an analog approximation of the closed forms resulting from the calculus of artificial networks than a digital one. Both are approximations. Analog circuitry has noise, and drift and digital circuitry has rounding error. Beyond rounding, digital multiplication is much more complex in terms of circuitry than analog, and multiplication is used quite a bit in artificial network implementations.

Limitation Interplay of Gödel and Turing

The idea from the tail end of the title of the book this question referenced, "Beyond the Turing Limit," is also a little fantastic. The thought experiment of Alan Turing leading to the Turing machine and the associated computability theory (including Turing completeness) was not developed to be a limit. Quite the opposite. It was an answer to Gödel's incompleteness theory. People in Turing's time saw the work Gödel's genius as the annoying but indismissable limit threatening the centuries-old vision of using machines to automatically expand human knowledge. To summarize this work, we can state with assurance this.

The theory limiting what computers can do is not related to how the numbers are represented in electronic circuit implementations. It is a limitation of information mechanics.

These principles are important but have little to do with the limitation.

• Analog computing
• Miniaturization
• Parallel computing
• Ways that stochastic injection can help
• How algorithms can be defined in programming languages

The above has to do with the feasibility of a project for which some person or corporation must pay and the intellectual capacities required to completing it, not the hard limit on what is possible.

Defining what a super-Turing capability might be would be a dismissal or a dismantling of what mathematicians consider to be well constructed theory. Dismantling or shifting the contextual frame of some computability theory is plausible. Dismissing the work that has been done would be naive and counterproductive.

Real Numbers are Now Less Real Than Integers

The compelling idea contained in the question is the reference to continuity, physicality, and the real valued nature of parameters that acquire a learned state during the training of artificial networks.

To say that real values cannot be represented in digital circuits is inaccurate. The IEEE standards for floating point numbers, when processed in a time series, represent real valued signals well. Analog circuits suffer from noise and drift as stated above. Both analog and digital signals only appear to be comprised of real number values. Real numbers are not real except in the world of mathematical models. What we call quantities in the laboratory are essentially measurements of means of distributions. Solidifying and integrating the probabilistic nature of reality into science and technology may be the primary triumph of the twentieth century,

For instance, when dealing in milli-Amps (mA), electric current seems to be a continuous phenomenon, but when dealing with nano-Amps (nA), the quantum nature of electric current begins to appear. This is much like what happens with the miniaturization of the transistor. Real numbers can only be represented in analog circuits through the flow of discrete electrons. The key to the advantage of an analog forward feed in artificial networks is solely that the density of network cells can be considerably higher, reducing the cost of the network in its VLSI space.

In summary, real numbers received the name for their type prior to the emergence of quantum physics. The idea that quantities formerly considered real and continuous were actually statistical averages of discrete activities at a quantum level revolutionized the field of thermodynamics and microelectronics. This is something that disturbed Einstein in his later years. In essence, mathematics using real numbers is effective in engineering because it simplifies what physicist now believe are distributions of a large numbers of quantum phenomena occurring in concert.

Summarizing the Probable Future of Analog Computing

This phrase from the question is not precisely scientific, even though it points to a strong likelihood.

This modified version is more consistent with scientific fact in the way it is phrased, and is also factual.

It is possible that computers dealing with signals at a near quantum level of miniaturization will have a higher proportion of analog circuitry.

This question and its answers have many links to papers and research work regarding analog computing: https://ai.stackexchange.com/questions/7328/if-digital-values-are-mere-estimates-why-not-return-to-analog-for-ai.