1
$\begingroup$

Apparently, one can buy a special-purpose integrated circuit (an IC like this one, for instance) to host a convolutional neural network.

QUESTION

Is such a circuit digital? Except for digital random-number generation, is the circuit's behavior deterministic?

What kind of hardware is this?

REFERENCE

Already asked and answered: "If digital values are mere estimates, why not return to analog for AI?"

FURTHER DETAILS

What I mean is this: suppose that you and I each acquired a special-purpose neural IC, same make, same model. Suppose that you and I configured and trained our circuits alike. Except for digital random-number generation, would our neural nets behave identically? Or would analog effects cause our nets to turn oppositely on some closely balanced edge case?

If it is plain to you that a misconception has caused me to pose the question improperly, then a correction to the question along with an answer would be appreciated.

If you wish to know my background, I am an electrical power engineer in my 50s.

$\endgroup$
1
$\begingroup$

Intel's Movidius uses digital circuitry for signal paths. It is a parallel processing VLSI chip that can host a sequence of convolution layers and other types of parallel operations, just as CUDA cores in NVidia products do. These all are extensions of the idea of DSPs (digital signal processors). They are adept as front ends to real time vision systems in robots and vehicle automation. They are also stuffed into racks and used for massively parallel operations in data centers, leveraged for large batch or mini-batch jobs related to indexing or supporting transaction based services.

There are three broad signal theory characteristics referenced conceptually in this question which may benefit from some disambiguation.

  • Analog versus digital
  • Deterministic versus stochastic
  • Predictable versus unpredictable outcome

Of the eight permutations, any of these five can be realized or reasonably accurately approximated. Under what conditions they are repeatable is in parentheses.

  • Analog, deterministic, predictable outcome — Yes, within the bounds of specifications such as signal to noise ratio
  • Analog, deterministic, unpredictable outcome — Usually no
  • Analog, stochastic, unpredictable — No
  • Digital, deterministic, predictable outcome — Yes
  • Digital, stochastic, unpredictable — No

The commercially available ICs used for artificial neural networks mentioned above are digital, but they may not be deterministic. For instance, the seed of random number generators can be the system time. When embedded into larger computing systems entropy can also be collected digitally from mouse movement, microphones, camera frames, or secure network sources and then used in the digital signal processing algorithms, including artificial network ones.

Future chips may actually return to analog processing for some of the workload. The accuracy of digital circuitry may be worth sacrificing in exchange for increase complexity and throughput per unit area on the VLSI substrate. The storage and retrieval advantage of flip flops and other digital structures can still be maintained by interfacing the storage to the forward signal paths with A-to-D and D-to-A conversion. It is difficult to predict whether the trade-off will gain traction and, if so, when.

The term, "Balanced edge case," references what mathematicians familiar with chaos theory sometimes call initial input sensitivity, colloquially known as, "The Butterfly Effect." Chaos can occur in nature where the only discrete operations are at the quantum level. Chaos can also occur in digital systems with no greater mathematical complexity than the Logistic Equation: $x_{t+1} = r \, x_t (1 - x_t)$, where interesting behavior is observed for values of $r > 2.0$.

In electrical power engineering, much of what appears on an oscilloscope in triphase circuits are not sine waves but chaos that exhibits a strong peak in the power spectrum at 50 or 60 Hertz.

$\endgroup$
  • $\begingroup$ I doubt that my admittedly clumsy question merits an answer this concise, complete and clear, so the answer is much appreciated by me. Your answer is just excellent. $\endgroup$ – thb Nov 20 '18 at 16:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.