10
$\begingroup$

I'm now reading a book titled Hands-On Machine Learning with Scikit-Learn and TensorFlow and in the Chapter 10 of the book, the author writes the following:

The architecture of biological neural networks (BNN)4 is still the subject of active research, but some parts of the brain have been mapped, and it seems that neurons are often organized in consecutive layers, as shown in Figure 10-2.

enter image description here

However there seems to be no link to any research there. And the author didn't say it assertively given that he used "it seems that neurons are often organized in consecutive layers".

Is this true and how strongly is it believed? What research is this from?

$\endgroup$
  • $\begingroup$ @JadenTravnik's answer is good, also see my comment there. The difference between the reference here and that answer is this citation seems to be implying that there is a feed-forward structure in a single column in neocortex (since that's the picture represented here). This is certainly false, though works as a model sometimes. Although there is evidence for some feed-forward processing in a column, there is also a ton of recurrence and feedback. The feed-forward structure makes more sense between cortical areas (this is what the answer below addresses). $\endgroup$ – Bryan Krause Aug 3 '17 at 1:58
  • $\begingroup$ The terminology of "layer" has a different meaning in the two contexts as well. When biologists talk about cortical "layers" they mean anatomical layers, not functional neural network-style layers. Cells within one layer are highly interconnected with each other, as well as to a lesser extent with cells of all of the other layers. Some of the connectivity is in this answer to a different question at biology.se: biology.stackexchange.com/questions/57495/… $\endgroup$ – Bryan Krause Aug 3 '17 at 2:00
6
$\begingroup$

Really short answer: yes

Slightly longer answer: kinda

Long answer:

Convolutional neural networks (CNNs), which are now a standard in image processing models, were inspired from work done by Hubel and Wiesel in the 1950-60s. They showed that the visual cortexes of cats and mokeys contain neurons which individually respond to small regions of the visual field.

To give some background, we have to first start from the rods and cones in eyes. These photosensitive cells are connected to a few layers of cells before even leaving the retina via ganglion cells.

Image of rods connected to bipolar cells connected to ganglion cells

These ganglion cells are then connected to several regions of the brain but primarily the Occipital lobe located at the back of the brain. The Occipital lobe is responsible for visual processing and is separated into cortical layers, the first named V1 which is the primary visual area. Most of the work by Hubel and Wiesel involved cells in V1 and showed how these cells were sensitive to orientation and color from their respective receptive areas on the retina.

enter image description here

The cells in V1 are connected to the cells in V2 which are sensitive to even more specific stimulus such as movement with orientation and this trend of specific sensitivity continues up from V2 to higher regions in the brain.

This layered approach to vision has been heavily exploited in CNNs, so much so that when the sensitivity of neurons in trained CNNs is displayed, similar responses (orientation) are found.

enter image description here

There is clear evidence of layers in biological optical systems and similarly layered structures in the other senses. Although there are many connections between different brain structures, the main structure of layers in the brain has helped understand what different areas of the brain do and has helped inspire many (if not all) advances in neural network research.

$\endgroup$
  • 1
    $\begingroup$ Got pinged from Biology.SE. This answer is good, certainly good enough for this field. Layers in real neocortex differ from most neural networks in that they are massively recurrent, consist of simultaneously active feed-forward and feed-back, and very dependent on recent history and overall state. And that's just in one visual area (like V1). Some artificial networks impart some of these features, others mimic them with other more computationally-friendly mechanisms. $\endgroup$ – Bryan Krause Aug 3 '17 at 1:56
1
$\begingroup$

Are biological neurons organized in consecutive layers?

Embracing the Reality of Complexity

To say, "Yes," would be a gross oversimplification, just as digital learning arising out of some simple form of recursion applied to a set of first order predicate logic rules has been like running along a leprechaun's rainbow to the pot of gold.

The final set of questions is apropos: "Is this true and how strongly is it believed? What research is this from?" You'd need a poll to determine how strongly it is believed that neurons in the brain are in a predominantly layered structure. References to layers in actual research does not seem to make any claims that layers are consecutive in most if any cases. There are consecutive layers to the skin, but skin with only layers would lack pores, hairs, interfaces with body orifices and many other features. In the human brain (or in animal brains) the three dimensional complexity is considerably augmented from that of skin.

It would be nice, from the AI researcher's perspective, if

  • A heuristic or theoretically proven recursive scheme applied to an expert system could produce learning or intelligence or
  • A map of the human (or bird) brain could be reduced to a set of like neurons in rows of identical neurons, arranged in layers.

The image provided in the question does not illustrate such simplicity. It actually illustrates the converse, that nature is rarely so transparent in it intricacies.

The characterization, "Since it seems that neurons are often organized in consecutive layers," is not accurate. The below more reasonable characterization of the particular slice shown indicates two regions that may be discretely distinct, the grid on the leftmost 8% and the largely horizontal connectedness in the remaining 92%.

An electrical engineer or mathematician would probably not call these two sections layers. The left side might be hypothesized to be a matrix of some form and the right 92% might be considered a complex processing circuit.

Characterization of the 2D Structure

  • The axons are directed primarily along the direction described by the unit vector (-1, 0, 0), otherwise describable as right to left.
  • The density of axons increase for lover values of x, due to the high proportion of axons that terminate at lower values of x.
  • The nucleus density is relatively even in the 0.1 through 1.0 proportional range of x.
  • The nucleus sizes and associated dendrite complexity conforms approximately to a gradient, with a primary maxima at 0.8 of the proportional value of x and a secondary maxima at 0.55 of the proportional value of x
  • At least two axons bifurcate in between those proportional x locations.
  • There are nearly equidistant axioms all roughly parallel to the z axis in the range of 0.0 to 0.08 of the proportional value of x.
  • Further structural patterns are either obscure or nonexistent.

Another Image with Chaotic Structure

enter image description here

Millions of Times the Complexity

Consider further that much of the complexity is hidden from the viewer in a single slice of a three dimensional neurological structure. If we arbitrarily decide that the image is a slice cut in parallel with the x-z plane, we can see the relationships in that x-z plan, but in neither the x-y nor the y-z. Any other slice from another direction or location in the brain will be as unique as an arbitrary window into the Mandelbrot Set.

More Misrepresentation of Research Results

The phrase, "Some parts of the brain have been mapped," is misleading too. The general connectivity between substructures of the human brain have been mapped, not the signals and criteria for signal propagation and strength in individual neurons. Circuits differ radically at the neuron level between two brains, both of which exhibit intelligent in vivo (in a living being).

The analogy is like a person the size of a microbe with a map of continents, major cities, and shipping routes but with no prior knowledge of transportation systems, no GPS, and no other detailed map who wishes to travel from the Eiffel tower to the center of town in Sidney Australia. There is an insufficient set of transportation system evolution or detailed directions with which the journey can successfully be made.

In the case of reaching a level of detail in the structure and function of human brains sufficient to build an electronic version of one, the missing pieces include a lack of comprehension of

  • The conditions under which an axon or a dendrite grows in length or bifurcates
  • The conditions under which the neuron fires based on internal structures that are known to hold state information within the cytoplasm.
  • The tie between the human genome and its varieties and the impact on structure of various genes, gene expression mechanisms, and their associated enzymes and proteins
  • Other complexities beyond my level of education in neuroscience.
  • Other complexities beyond everyone's level of education in neuroscience.

Layers and Hierarchies

It is common in science to seek layers or hierarchies to use in education and practice because they can assist in comprehending anatomical structure. That tendency has appeared in software engineering in operating system design, programming language design, application design, and now AI design. As these technology areas evolve, the trend is actually away from pure layer oriented or hierarchical design to more of a more unconstrained network of interconnecting parts. Simplicity is desired, but complexity is sometimes required.

Simulating intelligence is a demanding objective, and, simplicity having failed in the first half century of attempting to design intelligent digital systems, is clear that working solutions are going to require complexity and therefore considerable expertise.

It is reasonable to assume that there is no constraint guiding the evolution of human intelligence in the direction of structure characterized primarily by layer or hierarchic structure. Evolutionary processes take no notice of simplicity for the purpose easing academic study. There is nothing about DNA gene expression or the ways in which neurons grow in fetal or later stages that would enforce such rules of simplicity on structure or function.

How complex are the DNA expressions that lead to features of the brain that we consider to be intelligence? How complex are the neural systems that arise out those expressions? Some believe humanity will have to evolve before human minds can simulate themselves. Such conjecture could be true or false. Such is difficult to predict, even in order of magnitude.

Numerical Analysis of Optimistic Prediction

The expectation for exponential growth has been proposed for life expectancy, solar panel deployed capacity in Germany, CPU speed (microprocessor instruction executions per second), transistor density (Moore's "Law"), the size of the Communist Party, and many other metrics, but although growth rates in nature and human endeavors are often exponential in early stages, such has never proven sustainable. Growth rates are approximately linear for a short period afterward and become more arc-tangent shaped as saturation is approached. From saturation, values of the metric tend to decline and increase in chaotic fits and starts over long ranges of time.

In April 2005, Gordon Moore (author of Moore's "Law") stated, "[Exponential growth] can't continue forever. The nature of exponentials is that you push them out and eventually disaster happens." He later stated, "In terms of size [of transistors] you can see that we're approaching the size of atoms which is a fundamental barrier."

It is important to understand that Moore did not invent a law. He looked over two decades of data and noticed transistor density was roughly proportional to et, where t is the length of time since integrated circuits first reached mass market, and then predicted further exponential growth based on the clear trend data.

Realistic Prediction

Humans have never tried something as fundamentally threshold breaking as creating a simulation of self. Without related experience from which to know whether exponential growth, linear growth, arc tangent growth, or some other form is the most probable model, the safest model is probably the one Occam's Razor would prescribe, a linear prediction.

To make any prediction, one must collect some data points. Although this is a realistic prediction, it is not a very diligent one. Perhaps more work could be done to find a model that is more likely than a linear one, develop a system of theory and metrics to determine progress at any point in time, or collect more data points to establish a least squares fit. For the purposes of this answer, we will simply use two data points and do a linear extrapolation.

In 1660, Blaise Pascal wrote in his Pensées ("Thoughts"), "The arithmetical machine produces effects which approach nearer to thought than all the actions of animals. But it does nothing which would enable us to attribute will to it, as to the animals," so the search for mechanical simulations of human intelligence was already underway at that time.

Since then, computer programmers have developed code that accomplishes a number of human capabilities.

  • Generalization of numerical and logical calculations (CPUs)
  • Office automation
  • Pattern recognition (applied to writing, speech, and scenes)
  • Convergence on functionally optimal circuits (neural nets)
  • Application of probability to decisioning (Bayes' Theorem, etc)
  • Rules systems capable of excellence in discrete games

The features missing from the current extent digital simulations of intelligence of digital systems are important and numerous.

  • Intuition in enumerating general problem approaches
  • Excellence in natural language skills
  • Emotional expressiveness in art
  • Political expressiveness in art
  • Playing sports well (within robotic systems)
  • Doing a good job at work (given arbitrary instructions)
  • Learning to do new things at work
  • Getting a project started without prior experience in the domain
  • Comprehensive reductive analysis
  • Complex design of arbitrary physical devices per requirements)
  • Software development (software producing software per requirements)
  • Enlightened extension of an area of study
  • Identification of subterfuge in real time
  • Emotional intimacy
  • Compassion and empathy
  • Comprehensive self evaluation
  • Development of new areas of mathematics to prove an hypothesis
  • Go to class and learn more
  • Pick books and articles along a desired learning path and read them
  • Other capabilities along these lines

Given that this list is abridged and these items that have not yet been achieved in software are not as mechanically describable in natural language as those that have already been programmed successfully, we can be relatively sure that in 2017 we have digital systems that have achieved only a fraction of the breadth of the features of the complete set that people expect from a human being without calling the person mentally challenged. Judging from the list of accomplishments, no more than 10% of what intelligent people do has been simulated by computer software.

Without any reason to believe that the rate of discovery will wind down or ramp up (in spite of the claims that human advancement has been exponential 1) a simple linear approximation places the relatively complete electronic brain on the horizon for the year 5,587.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.