The Deep Learning book by Goodfellow et al. states
Convolutional networks stand out as an example of neuroscientific principles influencing deep learning.
Are convolutional neural networks (CNNs) really inspired by the human brain?
If so, how? In particular, what structures within the brain do CNN-like neuron groupings occur?
Yes, CNNs are inspired by the human brain [1, 2, 3]. More specifically, their operations, the convolution and pooling, are inspired by the human brain. However, note that, nowadays, CNNs are mainly trained with gradient descent (GD) and back-propagation (BP), which seems not to be a biologically plausible way of learning, but, given the success of GD and BP, there have been attempts to connect GB and BP with the way humans learn [4].
The neocognitron, the first convolutional neural network [1], proposed by Kunihiko Fukushima in 1979-1980, and described in the paper Neocognitron: A Self-organizing Neural Network Model for a Mechanism of Pattern Recognition Unaffected by Shift in Position, already uses convolutional and pooling (specifically, averaging pooling) layers [1]. The neocognitron was inspired by the work of Hubel and Wiesel described in the 1959 paper Receptive fields of single neurones in the cat's striate cortex.
Here is an excerpt from the 1980 Fukushima's paper.
The mechanism of pattern recognition in the brain is little known, and it seems to be almost impossible to reveal it only by conventional physiological experiments. So, we take a slightly different approach to this problem. If we could make a neural network model which has the same capability for pattern recognition as a human being, it would give us a powerful clue to the understanding of the neural mechanism in the brain. In this paper, we discuss how to synthesize a neural network model in order to endow it an ability of pattern recognition like a human being.
Several models were proposed with this intention (Rosenblatt, 1962; Kabrisky, 1966; Giebel, 1971; Fukushima, 1975). The response of most of these models, however, was severely affected by the shift in position and/or by the distortion in shape of the input patterns. Hence, their ability for pattern recognition was not so high.
In this paper, we propose an improved neural network model. The structure of this network has been suggested by that of the visual nervous system of the vertebrate. This network is self-organized by "learning without a teacher", and acquires an ability to recognize stimulus patterns based on the geometrical similarity (Gestalt) of their shapes without affected by their position nor by small distortion of their shapes. This network is given a nickname "neocognitron", because it is a further extention of the "cognitron", which also is a self-organizing multilayered neural network model proposed by the author before (Fukushima, 1975)
However, Fukushima did not train the neocognitron with gradient descent (and back-propagation) but with local learning rules (which are more biologically plausible), and that's probably why he doesn't get more credit, as I think he should.
You should read at least Fukushima's paper for more details, which I will not replicate here.
Section 9.4 of the Deep Learning book also contains details about how CNNs are inspired by neuroscience findings.
In the eye, the retinal ganglion cells have a receptive field that is equivalent to some types of convolution filters, most of them edge detectors.
The brain is a big unknown, nobody knows how it does to organize, memorize, create concepts, learns the language, ... . Thus, it is not possible to establish a parallelism.
In particular, brain has a capacity of handle independence in scale and rotation that CNN's are not able to reproduce.
As general remark about NN and brain: even when it is always said that "neural network cells" are "inspired" in biological neurons, there are critical differences that made this similitude only a "inspiration". Thus, comparison of a CNN or any other kind of NN with brain is always a fuzzy comparison. The biggest difference is probably the learning capacity: the human brain learns by itself, while the neural network needs an external system (the back-propagation algorithm) that feeds the NN with the learned parameters.
