What could loosely be considered inhibitory effect occurs in multilayer perceptrons (ANNs) as they are normally designed and implemented already.

• One cannot inhibit a pulse through an ANN because there ARE NO PULSES in ANNs.
• One cannot alter the signal attenuation between neurons by varying a numeric parameter either, since there is no numeric parameter array in a biological net.

Adding to the Disparity Between ANNs and Biology

Some researchers have deviated entirely from the multilayer perceptron design (used in ANNs) and favored a pulse based system that requires specialized hardware. Follow the money there. It is not an inexpensive research avenue yet. But it may become one if they have success.

(I brought up the absurdity of using an additive adjustment in ANN back propagation in a question I wrote for this site. The responses to the challenge to the status quo not particularly well understood by the majority of machine learning practitioners were not outstanding.)

What could loosely be considered inhibitory effect occurs in MLPs (multilayer perceptrons) as they are normally designed and implemented already.

• One cannot inhibit a pulse through a MLP because there ARE NO PULSES in MLPs.
• One cannot alter the signal attenuation between neurons by varying a numeric parameter either, since there is no numeric parameter array in a biological net.

Adding to the Disparity Between MLPs and Biology

Some researchers have deviated entirely from the multilayer perceptron design and favored a pulse based system that requires specialized hardware. Follow the money there. It is not an inexpensive research avenue yet. But it may become one if they have success.

(I brought up the absurdity of using an additive adjustment in MLP back propagation in a question I wrote for this site. The responses to the challenge to the status quo not particularly well understood by the majority of machine learning practitioners were not outstanding.)

What could loosely be considered as inhibitory effects occur in multilayer perceptrons (ANNs) as they are normally designed and implemented.

The gradient descent scheme implemented within a larger back propagation algorithm can produce a parameter adjustment delta that is either positive or negative. A positive value decreases the attenuation of that parameter's signal path, thereby increasing the signal strength there. A negative value increases the attenuation of that path, thereby decreasing signal strength to that connection.

A decrease in a parameter's value as a result of back propagation bears some similarity to the inhibition of a neural signal path, however, you may already be aware that the way signals travel in biological neurons is significantly different than the way signals travel through artificial networks used commonly in machine learning.

The term inhibition is, as mentioned, only loosely applicable. One cannot inhibit a pulse through an ANN because there ARE NO PULSES in ANNs.

Stimulation and inhibition in the brains of mammals are also different in that neuro-chemistry impacts the network regionally, so the terms stimulation and inhibition are a bit ambiguous, since we have agonists and antagonists ranging from dopamine to serotonin and from canibanoids to oxytosin receptors and from endorphins to other classes.

The former thinking was that a pulse travelling through a biological signal path strengthened that connection. No one adheres to that simplistic conception today. It is possible that a largely used signaling path followed by a sharp pain would cause the inhibition of further pulses along that path. Although I am not well trained in electro-chemical processes in neural pathways, I recall in vitro experiments supporting that model. The current view of addiction as a brain disease is that a breakdown of the interrelationship between chemica state change and learned inhibition vs transmission occurs is causal.

Also, stimulation and inhibition are not antonyms. The opposite of inhibiting a signal is the transmission of it. The opposite of stimulation is the lack of stimulation (no signal).

Trying to draw parallels between ReLU and the functions of synapses with the interplay with regional brain chemistry and the retention functions of the cell organelles will probably not produce value.

In a sense inhibition in the brain occurs at multiple architectural levels, inside the cell, between cells, and over structures of cells, and the alignment of pulses temporally (in the time domain) is not simulated at all in conventional machine learning constructs. Some researchers have deviated entirely from the multilayer perceptron design and favored a pulse based system that requires specialized hardware. Follow the money there. It is not an inexpensive research avenue yet. But it may become one if they have success.

You are correct that there are non-linearities of different types in biological neural circuits. Potential change is not only not a straight line function, but its function's curvature changes over short periods of time. It is temporally sensitive, and the memory in a cell forms through neural plasticity and internally to the cell membrane within the organelles. That memory also deteriorates at a roughly inverse exponential rate with respect to time.

The second non-linearity is not a sum either. It appears to be a surface that is not flat. Also, as mentioned, the temporal alignment presents a complexity, since perfect pulse alignment is not treated the same as a slight skew in time. I brought up the absurdity of using an additive adjustment in ANN back propagation in a question I wrote for this site, but the responses were not outstanding.

To a large degree linear thinking (in the wider sense of the term) pervades mainstream machine learning and data science today, the activation functions being a notable and welcomed exception. Over time, I expect that will improve. I see current leading edge research going beyond that linear thinking and considering short and long term memory as in the LSTM and attention based networks, the simulation of the curved surfaces that represent pulse propagation in mammalian nets and the consideration of various applications of exponential decay here and there in the latest literature.

The Degree to Which Inhibition is in Common Use

What could loosely be considered inhibitory effect occurs in multilayer perceptrons (ANNs) as they are normally designed and implemented already.

The gradient descent scheme implemented within a larger back propagation algorithm can produce a parameter adjustment delta that is either positive or negative.

• A positive value decreases the attenuation of that parameter's signal path, thereby increasing the signal strength there.
• A negative value increases the attenuation of that path, thereby decreasing signal strength through that connection.

A decrease in a parameter's value as a result of back propagation bears some similarity to the inhibition of a neural signal path, however, you may already be aware of the significant differences in the signaling between biological neurons and the signalling between layers in the type of artificial networks commonly in machine learning.

The term inhibition is, as mentioned, only loosely applicable.

• One cannot inhibit a pulse through an ANN because there ARE NO PULSES in ANNs.
• One cannot alter the signal attenuation between neurons by varying a numeric parameter either, since there is no numeric parameter array in a biological net.

Stimulation and inhibition in the brains of mammals are also different in that neuro-chemistry impacts the network regionally, so the terms stimulation and inhibition are a bit ambiguous, since we have agonists and antagonists ranging from dopamine to serotonin and from cannabanoids to oxytocin receptors and from endorphins to other classes.

Changes from Former Textbook Themes

The former thinking was that a pulse travelling through a biological signal path strengthened that connection. No one in neurology research adheres to that simplistic a conception today.

For example, it is know that a signal pathway may be in common use but may close down by repeated sharp pains following its use. Although I am not well trained in electro-chemical processes in neural pathways, I recall in vitro experiments supporting that this is neither neuro-plastic nor electrical, that it is related to regional chemical feedback.

The current view of addiction as a brain disease is that a breakdown of the interrelationship between chemical state change and learned inhibition or transmission is causal. Inhibition or transmission is no longer decided upon based on organism survival and socialization but on the addictive stimuli, leading to behavioral dysfunction.

It may be useful to point out that, stimulation and inhibition are not strictly antonyms. The opposite of inhibiting a signal is the transmission of it. The opposite of stimulation is the lack of stimulation (no signal).

Attempting Analogy in Largely Dissimilar Circuit Models

It may not be an aid to general understanding to draw parallels between ReLU activation functions and the functions of synapses with their sensitivity to regional brain chemistry and with cell-level retention functions orchestrated by organelles.

Adding to the Disparity Between ANNs and Biology

In a sense inhibition in the brain occurs at multiple architectural levels, inside the cell, between cells, and over structures of cells, and the alignment of pulses temporally (in the time domain) is not simulated at all in conventional machine learning constructs.

Some researchers have deviated entirely from the multilayer perceptron design (used in ANNs) and favored a pulse based system that requires specialized hardware. Follow the money there. It is not an inexpensive research avenue yet. But it may become one if they have success.

Curvature in Functions

Brief note on terminology: Nth degree polynomials fall under linear algebra, so the best term to use is 'curved functions' so as to not fall into the ambiguity of the term non-linear.

Nonetheless, you are correct that there are non-linearities of different types in biological neural circuits. Potential change is not only curved, but its function's curvature changes quickly. It is temporally sensitive.

On the longer time frame, the memory in a cell forms through neural plasticity and the cell behavior changes internally (within the cell membrane) employing cytoplasm and the suspended organelles. That memory function also attenuates at a roughly inverse exponential rate with respect to time but some have hypothesized based in empirical evidence that forgotten cellular function can be recalled. Again, this is at the cellular level.

The second non-linearity is not a sum of potentials. The surface of the function that aggregates incoming signals is not flat. It is curved. Also, as mentioned, the temporal alignment presents a complexity, since perfect pulse alignment is not treated the same as pulses not perfectly aligned in time.

(I brought up the absurdity of using an additive adjustment in ANN back propagation in a question I wrote for this site. The responses to the challenge to the status quo not particularly well understood by the majority of machine learning practitioners were not outstanding.)

Linear Thinking Prevails Currently

To a large degree linear thinking (in the wider sense of the term) pervades mainstream machine learning and data science today, the activation functions being a notable and welcomed exception.

Over time, I expect that will improve. I see current leading edge research going beyond that linear thinking and considering short and long term memory as in the LSTM and attention based networks, the simulation of the curved surfaces that represent pulse propagation in mammalian nets and the consideration of various applications of exponential decay here and there in the latest literature.

Gratitude for the Question