Why are inhibitory connections often used in virtual neural networks when they don't seem to exist in real life neural networks?

Question

Something I like about neural network AI is that we already have a blueprint for it - we know in great detail how different types of neurons work, and we don't have to invent the AI, we can just replicate what we already know works (neural networks) in a simplified way and train it.

So what's confusing me right now is why many popular neural models I've seen when studying neural networks include both inhibitory and stimulating connections. In real neural networks, from my understanding, there is no negative signal being transferred, rather the signals sent between neurons are comparable to values between 0.1 and 1. There's no mechanic for an inverse (inhibiting) signal to be sent.

For example, in this network (seen overlaying a snake being simulated), the red lines represent inhibiting connections, and the blue lines represent stimulating neurons:

Is this just an inconsequential detail of neural network design, where there's really no significant difference between a range of 0 to 1 and a range of -1 to 1? Or is there a reason that connections in our simulated neural networks benefit from being able to a express a range from -1 to 1 rather than just 0 to 1?

Answer 1

I think you are confused. The reason why some neural neural networks have neurons with an output in range (-1, 1) all depends on the activation function used. Some networks even have neurons with an output range of (-Infinity, +Infinity) ( aka the identity function ).

I advise you to take a look at this list: activation functions w/ ranges.

Nowadays a lot of neural networks have a mixture of different activation functions. E.g. combining ReLU with Sigmoid or TanH.

This neural network library even shows that for most simple problems, neural networks prefer a variety of activation functions.

The red lines just mean that that connection has a low value in comparison with other connections.

There is no rule of thumb for activation functions, some problems benefit from a certain function, some don't.

Answer 2

From what I've read on deep nets and neuroscience, your interpretation is backwards. Most 'neurons' in deep nets use RELU activations, which is strictly excitatory, while mammal brains are composed of both excitatory and inhibitory synapses/neurons.

https://en.wikipedia.org/wiki/Excitatory_postsynaptic_potential

https://en.wikipedia.org/wiki/Inhibitory_postsynaptic_potential

I think the reason for strictly excitatory activations like RELU in deep nets is that they provide a continuous gradient for backpropagation during learning. Inhibitory activations make the gradient discontinuous, greatly complicating the propagation of feedback/error.