19
votes
Accepted
How to find the optimal number of neurons per layer?
There is no direct way to find the optimal number of them: people empirically try and see (e.g., using cross-validation). The most common search techniques are random, manual, and grid searches.
...
- 2,804
12
votes
Accepted
What does it mean for a neuron in a neural network to be activated?
A neuron is said activated when its output is more than a threshold, generally 0.
For examples :
\begin{equation}
y = Relu(a) > 0
\end{equation}
when
\begin{equation}
a = w^Tx+b > 0
\end{...
- 515
8
votes
Accepted
Do we know what the units of neural networks will do before we train them?
In reverse order to how you asked:
all units in a layer become equal since initially the errors due to all of them are the same and thus we train them to be equal
This actually happens if you ...
- 30.2k
7
votes
How to find the optimal number of neurons per layer?
For a more intelligent approach than random or exhaustive searches, you could try a genetic algorithm such as NEAT http://nn.cs.utexas.edu/?neat. However, this has no guarantee to find a global optima,...
- 712
6
votes
How to find the optimal number of neurons per layer?
Paper Szegedy C, Vanhoucke V, Ioffe S, et al. Rethinking the inception architecture for computer vision[J]. arXiv preprint arXiv:1512.00567, 2015. gives some general design principles:
Avoid ...
- 161
5
votes
Accepted
Are neurons in layer $l$ only affected by neurons in the previous layer?
It depends on the architecture of the neural network. However, in general, no, neurons at layer $l$ are not only affected by neurons at layer $l-1$.
In the case of a multi-layer perceptron (or feed-...
- 39.5k
4
votes
Accepted
Which artificial neural network can mimic biological neurons the most?
Only a small portion of the habituation, sensitization, and classical conditioning behavior of neurons has been primitively simulated in ANN systems. Simulation of actin cytoskeletal machinery1 and ...
- 7,443
4
votes
Accepted
How do biological neurons weights get initialized?
In short
I mentioned in another post, how the Artificial Neural Network (ANN) weights are a relatively crude abstraction of connections between neurons in the brain. Similarly, the random weight ...
- 156
4
votes
Is there research that employs realistic models of neurons?
It looks like you really have two questions here. I'll try to answer the first one, and you should think about making a separate question for the second.
There is research into using simulated models ...
- 9,207
4
votes
What does it mean for a neuron in a neural network to be activated?
The term "activated" is mostly used when talking about activation functions which only outputs a value (except 0) when the input to the activation function is greater than a certain treshold.
...
- 558
4
votes
Accepted
Is there research that employs realistic models of neurons?
State of Rosehip Research
The Rosehip neuron is an important discovery, with vast implications to AI and its relationship to the dominant intelligence on earth for at least the last 50,000 years. ...
- 7,443
4
votes
How do layers in an artificial neural network transform inputs to outputs?
The basic calculation for a single neuron is of the form
$$\sigma\left(\sum_{i} x_i w_i \right),$$
where $x_i$ is the input to the neuron $w_i$ are the neuron-specific weights for every single ...
4
votes
Accepted
Can neurons in MLP and filters in CNN be compared?
tl;dr The equivalent to a neuron in a Fully-Connected (FC) layer is the kernel (or filter) of a Convolution layer
Differences
The neurons of these two types of layers have two key differences. These ...
- 3,173
3
votes
Is there research that employs realistic models of neurons?
It is true that the current Machine learning is based on treating neurons as a component in the whole complexity , mesh of neurons. The focus is more on the architecture rather than understanding or ...
- 160
3
votes
How to model inhibitory synapses in the artificial neuron?
Principles of Computational Modelling in Neuroscience by David Sterratt, Bruce Graham, Andrew Gillies and David Willshaw discuss it in Chapter 7 (The synapse) and also in Chapter 8 (Simplified models ...
- 131
3
votes
How many nodes/hidden layers are required to solve a classification problem where the boundary is a sinusoidal function?
It depends on the accuracy you want. If you had 1 neuron, it could discern things across a line, if you have 2, you could solve things across 2 lines, etc. As you increase the number of neurons, you ...
- 3,787
3
votes
How can I train a neural network for another input set, without losing the learning of the previous input set?
Yes, this is actually a limitation known as catastrophic forgetting.
A proposed way to deal with this is elastic weight consolidation that "remembers old tasks by selectively slowing down learning on ...
- 216
3
votes
Accepted
Building an AI that generates text by itself
There have been many methods proposed for text generating, but recurrent network dominates natural language processing with a key component: the perception of time.
Many networks have been tried for ...
- 1,735
3
votes
Accepted
What effect does a negative output of a neuron have on neighbouring neurons?
In the case of artificial neural networks, your question can be (partially) answered by looking at the definition of the operation that an artificial neuron performs. An artificial neuron is usually ...
- 39.5k
2
votes
Accepted
Is the neuron adequately comprehended?
No, here is why. No approach can simulate the mind with 100% accuracy. a major notion that AI theorist refuse to note is that you cant take an orange and by virtue of technology turn it into an apple ...
- 194
2
votes
Which artificial neural network can mimic biological neurons the most?
ANNs approximate biological neuronal networks. The approximation began with extreme simplicity in the early perceptron design. Spiking networks are examples of more accurate approximations. More ...
- 129
2
votes
Which artificial neural network can mimic biological neurons the most?
Most artificial neurons model biological neurons but in a very simplistic way. Nowadays, the main aim is to achieve better performance at prediction tasks. However, there is a body of literature in ...
2
votes
How to model inhibitory synapses in the artificial neuron?
In biology, when the presynaptic releases a neurotransmitter (a positive amount of them, obviously), this neurotransmitter reaches the postsynaptic vesicles causing an excitatory (depolarization) or ...
- 1,283
2
votes
How to model inhibitory synapses in the artificial neuron?
The Degree to Which Inhibition is in Common Use
What could loosely be considered inhibitory effect occurs in MLPs (multilayer perceptrons) as they are normally designed and implemented already.
The ...
- 7,443
2
votes
Method to compute the sum when the activation is a continuous function?
In terms of the normal use cases for machine learning, the equation does not have much utility, because:
Consider we have a curve $f(x)$ now if one wishes to . . .
In most AI problems, we don't ...
- 30.2k
2
votes
Accepted
Is input normalization built-in into mammals sensory neurons?
Yes, for many sensory inputs there is indeed something similar to normalization. But its not rally the same as in classical data analytics compared to what eg min/max normalization does or other ...
- 136
2
votes
Decide Number of input Parameters and Output Parameters - ANN
This should be possible given the fact that ANNs have the ability to do the feature engineering and feature selection tasks by themselves.
This means that given a lesser number of input parameters, ...
- 393
2
votes
Accepted
How do layers in an artificial neural network transform inputs to outputs?
Simon Krannig's answer provides the math notation behined exactly what is going on, but since you still seem a bit confused, I've made a visual representation of a neural network using only weights ...
- 1,346
Only top scored, non community-wiki answers of a minimum length are eligible