19 votes
Accepted

How to find the optimal number of neurons per layer?

There is no direct way to find the optimal number of them: people empirically try and see (e.g., using cross-validation). The most common search techniques are random, manual, and grid searches. ...
user avatar
11 votes
Accepted

What does it mean for a neuron in a neural network to be activated?

A neuron is said activated when its output is more than a threshold, generally 0. For examples : \begin{equation} y = Relu(a) > 0 \end{equation} when \begin{equation} a = w^Tx+b > 0 \end{...
user avatar
8 votes
Accepted

Do we know what the units of neural networks will do before we train them?

In reverse order to how you asked: all units in a layer become equal since initially the errors due to all of them are the same and thus we train them to be equal This actually happens if you ...
user avatar
  • 24.5k
7 votes

How to find the optimal number of neurons per layer?

For a more intelligent approach than random or exhaustive searches, you could try a genetic algorithm such as NEAT http://nn.cs.utexas.edu/?neat. However, this has no guarantee to find a global optima,...
user avatar
6 votes

How to find the optimal number of neurons per layer?

Paper Szegedy C, Vanhoucke V, Ioffe S, et al. Rethinking the inception architecture for computer vision[J]. arXiv preprint arXiv:1512.00567, 2015. gives some general design principles: Avoid ...
user avatar
  • 161
5 votes
Accepted

Are neurons in layer $l$ only affected by neurons in the previous layer?

It depends on the architecture of the neural network. However, in general, no, neurons at layer $l$ are not only affected by neurons at layer $l-1$. In the case of a multi-layer perceptron (or feed-...
user avatar
  • 34.4k
4 votes

How do layers in an artificial neural network transform inputs to outputs?

The basic calculation for a single neuron is of the form $$\sigma\left(\sum_{i} x_i w_i \right),$$ where $x_i$ is the input to the neuron $w_i$ are the neuron-specific weights for every single ...
user avatar
4 votes

Is there research that employs realistic models of neurons?

It looks like you really have two questions here. I'll try to answer the first one, and you should think about making a separate question for the second. There is research into using simulated models ...
user avatar
4 votes

What does it mean for a neuron in a neural network to be activated?

The term "activated" is mostly used when talking about activation functions which only outputs a value (except 0) when the input to the activation function is greater than a certain treshold. ...
user avatar
4 votes
Accepted

How do biological neurons weights get initialized?

In short I mentioned in another post, how the Artificial Neural Network (ANN) weights are a relatively crude abstraction of connections between neurons in the brain. Similarly, the random weight ...
user avatar
  • 156
3 votes

Is there research that employs realistic models of neurons?

It is true that the current Machine learning is based on treating neurons as a component in the whole complexity , mesh of neurons. The focus is more on the architecture rather than understanding or ...
user avatar
3 votes

How to model inhibitory synapses in the artificial neuron?

Principles of Computational Modelling in Neuroscience by David Sterratt, Bruce Graham, Andrew Gillies and David Willshaw discuss it in Chapter 7 (The synapse) and also in Chapter 8 (Simplified models ...
user avatar
  • 131
3 votes

How many nodes/hidden layers are required to solve a classification problem where the boundary is a sinusoidal function?

It depends on the accuracy you want. If you had 1 neuron, it could discern things across a line, if you have 2, you could solve things across 2 lines, etc. As you increase the number of neurons, you ...
user avatar
3 votes

How can I train a neural network for another input set, without losing the learning of the previous input set?

Yes, this is actually a limitation known as catastrophic forgetting. A proposed way to deal with this is elastic weight consolidation that "remembers old tasks by selectively slowing down learning on ...
user avatar
3 votes
Accepted

Building an AI that generates text by itself

There have been many methods proposed for text generating, but recurrent network dominates natural language processing with a key component: the perception of time. Many networks have been tried for ...
user avatar
  • 1,715
3 votes
Accepted

What effect does a negative output of a neuron have on neighbouring neurons?

In the case of artificial neural networks, your question can be (partially) answered by looking at the definition of the operation that an artificial neuron performs. An artificial neuron is usually ...
user avatar
  • 34.4k
3 votes
Accepted

Can neurons in MLP and filters in CNN be compared?

tl;dr The equivalent to a neuron in a Fully-Connected (FC) layer is the kernel (or filter) of a Convolution layer Differences The neurons of these two types of layers have two key differences. These ...
user avatar
  • 3,103
2 votes
Accepted

Is the neuron adequately comprehended?

No, here is why. No approach can simulate the mind with 100% accuracy. a major notion that AI theorist refuse to note is that you cant take an orange and by virtue of technology turn it into an apple ...
user avatar
2 votes

Which artificial neural network can mimic biological neurons the most?

ANNs approximate biological neuronal networks. The approximation began with extreme simplicity in the early perceptron design. Spiking networks are examples of more accurate approximations. More ...
user avatar
2 votes

Which artificial neural network can mimic biological neurons the most?

Most artificial neurons model biological neurons but in a very simplistic way. Nowadays, the main aim is to achieve better performance at prediction tasks. However, there is a body of literature in ...
2 votes

How to model inhibitory synapses in the artificial neuron?

In biology, when the presynaptic releases a neurotransmitter (a positive amount of them, obviously), this neurotransmitter reaches the postsynaptic vesicles causing an excitatory (depolarization) or ...
user avatar
2 votes

Method to compute the sum when the activation is a continuous function?

In terms of the normal use cases for machine learning, the equation does not have much utility, because: Consider we have a curve $f(x)$ now if one wishes to . . . In most AI problems, we don't ...
user avatar
  • 24.5k
2 votes
Accepted

Is input normalization built-in into mammals sensory neurons?

Yes, for many sensory inputs there is indeed something similar to normalization. But its not rally the same as in classical data analytics compared to what eg min/max normalization does or other ...
user avatar
  • 136
2 votes

Decide Number of input Parameters and Output Parameters - ANN

This should be possible given the fact that ANNs have the ability to do the feature engineering and feature selection tasks by themselves. This means that given a lesser number of input parameters, ...
user avatar
2 votes
Accepted

How do layers in an artificial neural network transform inputs to outputs?

Simon Krannig's answer provides the math notation behined exactly what is going on, but since you still seem a bit confused, I've made a visual representation of a neural network using only weights ...
user avatar
  • 1,246

Only top scored, non community-wiki answers of a minimum length are eligible