We changed our privacy policy. Read more.

Questions tagged [neural-networks]

For questions about a artificial networks, such as MLPs, CNNs, RNNs, LSTM, and GRU networks, their variants or any other AI system components that qualify as a neural networks in that they are, in part, inspired by biological neural networks.

Filter by
Sorted by
Tagged with
88
votes
7answers
16k views

Do scientists know what is happening inside artificial neural networks?

Do scientists or research experts know from the kitchen what is happening inside complex "deep" neural network with at least millions of connections firing at an instant? Do they understand the ...
72
votes
3answers
51k views

How can neural networks deal with varying input sizes?

As far as I can tell, neural networks have a fixed number of neurons in the input layer. If neural networks are used in a context like NLP, sentences or blocks of text of varying sizes are fed to a ...
54
votes
11answers
10k views

What are some well-known problems where neural networks don't do very well?

Background: It's well-known that neural networks offer great performance across a large number of tasks, and this is largely a consequence of their universal approximation capabilities. However, in ...
52
votes
4answers
14k views

Are neural networks prone to catastrophic forgetting?

Imagine you show a neural network a picture of a lion 100 times and label with "dangerous", so it learns that lions are dangerous. Now imagine that previously you have shown it millions of images of ...
50
votes
4answers
82k views

How to select number of hidden layers and number of memory cells in an LSTM?

I am trying to find some existing research on how to select the number of hidden layers and the size of these of an LSTM-based RNN. Is there an article where this problem is being investigated, i.e., ...
39
votes
6answers
21k views

How do capsule neural networks work?

Geoffrey Hinton has been researching something he calls "capsules theory" in neural networks. What is it? How do capsule neural networks work?
33
votes
5answers
64k views

What is the difference between a convolutional neural network and a regular neural network?

I've seen these terms thrown around this site a lot, specifically in the tags convolutional-neural-networks and neural-networks. I know that a neural network is a system based loosely on the human ...
33
votes
3answers
25k views

Why is Lisp such a good language for AI?

I've heard before from computer scientists and from researchers in the area of AI that that Lisp is a good language for research and development in artificial intelligence. Does this still apply, with ...
33
votes
4answers
1k views

How to find the optimal number of neurons per layer?

When you're writing your algorithm, how do you know how many neurons you need per single layer? Are there any methods for finding the optimal number of them, or is it a rule of thumb?
32
votes
4answers
15k views

Could a neural network detect primes?

I am not looking for an efficient way to find primes (which of course is a solved problem). This is more of a "what if" question. So, in theory, could you train a neural network to predict ...
30
votes
8answers
5k views

Is artificial intelligence vulnerable to hacking?

The paper The Limitations of Deep Learning in Adversarial Settings explores how neural networks might be corrupted by an attacker who can manipulate the data set that the neural network trains with. ...
30
votes
5answers
18k views

What is the purpose of an activation function in neural networks?

It is said that activation functions in neural networks help introduce non-linearity. What does this mean? What does non-linearity mean in this context? How does the introduction of this non-...
30
votes
2answers
1k views

How is a deep neural network different from other neural networks?

How is a neural network having the "deep" adjective actually distinguished from other similar networks?
30
votes
5answers
21k views

Is it possible to train the neural network to solve math equations?

I'm aware that neural networks are probably not designed to do that, however asking hypothetically, is it possible to train the deep neural network (or similar) to solve math equations? So given the ...
30
votes
5answers
30k views

What is the time complexity for training a neural network using back-propagation?

Suppose that a NN contains $n$ hidden layers, $m$ training examples, $x$ features, and $n_i$ nodes in each layer. What is the time complexity to train this NN using back-propagation? I have a basic ...
28
votes
4answers
4k views

Can neural networks be used to prove conjectures?

Imagine I have a list (in a computer-readable form) of all problems (or statements) and proofs that math relies on. Could I train a neural network in such a way that, for example, I enter a problem ...
26
votes
4answers
12k views

Is it possible to train a neural network as new classes are given?

I would like to train a neural network (NN) where the output classes are not (all) defined from the start. More and more classes will be introduced later based on incoming data. This means that, every ...
25
votes
4answers
407 views

Is the pattern recognition capability of CNNs limited to image processing?

Can a Convolutional Neural Network be used for pattern recognition in problem domains without image data? For example, by representing abstract data in an image-like format with spatial relations? ...
25
votes
5answers
29k views

How can I deal with images of variable dimensions when doing image segmentation?

I'm facing the problem of having images of different dimensions as inputs in a segmentation task. Note that the images do not even have the same aspect ratio. One common approach that I found in ...
23
votes
3answers
38k views

How do I handle large images when training a CNN?

Suppose that I have 10K images of sizes $2400 \times 2400$ to train a CNN. How do I handle such large image sizes without downsampling? Here are a few more specific questions. Are there any ...
23
votes
5answers
29k views

Can a neural network be used to predict the next pseudo random number?

Is it possible to feed a neural network the output from a random number generator and expect it learn the hashing (or generator) function, so that it can predict what will be the next generated pseudo-...
22
votes
3answers
9k views

How to choose an activation function for the hidden layers?

I choose the activation function for the output layer depending on the output that I need and the properties of the activation function that I know. For example, I choose the sigmoid function when I'm ...
22
votes
4answers
13k views

What is a Dynamic Computational Graph?

Frameworks like PyTorch and TensorFlow through TensorFlow Fold support Dynamic Computational Graphs and are receiving attention from data scientists. However, there seems to be a lack of resource to ...
21
votes
3answers
25k views

Can BERT be used for sentence generating tasks?

I am a new learner in NLP. I am interested in the sentence generating task. As far as I am concerned, one state-of-the-art method is the CharRNN, which uses RNN to generate a sequence of words. ...
21
votes
4answers
6k views

How could we build a neural network that is invariant to permutations of the inputs?

Given a neural network $f$ that takes as input $n$ data points: $x_1, \dots, x_n$. We say $f$ is permutation invariant if $$f(x_1 ... x_n) = f(\sigma(x_1 ... x_n))$$ for any permutation $\sigma$. How ...
20
votes
3answers
25k views

How can we process the data from both the true distribution and the generator?

I'm struggling to understand the GAN loss function as provided in Understanding Generative Adversarial Networks (a blog post written by Daniel Seita). In the standard cross-entropy loss, we have an ...
20
votes
3answers
2k views

Are there any computational models of mirror neurons?

From Wikipedia: A mirror neuron is a neuron that fires both when an animal acts and when the animal observes the same action performed by another. Mirror neurons are related to imitation learning, ...
19
votes
3answers
2k views

How are Artificial Neural Networks and the Biological Neural Networks similar and different?

I've heard multiple times that "Neural Networks are the best approximation we have to model the human brain", and I think it is commonly known that Neural Networks are modelled after our brain. I ...
18
votes
3answers
21k views

What are "bottlenecks" in neural networks?

What are "bottlenecks" in the context of neural networks? This term is mentioned, for example, in this TensorFlow article, which also uses the term "bottleneck values". How does ...
18
votes
1answer
449 views

Are these two versions of back-propagation equivalent?

Just for fun, I am trying to develop a neural network. Now, for backpropagation I saw two techniques. The first one is used here and in many other places too. What it does is: It computes the ...
17
votes
2answers
296 views

How do I decide the optimal number of layers for a neural network?

How do I decide the optimal number of layers for a neural network (feedforward or recurrent)?
17
votes
2answers
396 views

Are Modular Neural Networks more effective than large, monolithic networks at any tasks?

Modular/Multiple Neural networks (MNNs) revolve around training smaller, independent networks that can feed into each other or another higher network. In principle, the hierarchical organization ...
17
votes
1answer
414 views

Could a Boltzmann machine store more patterns than a Hopfield net?

This is from a closed beta for AI, with this question being posted by user number 47. All credit to them. According to Wikipedia, Boltzmann machines can be seen as the stochastic, generative ...
16
votes
3answers
6k views

Where can I find the proof of the universal approximation theorem?

The Wikipedia article for the universal approximation theorem cites a version of the universal approximation theorem for Lebesgue-measurable functions from this conference paper. However, the paper ...
16
votes
5answers
14k views

How can I design and train a neural network to play a card game (similar to Magic: The Gathering)?

Introduction I am currently writing an engine to play a card game, as there is no engine yet for this particular game. About the game The game is similar to Magic: The Gathering. There is a commander, ...
15
votes
3answers
780 views

How does noise affect generalization?

Does increasing the noise in data help to improve the learning ability of a network? Does it make any difference or does it depend on the problem being solved? How is it affect the generalization ...
15
votes
4answers
3k views

What activation function does the human brain use?

Does the human brain use a specific activation function? I've tried doing some research, and as it's a treshold for whether the signal is sent through a neuron or not, it sounds a lot like ReLU. ...
15
votes
4answers
21k views

1 hidden layer with 1000 neurons vs. 10 hidden layers with 100 neurons

These types of questions may be problem-dependent, but I have tried to find research that addresses the question whether the number of hidden layers and their size (number of neurons in each layer) ...
15
votes
2answers
673 views

How can I automate the choice of the topology of a neural network for an arbitrary problem?

Assume that I want to solve an issue with a neural network that either I can't fit to already existing topologies (perceptron, Konohen, etc) or I'm simply not aware of the existence of those or I'm ...
15
votes
4answers
2k views

What makes neural networks so good at predictions?

I am new to neural-network and I am trying to understand mathematically what makes neural networks so good at classification problems. By taking the example of a small neural network (for example, ...
15
votes
3answers
1k views

Has anyone thought about making a neural network ask questions, instead of only answering them?

Most of the people is trying to answer question with a neural network. However, has anyone came up with some thoughts about how to make neural network ask questions, instead of answer questions? For ...
14
votes
2answers
438 views

How should I encode the structure of a neural network into a genome?

For a deterministic problem space, I need to find a neural network with the optimal node and link structure. I want to use a genetic algorithm to simulate many neural networks to find the best network ...
14
votes
2answers
204 views

Is there any artificial intelligence that possesses "concentration"?

Humans can do multiple tasks at the same (e.g. reading while listening to music), but we memorize information from less focused sources with worse efficiency than we do from our main focus or task. ...
14
votes
2answers
719 views

Should deep residual networks be viewed as an ensemble of networks?

The question is about the architecture of Deep Residual Networks (ResNets). The model that won the 1-st places at "Large Scale Visual Recognition Challenge 2015" (ILSVRC2015) in all five main tracks: ...
14
votes
0answers
2k views

What's the main concept behind capsule networks? [duplicate]

As you might know, capsule networks have been recently introduced by Hinton. There also have been several heads up within his talks. As expected, the paper elaborates on the idea way theoretically! ...
13
votes
3answers
2k views

Why are the initial weights of neural networks randomly initialised?

This might sound silly to someone who has plenty of experience with neural networks but it bothers me... Random initial weights might give you better results that would be somewhat closer to what a ...
13
votes
1answer
16k views

What are the advantages of ReLU vs Leaky ReLU and Parametric ReLU (if any)?

I think that the advantage of using Leaky ReLU instead of ReLU is that in this way we cannot have vanishing gradient. Parametric ReLU has the same advantage with the only difference that the slope of ...
13
votes
4answers
8k views

Why do we need floats for using neural networks?

Is it possible to make a neural network that uses only integers by scaling input and output of each function to [-INT_MAX, INT_MAX]? Is there any drawbacks?
13
votes
2answers
2k views

How are generative adversarial networks trained?

I am reading about generative adversarial networks (GANs) and I have some doubts regarding it. So far, I understand that in a GAN there are two different types of neural networks: one is generative ($...
13
votes
3answers
4k views

What is the relationship between the size of the hidden layer and the size of the cell state layer in an LSTM?

I was following some examples to get familiar with TensorFlow's LSTM API, but noticed that all LSTM initialization functions require only the num_units parameter, ...

1
2 3 4 5
44