Questions tagged [neural-networks]

For questions about a artificial networks, such as MLPs, CNNs, RNNs, LSTM, and GRU networks, their variants or any other AI system components that qualify as a neural networks in that they are, in part, inspired by biological neural networks.

Filter by
Sorted by
Tagged with
99 votes
5 answers
84k views

How can neural networks deal with varying input sizes?

As far as I can tell, neural networks have a fixed number of neurons in the input layer. If neural networks are used in a context like NLP, sentences or blocks of text of varying sizes are fed to a ...
Asciiom's user avatar
  • 1,141
97 votes
7 answers
19k views

Do scientists know what is happening inside artificial neural networks?

Do scientists or research experts know from the kitchen what is happening inside complex "deep" neural network with at least millions of connections firing at an instant? Do they understand ...
kenorb's user avatar
  • 10.5k
64 votes
4 answers
17k views

Are neural networks prone to catastrophic forgetting?

Imagine you show a neural network a picture of a lion 100 times and label it with "dangerous", so it learns that lions are dangerous. Now imagine that previously you have shown it millions ...
zooby's user avatar
  • 2,196
56 votes
11 answers
12k views

What are some well-known problems where neural networks don't do very well?

Background: It's well-known that neural networks offer great performance across a large number of tasks, and this is largely a consequence of their universal approximation capabilities. However, in ...
ABIM's user avatar
  • 545
43 votes
5 answers
79k views

What is the difference between a convolutional neural network and a regular neural network?

I've seen these terms thrown around this site a lot, specifically in the tags convolutional-neural-networks and neural-networks. I know that a neural network is a system based loosely on the human ...
Mithical's user avatar
  • 2,895
43 votes
6 answers
21k views

How do capsule neural networks work?

Geoffrey Hinton has been researching something he calls "capsules theory" in neural networks. What is it? How do capsule neural networks work?
rcpinto's user avatar
  • 2,119
43 votes
5 answers
23k views

Could a neural network detect primes?

I am not looking for an efficient way to find primes (which of course is a solved problem). This is more of a "what if" question. So, in theory, could you train a neural network to predict ...
Fullk33's user avatar
  • 543
41 votes
5 answers
26k views

What is the purpose of an activation function in neural networks?

It is said that activation functions in neural networks help introduce non-linearity. What does this mean? What does non-linearity mean in this context? How does the introduction of this non-...
Mohsin's user avatar
  • 982
41 votes
4 answers
56k views

What is the time complexity for training a neural network using back-propagation?

Suppose that a NN contains $n$ hidden layers, $m$ training examples, $x$ features, and $n_i$ nodes in each layer. What is the time complexity to train this NN using back-propagation? I have a basic ...
user avatar
39 votes
3 answers
32k views

Why is Lisp such a good language for AI?

I've heard before from computer scientists and from researchers in the area of AI that that Lisp is a good language for research and development in artificial intelligence. Does this still apply, ...
Alecto Irene Perez's user avatar
35 votes
6 answers
27k views

Is it possible to train the neural network to solve math equations?

I'm aware that neural networks are probably not designed to do that, however asking hypothetically, is it possible to train the deep neural network (or similar) to solve math equations? So given the ...
kenorb's user avatar
  • 10.5k
33 votes
4 answers
2k views

How to find the optimal number of neurons per layer?

When you're writing your algorithm, how do you know how many neurons you need per single layer? Are there any methods for finding the optimal number of them, or is it a rule of thumb?
kenorb's user avatar
  • 10.5k
33 votes
5 answers
42k views

How can I deal with images of variable dimensions when doing image segmentation?

I'm facing the problem of having images of different dimensions as inputs in a segmentation task. Note that the images do not even have the same aspect ratio. One common approach that I found in ...
MattSt's user avatar
  • 597
31 votes
3 answers
41k views

Can BERT be used for sentence generating tasks?

I am a new learner in NLP. I am interested in the sentence generating task. As far as I am concerned, one state-of-the-art method is the CharRNN, which uses RNN to generate a sequence of words. ...
ch271828n's user avatar
  • 413
31 votes
7 answers
5k views

Is artificial intelligence vulnerable to hacking? [closed]

The paper The Limitations of Deep Learning in Adversarial Settings explores how neural networks might be corrupted by an attacker who can manipulate the data set that the neural network trains with. ...
Surya Sg's user avatar
  • 495
31 votes
2 answers
1k views

How is a deep neural network different from other neural networks?

How is a neural network having the "deep" adjective actually distinguished from other similar networks?
kenorb's user avatar
  • 10.5k
31 votes
5 answers
14k views

Is it possible to train a neural network as new classes are given?

I would like to train a neural network (NN) where the output classes are not (all) defined from the start. More and more classes will be introduced later based on incoming data. This means that, every ...
Fr_nkenstien's user avatar
30 votes
2 answers
36k views

What are "bottlenecks" in neural networks?

What are "bottlenecks" in the context of neural networks? This term is mentioned, for example, in this TensorFlow article, which also uses the term "bottleneck values". How does ...
Anurag Singh's user avatar
29 votes
3 answers
14k views

Where can I find the proof of the universal approximation theorem?

The Wikipedia article for the universal approximation theorem cites a version of the universal approximation theorem for Lebesgue-measurable functions from this conference paper. However, the paper ...
Leroy Od's user avatar
  • 455
29 votes
4 answers
41k views

Can a neural network be used to predict the next pseudo random number?

Is it possible to feed a neural network the output from a random number generator and expect it learn the hashing (or generator) function, so that it can predict what will be the next generated pseudo-...
AshTyson's user avatar
  • 401
29 votes
4 answers
5k views

Can neural networks be used to prove conjectures?

Imagine I have a list (in a computer-readable form) of all problems (or statements) and proofs that math relies on. Could I train a neural network in such a way that, for example, I enter a problem ...
Max Mustermann Junior's user avatar
28 votes
4 answers
11k views

How could we build a neural network that is invariant to permutations of the inputs?

Given a neural network $f$ that takes as input $n$ data points: $x_1, \dots, x_n$. We say $f$ is permutation invariant if $$f(x_1 ... x_n) = f(\sigma(x_1 ... x_n))$$ for any permutation $\sigma$. How ...
Josef Ondrej's user avatar
27 votes
4 answers
466 views

Is the pattern recognition capability of CNNs limited to image processing?

Can a Convolutional Neural Network be used for pattern recognition in problem domains without image data? For example, by representing abstract data in an image-like format with spatial relations? ...
dynrepsys's user avatar
  • 1,363
25 votes
3 answers
47k views

How do I handle large images when training a CNN?

Suppose that I have 10K images of sizes $2400 \times 2400$ to train a CNN. How do I handle such large image sizes without downsampling? Here are a few more specific questions. Are there any ...
WaterRocket8236's user avatar
25 votes
3 answers
84k views

How do I choose the optimal batch size?

Batch size is a term used in machine learning and refers to the number of training examples utilised in one iteration. The batch size can be one of three options: batch mode: where the batch size is ...
Sebastian Nielsen's user avatar
25 votes
4 answers
16k views

What is a Dynamic Computational Graph?

Frameworks like PyTorch and TensorFlow through TensorFlow Fold support Dynamic Computational Graphs and are receiving attention from data scientists. However, there seems to be a lack of resource to ...
Blaszard's user avatar
  • 1,037
24 votes
3 answers
12k views

How to choose an activation function for the hidden layers?

I choose the activation function for the output layer depending on the output that I need and the properties of the activation function that I know. For example, I choose the sigmoid function when I'm ...
gvgramazio's user avatar
23 votes
1 answer
29k views

What are the advantages of ReLU vs Leaky ReLU and Parametric ReLU (if any)?

I think that the advantage of using Leaky ReLU instead of ReLU is that in this way we cannot have vanishing gradient. Parametric ReLU has the same advantage with the only difference that the slope of ...
gvgramazio's user avatar
21 votes
5 answers
3k views

Why does Batch Normalization work?

Adding BatchNorm layers improves training time and makes the whole deep model more stable. That's an experimental fact that is widely used in machine learning practice. My question is - why does it ...
Kostya's user avatar
  • 2,516
20 votes
2 answers
8k views

How do neural networks play chess?

I have been spending a few days trying to wrap my head around how and why neural networks are used to play chess. Although I know very little about how the game of chess works, I can understand the ...
stats_noob's user avatar
20 votes
3 answers
4k views

How are Artificial Neural Networks and the Biological Neural Networks similar and different?

I've heard multiple times that "Neural Networks are the best approximation we have to model the human brain", and I think it is commonly known that Neural Networks are modelled after our brain. I ...
Andreas Storvik Strauman's user avatar
20 votes
4 answers
13k views

Why do we need floats for using neural networks?

Is it possible to make a neural network that uses only integers by scaling input and output of each function to [-INT_MAX, INT_MAX]? Is there any drawbacks?
elimohl's user avatar
  • 311
20 votes
3 answers
25k views

How can we process the data from both the true distribution and the generator?

I'm struggling to understand the GAN loss function as provided in Understanding Generative Adversarial Networks (a blog post written by Daniel Seita). In the standard cross-entropy loss, we have an ...
tryingtolearn's user avatar
19 votes
5 answers
20k views

How can I design and train a neural network to play a card game (similar to Magic: The Gathering)?

Introduction I am currently writing an engine to play a card game, as there is no engine yet for this particular game. About the game The game is similar to Magic: The Gathering. There is a commander, ...
pcaston2's user avatar
  • 311
19 votes
4 answers
2k views

What makes neural networks so good at predictions?

I am new to neural-network and I am trying to understand mathematically what makes neural networks so good at classification problems. By taking the example of a small neural network (for example, ...
Aditya Gupta's user avatar
19 votes
2 answers
543 views

Are Modular Neural Networks more effective than large, monolithic networks at any tasks?

Modular/Multiple Neural networks (MNNs) revolve around training smaller, independent networks that can feed into each other or another higher network. In principle, the hierarchical organization ...
Harsh Sikka's user avatar
19 votes
3 answers
2k views

Are there any computational models of mirror neurons?

From Wikipedia: A mirror neuron is a neuron that fires both when an animal acts and when the animal observes the same action performed by another. Mirror neurons are related to imitation learning, ...
rcpinto's user avatar
  • 2,119
19 votes
1 answer
986 views

What is the number of neurons required to approximate a polynomial of degree n?

I learned about the universal approximation theorem from this guide. It states that a network even with a single hidden layer can approximate any function within some bound, given a sufficient number ...
mark mark's user avatar
  • 763
18 votes
2 answers
508 views

How do I decide the optimal number of layers for a neural network?

How do I decide the optimal number of layers for a neural network (feedforward or recurrent)?
v01d's user avatar
  • 283
18 votes
1 answer
490 views

Are these two versions of back-propagation equivalent?

Just for fun, I am trying to develop a neural network. Now, for backpropagation I saw two techniques. The first one is used here and in many other places too. What it does is: It computes the ...
Aspie96's user avatar
  • 181
18 votes
1 answer
592 views

Could a Boltzmann machine store more patterns than a Hopfield net?

This is from a closed beta for AI, with this question being posted by user number 47. All credit to them. According to Wikipedia, Boltzmann machines can be seen as the stochastic, generative ...
Mithical's user avatar
  • 2,895
17 votes
4 answers
25k views

1 hidden layer with 1000 neurons vs. 10 hidden layers with 100 neurons

These types of questions may be problem-dependent, but I have tried to find research that addresses the question whether the number of hidden layers and their size (number of neurons in each layer) ...
Stephen Johnson's user avatar
16 votes
8 answers
27k views

How to classify data which is spiral in shape?

I have been messing around in tensorflow playground. One of the input data sets is a spiral. No matter what input parameters I choose, no matter how wide and deep the neural network I make, I cannot ...
Souradeep Nanda's user avatar
16 votes
5 answers
3k views

Why are the initial weights of neural networks randomly initialised?

This might sound silly to someone who has plenty of experience with neural networks but it bothers me... Random initial weights might give you better results that would be somewhat closer to what a ...
Matas Vaitkevicius's user avatar
16 votes
2 answers
740 views

How can I automate the choice of the architecture of a neural network for an arbitrary problem?

Assume that I want to solve an issue with a neural network that either I can't fit to existing architectures (perceptron, Konohen, etc) or I'm simply not aware of the existence of those or I'm unable ...
Zoltán Schmidt's user avatar
15 votes
1 answer
3k views

Why does the policy network in AlphaZero work?

In AlphaZero, the policy network (or head of the network) maps game states to a distribution of the likelihood of taking each action. This distribution covers all possible actions from that state. ...
chessprogrammer's user avatar
15 votes
4 answers
9k views

Why do activation functions need to be differentiable in the context of neural networks?

Why should an activation function of a neural network be differentiable? Is it strictly necessary or is it just advantageous?
user avatar
14 votes
3 answers
6k views

Why exactly do neural networks require i.i.d. data?

In reinforcement learning, successive states (actions and rewards) can be correlated. An experience replay buffer was used, in the DQN architecture, to avoid training the neural network (NN), which ...
nbro's user avatar
  • 40.2k
14 votes
3 answers
1k views

How does noise affect generalization?

Does increasing the noise in data help to improve the learning ability of a network? Does it make any difference or does it depend on the problem being solved? How is it affect the generalization ...
kenorb's user avatar
  • 10.5k
14 votes
2 answers
575 views

How should I encode the structure of a neural network into a genome?

For a deterministic problem space, I need to find a neural network with the optimal node and link structure. I want to use a genetic algorithm to simulate many neural networks to find the best network ...
Mithical's user avatar
  • 2,895

1
2 3 4 5
52