Questions tagged [neural-networks]

For questions about a artificial networks, such as MLPs, CNNs, RNNs, LSTM, and GRU networks, their variants or any other AI system components that qualify as a neural networks in that they are, in part, inspired by biological neural networks.

Filter by
Sorted by
Tagged with
69
votes
8answers
11k views

Do scientists know what is happening inside artificial neural networks?

Do scientists or research experts know from the kitchen what is happening inside complex "deep" neural network with at least millions of connections firing at an instant? Do they understand the ...
41
votes
3answers
27k views

How can neural networks deal with varying input sizes?

As far as I can tell, neural networks have a fixed number of neurons in the input layer. If neural networks are used in a context like NLP, sentences or blocks of text of varying sizes are fed to a ...
37
votes
4answers
9k views

Are neural networks prone to catastrophic forgetting?

Imagine you show a neural network a picture of a lion 100 times and label with "dangerous", so it learns that lions are dangerous. Now imagine that previously you have shown it millions of images of ...
34
votes
5answers
21k views

How does Hinton's “capsules theory” work?

Geoffrey Hinton has been researching something he calls "capsules theory" in neural networks. What is this and how does it work?
29
votes
3answers
17k views

Why is Lisp such a good language for AI?

I've heard before from computer scientists and from researchers in the area of AI that that Lisp is a good language for research and development in artificial intelligence. Does this still apply, with ...
28
votes
4answers
38k views

How to select number of hidden layers and number of memory cells in an LSTM?

I am trying to find some existing research on how to select the number of hidden layers and the size of these of an LSTM-based RNN. Is there an article where this problem is being investigated, i.e., ...
27
votes
9answers
5k views

Is artificial intelligence vulnerable to hacking?

The paper The Limitations of Deep Learning in Adversarial Settings explores how neural networks might be corrupted by an attacker who can manipulate the data set that the neural network trains with. ...
24
votes
2answers
811 views

How is a deep neural network different from other neural networks?

How is a neural network having the "deep" adjective actually distinguished from other similar networks?
23
votes
5answers
14k views

Is it possible to train the neural network to solve math equations?

I'm aware that neural networks are probably not designed to do that, however asking hypothetically, is it possible to train the deep neural network (or similar) to solve math equations? So given the ...
23
votes
2answers
7k views

Is it possible to train a neural network incrementally?

I would like to train a neural network where the output classes are not (all) defined from the start. More and more classes will be introduced later based on incoming data. This means that, every time ...
21
votes
4answers
314 views

Is the pattern recognition capability of CNNs limited to image processing?

Can a Convolutional Neural Network be used for pattern recognition in a problem domain where there are no pre-existing images, say by representing abstract data graphically? Would that always be less ...
21
votes
4answers
7k views

Could a neural network detect primes?

I am not looking for an efficient way to find primes (which of course is a solved problem). This is more of a "what if" question. So, in theory: Could you train a neural network to predict whether ...
21
votes
4answers
3k views

Can deep networks be trained to prove theorems?

Assume we have a large number of proofs in first order predicate calculus. Assume we also have the axioms, corollaries, and theorems in that area of mathematics in that form too. Consider the each ...
18
votes
4answers
9k views

What is a Dynamic Computational Graph?

Frameworks like PyTorch and TensorFlow through TensorFlow Fold support Dynamic Computational Graphs and are receiving attention from data scientists. However, there seems to be a lack of resource to ...
17
votes
3answers
1k views

Are there any computational models of mirror neurons?

From Wikipedia: A mirror neuron is a neuron that fires both when an animal acts and when the animal observes the same action performed by another. Mirror neurons are related to imitation learning, ...
16
votes
3answers
27k views

What is the difference between a Convolutional Neural Network and a regular Neural Network?

I've seen these terms thrown around this site a lot, specifically in the tags convolutional-neural-networks and neural-networks. I know that a Neural Network is a system based loosely on the human ...
16
votes
3answers
21k views

Understanding GAN loss function

I'm struggling to understand the GAN loss function as provided in Understanding Generative Adversarial Networks (a blog post written by Daniel Seita). In the standard cross-entropy loss, we have an ...
16
votes
1answer
341 views

Could a Boltzmann machine store more patterns than a Hopfield net?

This is from a closed beta for AI, with this question being posted by user number 47. All credit to them. According to Wikipedia, Boltzmann machines can be seen as the stochastic, generative ...
15
votes
1answer
341 views

Differences between backpropagation techniques

Just for fun, I am trying to develop a neural network. Now, for backpropagation I saw two techniques. The first one is used here and in many other places too. What it does is: It computes the ...
14
votes
3answers
21k views

How to handle images of large sizes in CNN?

Suppose there are 10K images of sizes 2400 x 2400 are required to use in CNN.Acc to my view conventional computers the people use will be of use. Now the question is how to handle such large image ...
14
votes
3answers
1k views

How are Artificial Neural Networks and the Biological Neural Networks similar and different?

I've heard multiple times that "Neural Networks are the best approximation we have to model the human brain", and I think it is commonly known that Neural Networks are modelled after our brain. I ...
14
votes
2answers
8k views

What is the time complexity for training a neural network using back-propagation?

Suppose that a NN contains $n$ hidden layers, $m$ training examples, $x$ features, and $n_i$ nodes in each layer. What is the time complexity to train this NN using back-propagation? I have a basic ...
14
votes
3answers
2k views

Permutation invariant neural networks

Given a neural network $f$ that takes as input $n$ data points: $x_1, \dots, x_n$. We say $f$ is permutation invariant if $$f(x_1 ... x_n) = f(pi(x_1 ... x_n))$$ for any permutation $pi$. ...
14
votes
2answers
1k views

What's the main concept behind Capsule Networks? [duplicate]

As you might know, Capsule Networks have been recently introduced by Hinton. There also have been several heads up within his talks. As expected, the paper elaborates on the idea way theoretically! ...
14
votes
3answers
951 views

Has anyone thought about making a neural network ask questions, instead of only answering them?

Most of the people is trying to answer question with a neural network. However, has anyone came up with some thoughts about how to make neural network ask questions, instead of answer questions? For ...
14
votes
2answers
183 views

Are Modular Neural Networks more effective than large, monolithic networks at any tasks?

Modular/Multiple Neural networks (MNNs) revolve around training smaller, independent networks that can feed into each other or another higher network. In principle, the hierarchical organization ...
13
votes
6answers
6k views

What is the purpose of an activation function in Neural Networks?

It is said that activation functions in neural networks help introduce non-linearity. What does this mean? What does non-linearity mean in this context? How does introduction of this non-linearity ...
13
votes
2answers
361 views

How should I encode the structure of a neural network into a genome?

For a deterministic problem space, I need to find a neural network with the optimal node and link structure. I want to use a genetic algorithm to simulate many neural networks to find the best network ...
13
votes
2answers
575 views

How can I automate the choice of the topology of a neural network for an arbitrary problem?

Assume that I want to solve an issue with a neural network that either I can't fit to already existing topologies (perceptron, Konohen, etc) or I'm simply not aware of the existence of those or I'm ...
13
votes
4answers
1k views

What activation function does the human brain use?

Does the human brain use a specific activation function? I've tried doing some research, and as it's a treshold for whether the signal is sent through a neuron or not, it sounds a lot like ReLU. ...
13
votes
2answers
218 views

Optimal number of layers in a neural network?

How to decide the optimum number of layers to be created while implementing a Neural Network (Feedforward, back propagation or RNN)?
13
votes
2answers
1k views

Input/output encoding for a neural network to learn a grid-based game

I am writing a simple toy game with the intent of training a deep neural network on top of it. The games rules are roughly the following: The game has a board made up of hexagonal cells. Both players ...
12
votes
5answers
18k views

Why does C++ seem less widely used in AI?

I just want to know why do Machine Learning engineers and AI programmers use languages like python to perform AI task and not C++ even though C++ is technically a more powerful language than python.
12
votes
4answers
13k views

1 hidden layer with 1000 neurons vs. 10 hidden layers with 100 neurons

These types of questions may be problem-dependent, but I have tried to find research that addresses the question whether the number of hidden layers and their size (number of neurons in each layer) ...
12
votes
2answers
1k views

How do generative adversarial networks work?

I am reading about generative adversarial networks (GANs) and I have some doubts regarding it. So far, I understand that in a GAN there are two different types of neural networks: one is generative ($...
12
votes
4answers
1k views

What makes neural networks so good at predictions?

I am new to neural-network and I am trying to understand mathematically what makes neural networks so good at classification problems. By taking the example of a small neural network (for example, ...
12
votes
2answers
586 views

Should deep residual networks be viewed as an ensemble of networks?

The question is about the architecture of Deep Residual Networks (ResNets). The model that won the 1-st places at "Large Scale Visual Recognition Challenge 2015" (ILSVRC2015) in all five main tracks: ...
11
votes
1answer
180 views

What are all the different kinds of neural networks used for?

I found the following neural network cheat sheet (Cheat Sheets for AI, Neural Networks, Machine Learning, Deep Learning & Big Data). What are all these different kinds of neural networks used ...
11
votes
1answer
257 views

What are the advantages of complex-valued neural networks?

During my research, I've stumbled upon "complex-valued neural networks", which are neural networks that work with complex-valued inputs (probably weights too). What are the advantages (or simply the ...
11
votes
2answers
3k views

How to choose an activation function?

I choose the activation function for the output layer depending on the output that I need and the properties of the activation function that I know. For example, I choose the sigmoid function when I'm ...
11
votes
1answer
562 views

Are Cellular Neural Networks one type of Neural Networks?

I am researching Cellular Neural Networks and have already read Chua's two articles (1988). In cellular neural networks, a cell is only in relation with its neighbors. So its is easy to use it for ...
10
votes
3answers
383 views

Are neural networks and its variants the only way to reach true artificial intelligence?

According to my knowledge most of the current artificial intelligence study uses of some kind of neural network or its variants. A good example would be DeepMind's alphago which I believe is a deep ...
10
votes
3answers
467 views

How does noise affect generalization?

Does increasing the noise in data help to improve the learning ability of a network? Does it make any difference or does it depend on the problem being solved? How is it affect the generalization ...
10
votes
2answers
2k views

Which layer consumes more time in CNN training ? Convolution layers vs FC layers

In Convolutional Neural Network, which layer consumes maximum time in training? Convolution layers or Fully Connected layers? We can take AlexNet architecture to understand this. I want to see time ...
10
votes
2answers
131 views

What does it mean for a neuron in a neural network to be activated?

I just stumbled upon the concept of neuron coverage, which is the ratio of activated neurons and total neurons in a neural network. But what does it mean for a neuron to be "activated"? I know what ...
10
votes
1answer
1k views

Are there neural networks with very few nodes that decently solve non-trivial problems?

I'm interested in knowing whether there exist any neural network, that solves (with >=80% accuracy) any nontrivial problem, that uses very few nodes (where 20 nodes is not a hard limit). I want to ...
10
votes
1answer
775 views

Why is the merged neural network of AlphaGo Zero more efficient than two separate neural networks?

AlphaGo Zero contains several improvements compared to its predecessors. Architectural details of Alpha Go Zero can be seen in this cheat sheet. One of those improvements is using a single neural ...
10
votes
2answers
639 views

Will computers be able to understand user emotions? How far are we?

My research is in the field of the Affective Computing, particularly I'm studying the part of emotion recognition which is, indeed recognising the emotions that are being felt by the user/subject. ...
10
votes
4answers
318 views

Can a neural network work out the concept of distance?

Imagine a game where it is a black screen apart from a red pixel and a blue pixel. Given this game to a human, they will first see that pressing the arrow keys will move the red pixel. The next ...
10
votes
1answer
345 views

AI that can generate programs

I have been looking into Viv an artificial intelligent agent in development. Based on what I understand, this AI can generate new code and execute it based on a query from the user. What I am curious ...