Questions tagged [neural-networks]

For questions about a artificial networks, such as MLPs, CNNs, RNNs, LSTM, and GRU networks, their variants or any other AI system components that qualify as a neural networks in that they are, in part, inspired by biological neural networks.

Filter by
Sorted by
Tagged with
52
votes
4answers
14k views

Are neural networks prone to catastrophic forgetting?

Imagine you show a neural network a picture of a lion 100 times and label with "dangerous", so it learns that lions are dangerous. Now imagine that previously you have shown it millions of images of ...
3
votes
2answers
376 views

Do convolutional neural networks perform convolution or cross-correlation?

Typically, people say that convolutional neural networks (CNN) perform the convolution operation, hence their name. However, some people have also said that a CNN actually performs the cross-...
88
votes
7answers
16k views

Do scientists know what is happening inside artificial neural networks?

Do scientists or research experts know from the kitchen what is happening inside complex "deep" neural network with at least millions of connections firing at an instant? Do they understand the ...
16
votes
3answers
5k views

Where can I find the proof of the universal approximation theorem?

The Wikipedia article for the universal approximation theorem cites a version of the universal approximation theorem for Lebesgue-measurable functions from this conference paper. However, the paper ...
5
votes
2answers
479 views

How to estimate the capacity of a neural network?

Is it possible to estimate the capacity of a neural network model? If so, what are the techniques involved?
0
votes
0answers
127 views

What could be a good way to visualise the feature extraction process with MobileNet?

I am trying to create a visualisation for how transfer learning (feature extraction in particular) works with MobileNet. With the ml5.js library, you can extract a ...
20
votes
3answers
9k views

How to choose an activation function for the hidden layers?

I choose the activation function for the output layer depending on the output that I need and the properties of the activation function that I know. For example, I choose the sigmoid function when I'm ...
29
votes
5answers
28k views

What is the time complexity for training a neural network using back-propagation?

Suppose that a NN contains $n$ hidden layers, $m$ training examples, $x$ features, and $n_i$ nodes in each layer. What is the time complexity to train this NN using back-propagation? I have a basic ...
9
votes
1answer
2k views

What is the time complexity of the forward pass algorithm of a feedforward neural network?

How do I determine the time complexity of the forward pass algorithm of a feedforward neural network? How many multiplications are done to generate the output?
33
votes
3answers
24k views

Why is Lisp such a good language for AI?

I've heard before from computer scientists and from researchers in the area of AI that that Lisp is a good language for research and development in artificial intelligence. Does this still apply, with ...
9
votes
3answers
3k views

Why exactly do neural networks require i.i.d. data?

In reinforcement learning, successive states (actions and rewards) can be correlated. An experience replay buffer was used, in the DQN architecture, to avoid training the neural network (NN), which ...
9
votes
3answers
387 views

Are there any rules of thumb for having some idea of what capacity a neural network needs to have for a given problem?

To give an example. Let's just consider the MNIST dataset of handwritten digits. Here are some things which might have an impact on the optimum model capacity: There are 10 output classes The inputs ...
4
votes
1answer
2k views

Is back-propagation applied for each data point or for a batch of data points?

I am new to deep learning and trying to understand the concept of back-propagation. I have a doubt about when the back-propagation is applied. Assume that I have a training data set of 1000 images ...
30
votes
5answers
17k views

What is the purpose of an activation function in neural networks?

It is said that activation functions in neural networks help introduce non-linearity. What does this mean? What does non-linearity mean in this context? How does the introduction of this non-...
32
votes
4answers
1k views

How to find the optimal number of neurons per layer?

When you're writing your algorithm, how do you know how many neurons you need per single layer? Are there any methods for finding the optimal number of them, or is it a rule of thumb?
27
votes
4answers
4k views

Can neural networks be used to prove conjectures?

Imagine I have a list (in a computer-readable form) of all problems (or statements) and proofs that math relies on. Could I train a neural network in such a way that, for example, I enter a problem ...
2
votes
1answer
251 views

Why do we need convolutional neural networks instead of feed-forward neural networks?

Why do we need convolutional neural networks instead of feed-forward neural networks? What is the significance of a CNN? Even a feed-forward neural network will able to solve the image classification ...
33
votes
5answers
63k views

What is the difference between a convolutional neural network and a regular neural network?

I've seen these terms thrown around this site a lot, specifically in the tags convolutional-neural-networks and neural-networks. I know that a neural network is a system based loosely on the human ...
30
votes
8answers
5k views

Is artificial intelligence vulnerable to hacking?

The paper The Limitations of Deep Learning in Adversarial Settings explores how neural networks might be corrupted by an attacker who can manipulate the data set that the neural network trains with. ...
19
votes
3answers
2k views

How are Artificial Neural Networks and the Biological Neural Networks similar and different?

I've heard multiple times that "Neural Networks are the best approximation we have to model the human brain", and I think it is commonly known that Neural Networks are modelled after our brain. I ...
49
votes
4answers
79k views

How to select number of hidden layers and number of memory cells in an LSTM?

I am trying to find some existing research on how to select the number of hidden layers and the size of these of an LSTM-based RNN. Is there an article where this problem is being investigated, i.e., ...
23
votes
5answers
28k views

Can a neural network be used to predict the next pseudo random number?

Is it possible to feed a neural network the output from a random number generator and expect it learn the hashing (or generator) function, so that it can predict what will be the next generated pseudo-...
12
votes
2answers
2k views

What is a recurrent neural network?

Surprisingly, this wasn't asked before - at least I didn't find anything besides some vaguely related questions. So, what is a recurrent neural network, and what are their advantages over regular (or ...
5
votes
2answers
3k views

Are neural networks statistical models?

By reading the abstract of Neural Networks and Statistical Models paper it would seem that ANNs are statistical models. In contrast Machine Learning is not just glorified Statistics. I am looking ...
7
votes
3answers
848 views

What makes the animal brain so special?

Whenever I read any book about neural networks or machine learning, their introductory chapter says that we haven't been able to replicate the brain's power due to its massive parallelism. Now, in ...
4
votes
2answers
4k views

How to calculate the number of parameters of a convolutional layer?

I was recently asked at an interview to calculate the number of parameters for a convolutional layer. I am deeply ashamed to admit I didn't know how to do that, even though I've been working and using ...
3
votes
1answer
290 views

Is it possible to build an AGI with neural networks on neuromorphic chips?

I read a lot about the structure of the human brain and artificial neural networks. Is it possible to build an AGI (or human-level AI) with artificial neural networks on neuromorphic chips, which ...
0
votes
1answer
123 views

What is a convolutional neural network?

Given that this question has not yet been asked on this site, although similar questions have already been asked in the past (e.g. here or here), what is essentially a convolutional neural network (...
70
votes
3answers
50k views

How can neural networks deal with varying input sizes?

As far as I can tell, neural networks have a fixed number of neurons in the input layer. If neural networks are used in a context like NLP, sentences or blocks of text of varying sizes are fed to a ...
39
votes
6answers
21k views

How do capsule neural networks work?

Geoffrey Hinton has been researching something he calls "capsules theory" in neural networks. What is it? How do capsule neural networks work?
54
votes
11answers
10k views

What are some well-known problems where neural networks don't do very well?

Background: It's well-known that neural networks offer great performance across a large number of tasks, and this is largely a consequence of their universal approximation capabilities. However, in ...
29
votes
5answers
21k views

Is it possible to train the neural network to solve math equations?

I'm aware that neural networks are probably not designed to do that, however asking hypothetically, is it possible to train the deep neural network (or similar) to solve math equations? So given the ...
26
votes
4answers
12k views

Is it possible to train a neural network as new classes are given?

I would like to train a neural network (NN) where the output classes are not (all) defined from the start. More and more classes will be introduced later based on incoming data. This means that, every ...
29
votes
2answers
1k views

How is a deep neural network different from other neural networks?

How is a neural network having the "deep" adjective actually distinguished from other similar networks?
32
votes
4answers
15k views

Could a neural network detect primes?

I am not looking for an efficient way to find primes (which of course is a solved problem). This is more of a "what if" question. So, in theory, could you train a neural network to predict ...
11
votes
5answers
7k views

What is the fundamental difference between CNN and RNN?

What is the fundamental difference between convolutional neural networks and recurrent neural networks? Where are they applied?
16
votes
2answers
294 views

How do I decide the optimal number of layers for a neural network?

How do I decide the optimal number of layers for a neural network (feedforward or recurrent)?
12
votes
3answers
29k views

How do I choose the optimal batch size?

Batch size is a term used in machine learning and refers to the number of training examples utilised in one iteration. The batch size can be one of three options: batch mode: where the batch size is ...
7
votes
1answer
623 views

How should the neural network deal with unexpected inputs?

I recently wrote an application using a deep learning model designed to classify inputs. There are plenty of examples of this using images of irises, cats, and other objects. If I trained a data ...
11
votes
2answers
203 views

Why does Batch Normalization work?

Adding BatchNorm layers improves training time and makes the whole deep model more stable. That's an experimental fact that is widely used in machine learning practice. My question is - why does it ...
7
votes
2answers
4k views

When should I use 3D convolution?

I am new to convolutional neural networks, and I am learning 3D convolution. What I could understand is that 2D convolution gives us relationships between low-level features in the X-Y dimension, ...
5
votes
3answers
260 views

How is neural architecture search performed?

I have come across something that IBM offers called neural Architecture search. You feed it a data set and it outputs an initial neural Architecture that you can train. How is neural architecture ...
10
votes
1answer
2k views

How exactly can ReLUs approximate non-linear and curved functions?

Currently, the most commonly used activation functions are ReLUs. So I answered this question What is the purpose of an activation function in neural networks? and, while writing the answer, it struck ...
4
votes
1answer
1k views

Do all neurons in a layer have the same activation function?

I'm new to machine learning (so excuse my nomenclature), and not being a python developer, I decided to jump in at the deep (no pun intended) end writing my own framework in C++. In my current design ...
4
votes
2answers
242 views

Are Neural Net architectures accidental discoveries?

Recently, I have been learning about new neural networks, which are used for specialised purposes, like speech recognition, image recognition, etc. The more I discover the more I get amazed by the ...
13
votes
3answers
2k views

Why are the initial weights of neural networks randomly initialised?

This might sound silly to someone who has plenty of experience with neural networks but it bothers me... Random initial weights might give you better results that would be somewhat closer to what a ...
7
votes
2answers
2k views

How to write a C decompiler using AI?

I would like to learn more about whether it is possible and how to write a program that decompiles executable binary (an object file) to the C source. I'm not asking exactly 'how', but rather how this ...
6
votes
1answer
205 views

What is the formula used to calculate the loss in the FaceNet model?

The FaceNet model returns the loss of the predictions and ground-truth classes. How is this loss calculated?
5
votes
1answer
273 views

Which functions can be activation functions?

What are the required characteristics of an activation function (in a neural network)? Which functions can be activation functions? For example, which of the functions below can be used as an ...
4
votes
1answer
77 views

Is there anything that ensures that convolutional filters don't end up the same?

I trained a simple model to recognize handwritten numbers from the mnist dataset. Here it is: ...