Questions tagged [deep-learning]

For questions related to deep learning, which refers to a subset of machine learning methods based on artificial neural networks (ANNs) with multiple hidden layers. The adjective deep thus refers to the number of layers of the ANNs. The expression deep learning was apparently introduced (although not in the context of machine learning or ANNs) in 1986 by Rina Dechter in the paper "Learning while searching in constraint-satisfaction-problems".

Filter by
Sorted by
Tagged with
90
votes
7answers
16k views

Do scientists know what is happening inside artificial neural networks?

Do scientists or research experts know from the kitchen what is happening inside complex "deep" neural network with at least millions of connections firing at an instant? Do they understand ...
77
votes
3answers
55k views

How can neural networks deal with varying input sizes?

As far as I can tell, neural networks have a fixed number of neurons in the input layer. If neural networks are used in a context like NLP, sentences or blocks of text of varying sizes are fed to a ...
57
votes
11answers
10k views

What are some well-known problems where neural networks don't do very well?

Background: It's well-known that neural networks offer great performance across a large number of tasks, and this is largely a consequence of their universal approximation capabilities. However, in ...
45
votes
19answers
15k views

Can digital computers understand infinity?

As a human being, we can think infinity. In principle, if we have enough resources (time etc.), we can count infinitely many things (including abstract, like numbers, or real). For example, at least, ...
34
votes
3answers
26k views

Why is Lisp such a good language for AI?

I've heard before from computer scientists and from researchers in the area of AI that that Lisp is a good language for research and development in artificial intelligence. Does this still apply, ...
34
votes
5answers
19k views

What is the purpose of an activation function in neural networks?

It is said that activation functions in neural networks help introduce non-linearity. What does this mean? What does non-linearity mean in this context? How does the introduction of this non-...
27
votes
5answers
12k views

Is it possible to train a neural network as new classes are given?

I would like to train a neural network (NN) where the output classes are not (all) defined from the start. More and more classes will be introduced later based on incoming data. This means that, every ...
26
votes
4answers
31k views

Can a neural network be used to predict the next pseudo random number?

Is it possible to feed a neural network the output from a random number generator and expect it learn the hashing (or generator) function, so that it can predict what will be the next generated pseudo-...
23
votes
3answers
39k views

How do I handle large images when training a CNN?

Suppose that I have 10K images of sizes $2400 \times 2400$ to train a CNN. How do I handle such large image sizes without downsampling? Here are a few more specific questions. Are there any ...
22
votes
3answers
28k views

Can BERT be used for sentence generating tasks?

I am a new learner in NLP. I am interested in the sentence generating task. As far as I am concerned, one state-of-the-art method is the CharRNN, which uses RNN to generate a sequence of words. ...
22
votes
3answers
10k views

How to choose an activation function for the hidden layers?

I choose the activation function for the output layer depending on the output that I need and the properties of the activation function that I know. For example, I choose the sigmoid function when I'm ...
21
votes
2answers
22k views

What are "bottlenecks" in neural networks?

What are "bottlenecks" in the context of neural networks? This term is mentioned, for example, in this TensorFlow article, which also uses the term "bottleneck values". How does ...
21
votes
3answers
4k views

Why do most deep learning papers not include an implementation?

I'm a novice researcher, and as I started to read papers in the area of deep learning I noticed that the implementation is normally not added and is needed to be searched elsewhere, and my question is ...
21
votes
3answers
25k views

How can we process the data from both the true distribution and the generator?

I'm struggling to understand the GAN loss function as provided in Understanding Generative Adversarial Networks (a blog post written by Daniel Seita). In the standard cross-entropy loss, we have an ...
20
votes
5answers
3k views

What is the difference between machine learning and deep learning?

Can someone explain to me the difference between machine learning and deep learning? Is it possible to learn deep learning without knowing machine learning?
17
votes
3answers
4k views

What is geometric deep learning?

What is geometric deep learning (GDL)? Here are a few sub-questions How is it different from deep learning? Why do we need GDL? What are some applications of GDL?
17
votes
4answers
10k views

What is non-Euclidean data?

What is non-Euclidean data? Here are some sub-questions Where does this type of data arise? I have come across this term in the context of geometric deep learning and graph neural networks. ...
17
votes
4answers
4k views

Issues with and alternatives to Deep Learning approaches?

Over the last 50 years, the rise/fall/rise in popularity of neural nets has acted as something of a 'barometer' for AI research. It's clear from the questions on this site that people are interested ...
15
votes
2answers
1k views

When is deep learning overkill?

For example, for classifying emails as spam, is it worthwhile - from a time/accuracy perspective - to apply deep learning (if possible) instead of another machine learning algorithm? Will deep ...
15
votes
1answer
7k views

How can policy gradients be applied in the case of multiple continuous actions?

Trusted Region Policy Optimization (TRPO) and Proximal Policy Optimization (PPO) are two cutting edge policy gradients algorithms. When using a single continuous action, normally, you would use some ...
15
votes
2answers
5k views

Why would you implement the position-wise feed-forward network of the transformer with convolution layers?

The Transformer model introduced in "Attention is all you need" by Vaswani et al. incorporates a so-called position-wise feed-forward network (FFN): In addition to attention sub-layers, each of the ...
15
votes
2answers
9k views

How to implement an "unknown" class in multi-class classification with neural networks?

For example, I need to detect classes for MNIST data. But I want to have not 10 classes for digits, but also I want to have 11th class "not a digit", so that any letter, any other type of ...
14
votes
2answers
729 views

Should deep residual networks be viewed as an ensemble of networks?

The question is about the architecture of Deep Residual Networks (ResNets). The model that won the 1-st places at "Large Scale Visual Recognition Challenge 2015" (ILSVRC2015) in all five main tracks: ...
14
votes
2answers
716 views

Is there any scientific/mathematical argument that prevents deep learning from ever producing strong AI?

I read Judea Pearl's The Book of Why, in which he mentions that deep learning is just a glorified curve fitting technology, and will not be able to produce human-like intelligence. From his book there ...
14
votes
3answers
1k views

Has anyone thought about making a neural network ask questions, instead of only answering them?

Most of the people is trying to answer question with a neural network. However, has anyone came up with some thoughts about how to make neural network ask questions, instead of answer questions? For ...
14
votes
1answer
221 views

Will parameter sweeping on one split of data followed by cross validation discover the right hyperparameters?

Let's call our dataset splits train/test/evaluate. We're in a situation where we require months of data. So we prefer to use the evaluation dataset as infrequently as possible to avoid polluting our ...
13
votes
2answers
6k views

Which layer in a CNN consumes more training time: convolution layers or fully connected layers?

In a convolutional neural network, which layer consumes more training time: convolution layers or fully connected layers? We can take AlexNet architecture to understand this. I want to see the time ...
13
votes
2answers
5k views

Why did machine learning only become viable after Nvidia's chips were available?

I listened to a talk by a panel consisting of two influential Chinese scientists: Wang Gang and Yu Kai and others. When being asked about the biggest bottleneck of the development of artificial ...
13
votes
2answers
2k views

How are generative adversarial networks trained?

I am reading about generative adversarial networks (GANs) and I have some doubts regarding it. So far, I understand that in a GAN there are two different types of neural networks: one is generative ($...
13
votes
2answers
2k views

Input/output encoding for a neural network to learn a grid-based game

I am writing a simple toy game with the intent of training a deep neural network on top of it. The games rules are roughly the following: The game has a board made up of hexagonal cells. Both players ...
13
votes
3answers
470 views

Can some one help me understand this paragraph from Nvidia's progressive GAN paper?

Furthermore, we observe that mode collapses traditionally plaguing GANs tend to happen very quickly, over the course of a dozen mini-batches. Commonly they start when the discriminator overshoots, ...
12
votes
6answers
3k views

Is there actually a lack of fundamental theory on deep learning?

I heard several times that one of the fundamental/open problems of deep learning is the lack of "general theory" on it, because, actually, we don't know why deep learning works so well. Even ...
12
votes
1answer
1k views

What are all the different kinds of neural networks used for? [closed]

I found the following neural network cheat sheet (Cheat Sheets for AI, Neural Networks, Machine Learning, Deep Learning & Big Data). What are all these different kinds of neural networks used ...
12
votes
3answers
20k views

Is it possible to train a neural network to estimate a vehicle's length?

I have a large dataset (over 100k samples) of vehicles with the ground truth of their lengths. Is it possible to train a deep network to measure/estimate vehicle length? I haven't seen any papers ...
12
votes
1answer
354 views

What are the state-of-the-art results on the generalization ability of deep learning methods?

I've read a few classic papers on different architectures of deep CNNs used to solve varied image-related problems. I'm aware there's some paradox in how deep networks generalize well despite ...
12
votes
1answer
2k views

How would DeepMind's new differentiable neural computer scale?

DeepMind just published a paper about a differentiable neural computer, which basically combines a neural network with a memory. The idea is to teach the neural network to create and recall useful ...
12
votes
1answer
436 views

How can Viv generate new code based on some user's query?

I have been looking into Viv, an artificial intelligent agent in development. Here is a demonstration of Viv (by Dag Kittlaus). Based on what I understand, this AI can generate new code and execute it ...
11
votes
5answers
3k views

Why are deep neural networks and deep learning insufficient to achieve general intelligence?

Everything related to Deep Learning (DL) and deep(er) networks seems "successful", at least progressing very fast, and cultivating the belief that AGI is at reach. This is popular imagination. DL is a ...
11
votes
3answers
290 views

What is a deep neural network? [duplicate]

What is the definition of a deep neural network? Why are they so popular or important?
11
votes
2answers
945 views

How much of Deep Mind's work is actually reproducible?

DeepMind has published a lot of works on deep learning in the last years, most of them are state-of-the-art on their respective tasks. But how much of this work has actually been reproduced by the AI ...
11
votes
3answers
3k views

Can the decoder in a transformer model be parallelized like the encoder?

Can the decoder in a transformer model be parallelized like the encoder? As far as I understand, the encoder has all the tokens in the sequence to compute the self-attention scores. But for a decoder,...
11
votes
2answers
1k views

Do deep learning algorithms represent ensemble-based methods?

According to the Wikipedia article on deep learning: Deep learning is a branch of machine learning based on a set of algorithms that attempt to model high-level abstractions in data by using a deep ...
11
votes
1answer
519 views

Can layers of deep neural networks be seen as Hopfield networks?

Hopfield networks are able to store a vector and retrieve it starting from a noisy version of it. They do so setting weights in order to minimize the energy function when all neurons are set equal to ...
10
votes
3answers
3k views

Why is the derivative of the activation functions in neural networks important?

I'm new to NN. I am trying to understand some of its foundations. One question that I have is: why the derivative of an activation function is important (not the function itself), and why it's the ...
10
votes
2answers
920 views

Is neural networks training done one-by-one? [duplicate]

I'm trying to learn neural networks by watching this series of videos and implementing a simple neural network in Python. Here's one of the things I'm wondering about: I'm training the neural network ...
10
votes
2answers
5k views

Is Lisp still worth learning today in the particular context of Machine learning? [closed]

Lisp was originally created as a practical mathematical notation for computer programs, influenced by the notation of Alonzo Church's lambda calculus. It quickly became the favored programming ...
10
votes
1answer
4k views

Loss jumps abruptly when I decay the learning rate with Adam optimizer in PyTorch

I'm training an auto-encoder network with Adam optimizer (with amsgrad=True) and ...
10
votes
3answers
894 views

What do you call a machine learning system that keeps on learning?

As I understand it from this video lecture, there are three types of deep learning: Supervised Unsupervised Reinforcement All these can serve to train a neural network either only prior to its ...
10
votes
1answer
1k views

What are the best known gradient-free training methods for deep learning?

As I know, the current state of the art methods for training deep learning networks are variants of gradient descent or stochastic gradient descent. What are the best known gradient-free training ...
10
votes
2answers
1k views

What benefits can be got by applying Graph Convolutional Neural Network instead of ordinary CNN?

What benefits can we got by applying Graph Convolutional Neural Network instead of ordinary CNN? I mean if we can solve a problem by CNN, what is the reason should we convert to Graph Convolutional ...

1
2 3 4 5
33