Questions tagged [deep-learning]

For questions related to deep learning, which refers to a subset of machine learning methods based on artificial neural networks (ANNs) with multiple hidden layers. The adjective deep thus refers to the number of layers of the ANNs. The expression deep learning was apparently introduced (although not in the context of machine learning or ANNs) in 1986 by Rina Dechter in the paper "Learning while searching in constraint-satisfaction-problems".

Filter by
Sorted by
Tagged with
69
votes
8answers
11k views

Do scientists know what is happening inside artificial neural networks?

Do scientists or research experts know from the kitchen what is happening inside complex "deep" neural network with at least millions of connections firing at an instant? Do they understand the ...
41
votes
3answers
27k views

How can neural networks deal with varying input sizes?

As far as I can tell, neural networks have a fixed number of neurons in the input layer. If neural networks are used in a context like NLP, sentences or blocks of text of varying sizes are fed to a ...
34
votes
1answer
19k views

Which library would you recommend to begin with deep learning?

Which library (TensorFlow or Keras) would you recommend for a first approach to deep learning? I'm a neuroscience student trying for the first time computational approaches, if that matters.
31
votes
21answers
10k views

Can digital computers understand infinity?

As a human being, we can think infinity. In principle, if we have enough resources (time etc.), we can count infinitely many things (including abstract, like numbers, or real). For example, at least, ...
29
votes
3answers
17k views

Why is Lisp such a good language for AI?

I've heard before from computer scientists and from researchers in the area of AI that that Lisp is a good language for research and development in artificial intelligence. Does this still apply, with ...
28
votes
8answers
24k views

In a CNN, does each new filter have different weights for each input channel, or are the same weights of each filter used across input channels?

My understanding is that the convolutional layer of a convolutional neural network has four dimensions: input_channels, filter_height, filter_width, number_of_filters. Furthermore, it is my ...
21
votes
4answers
3k views

Can deep networks be trained to prove theorems?

Assume we have a large number of proofs in first order predicate calculus. Assume we also have the axioms, corollaries, and theorems in that area of mathematics in that form too. Consider the each ...
16
votes
3answers
21k views

Understanding GAN loss function

I'm struggling to understand the GAN loss function as provided in Understanding Generative Adversarial Networks (a blog post written by Daniel Seita). In the standard cross-entropy loss, we have an ...
15
votes
4answers
2k views

Issues with and alternatives to Deep Learning approaches?

Over the last 50 years, the rise/fall/rise in popularity of neural nets has acted as something of a 'barometer' for AI research. It's clear from the questions on this site that people are interested ...
14
votes
5answers
2k views

What is the difference between machine learning and deep learning?

Can someone explain to me the difference between machine learning and deep learning? Is it possible to learn deep learning without knowing machine learning?
14
votes
3answers
21k views

How to handle images of large sizes in CNN?

Suppose there are 10K images of sizes 2400 x 2400 are required to use in CNN.Acc to my view conventional computers the people use will be of use. Now the question is how to handle such large image ...
14
votes
2answers
1k views

What's the main concept behind Capsule Networks? [duplicate]

As you might know, Capsule Networks have been recently introduced by Hinton. There also have been several heads up within his talks. As expected, the paper elaborates on the idea way theoretically! ...
14
votes
3answers
949 views

Has anyone thought about making a neural network ask questions, instead of only answering them?

Most of the people is trying to answer question with a neural network. However, has anyone came up with some thoughts about how to make neural network ask questions, instead of answer questions? For ...
13
votes
6answers
6k views

What is the purpose of an activation function in Neural Networks?

It is said that activation functions in neural networks help introduce non-linearity. What does this mean? What does non-linearity mean in this context? How does introduction of this non-linearity ...
13
votes
2answers
888 views

When is deep learning overkill?

For example, for classifying emails as spam, is it worthwhile - from a time/accuracy perspective - to apply deep learning (if possible) instead of another machine learning algorithm? Will deep ...
13
votes
2answers
1k views

Input/output encoding for a neural network to learn a grid-based game

I am writing a simple toy game with the intent of training a deep neural network on top of it. The games rules are roughly the following: The game has a board made up of hexagonal cells. Both players ...
12
votes
2answers
1k views

How do generative adversarial networks work?

I am reading about generative adversarial networks (GANs) and I have some doubts regarding it. So far, I understand that in a GAN there are two different types of neural networks: one is generative ($...
12
votes
2answers
585 views

Should deep residual networks be viewed as an ensemble of networks?

The question is about the architecture of Deep Residual Networks (ResNets). The model that won the 1-st places at "Large Scale Visual Recognition Challenge 2015" (ILSVRC2015) in all five main tracks: ...
12
votes
1answer
2k views

How would Deepmind's new “differentiable neural computer” scale?

Deepmind just published a paper about a "differentiable neural computer", which basically combines a neural network with a memory. The idea is to teach the neural network to create and recall useful ...
11
votes
1answer
179 views

What are all the different kinds of neural networks used for?

I found the following neural network cheat sheet (Cheat Sheets for AI, Neural Networks, Machine Learning, Deep Learning & Big Data). What are all these different kinds of neural networks used ...
11
votes
3answers
2k views

How to implement a constrained action space in reinforcement learning?

I'm coding a reinforcement learning model with a PPO agent thanks to the very good Tensorforce library, built on top of Tensorflow. The first version was very simple and I'm now diving into a more ...
11
votes
2answers
367 views

Is there any scientific/mathematical argument that prevents deep learning from ever producing strong AI?

I read Judea Pearl's The Book of Why, in which he mentions that deep learning is just a glorified curve fitting technology, and will not be able to produce human-like intelligence. From his book ...
10
votes
5answers
13k views

Using Machine/Deep learning for guessing Pseudo Random generator

Is it possible to feed a neural network, the output from a random number generator and expect it learn the hashing/generator function. So that it can predict what will be the next generated number? ...
10
votes
5answers
2k views

Why are deep neural networks and deep learning insufficient to achieve general intelligence?

Everything related to Deep Learning (DL) and deep(er) networks seems "successful", at least progressing very fast, and cultivating the belief that AGI is at reach. This is popular imagination. DL is a ...
10
votes
2answers
2k views

Which layer consumes more time in CNN training ? Convolution layers vs FC layers

In Convolutional Neural Network, which layer consumes maximum time in training? Convolution layers or Fully Connected layers? We can take AlexNet architecture to understand this. I want to see time ...
10
votes
3answers
10k views

Measuring Object size using Deep Neural Network

I have a large dataset of vehicles with the ground truth of their lengths (Over 100k samples). Is it possible to train a deep network to measure/estimate vehicle length ? I haven't seen any papers ...
10
votes
1answer
200 views

A deep learning algorithm to optimize the outcome

I'm am quite new to deep learning but I think I found just the right real-world situation to start using it. The problem is that I have only used such algorithms to predict outcomes. For my new ...
10
votes
1answer
345 views

AI that can generate programs

I have been looking into Viv an artificial intelligent agent in development. Based on what I understand, this AI can generate new code and execute it based on a query from the user. What I am curious ...
9
votes
3answers
9k views

What is the concept of Tensorflow Bottlenecks?

What is the concept and how does one calculate Bottleneck values? How do these values help image classification? Please explain in simple words.
9
votes
2answers
779 views

How much of Deep Mind's work is actually reproducible?

Deep Mind has published a lot of works on deep learning in the last years, most of them state-of-the-art on their respective tasks. But how much of this work has actually been reproduced by the AI ...
9
votes
4answers
1k views

Is there actually a lack of fundamental theory on deep learning?

I heard several times that one of the fundamental/open problems of deep learning is the lack of "general theory" on it because actually we don't know why deep learning works so well. Even the ...
9
votes
1answer
2k views

Loss jumps abruptly when I decay the learning rate with Adam optimizer in PyTorch

I'm training an auto-encoder network with Adam optimizer (with amsgrad=True) and ...
9
votes
1answer
4k views

Policy gradients for multiple continuous actions

Question is regarding Deep Reinforcement Learning using Policy Gradients. Cutting edge policy gradients algorithms are TRPO (Trusted Region Policy Optimization) and PPO (Proximal Policy Optimization)....
9
votes
2answers
462 views

Was DeepMind's DQN Atari game learning simultaneous?

DeepMind state that their deep Q-network (DQN) was able to continually adapt its behavior while learning to play 49 Atari games. After learning all games with the same neural net, was the agent ...
9
votes
2answers
489 views

Is topological sophistication necessary to the furtherance of AI?

The current machine learning trend is interpreted by some new to the disciplines of AI as meaning that MLPs, CNNs, and RNNs can exhibit human intelligence. It is true that these orthogonal structures ...
9
votes
0answers
60 views

Will parameter sweeping on one split of data followed by cross validation discover the right hyperparameters?

Let's call our dataset splits train/test/evaluate. We're in a situation where we require months of data so we prefer to use the evaluation dataset as infrequently as possible to avoid polluting our ...
9
votes
1answer
197 views

Deep Networks and generalisation of Hopfield Networks

Hopfield Nets are able to store a vector and retrieve it starting from a noisy version of it. They do so setting weights in order to minimise the energy function when all neurons are set equal the ...
9
votes
5answers
310 views

Can an AI be trained to generate the outline of a story?

I know that one of the recent fads right now is to train a neural network to generate screenplays and new episodes of e.g. the Friends or The Simpsons, and that's fine: it's interesting and might be ...
8
votes
3answers
5k views

Can BERT be used for sentence generating tasks?

I am a new learner in NLP. I am interested in the sentence generating task. As far as I am concerned, one state-of-the-art method is the CharRNN, which uses RNN to generate a sequence of words. ...
8
votes
3answers
171 views

What is a deep neural network?

What is the definition of a deep neural network? Why are they so popular or important?
8
votes
2answers
3k views

Is Lisp still worth learning today in the particular context of Machine learning?

Lisp was originally created as a practical mathematical notation for computer programs, influenced by the notation of Alonzo Church's lambda calculus. It quickly became the favored programming ...
8
votes
2answers
125 views

What is the difference between search and learning?

I came across an article, The Bitter Truth, via the Two Minute Papers YouTube Channel. Rich Sutton says... One thing that should be learned from the bitter lesson is the great power of general ...
8
votes
2answers
5k views

What are the latest 'hot' research topics for deep learning and AI?

I did my Master's thesis on Deep Generative Models and I'm currently looking for a new subject. Q: What are the "hottest" research topics that are taking a lot of attention of the deep learning ...
8
votes
3answers
1k views

Do deep learning algorithms represent ensemble-based methods?

Shortly about deep learning (for reference): Deep learning is a branch of machine learning based on a set of algorithms that attempt to model high-level abstractions in data by using a deep ...
8
votes
2answers
478 views

What benefits can be got by applying Graph Convolutional Neural Network instead of ordinary CNN?

What benefits can we got by applying Graph Convolutional Neural Network instead of ordinary CNN? I mean if we can solve a problem by CNN, what is the reason should we convert to Graph Convolutional ...
8
votes
1answer
111 views

5 years later, are maxout networks dead, and why?

Maxout networks were a simple yet brilliant idea of Goodfellow et al. from 2013 to max feature maps to get a universal approximator of convex activations. The design was tailored for use in ...
7
votes
2answers
582 views

What is geometric deep learning?

What is geometric deep learning (GDL)? How is it different from deep learning? Why do we need GDL? What are some applications of GDL?
7
votes
1answer
183 views

What is “early stopping” in machine learning?

What is early stopping in machine learning and, in general, artificial intelligence? What are the advantages of using this method? How does it help exactly? I'd be interested in perspectives and ...
7
votes
1answer
183 views

Paradigm shift in Machine Learning

In some tweets about NeurIPS 2018, this video from NVIDIA appeared. At around 0.37, she says: ... If you think about the current computations in our deep learning systems, they are all based on ...
7
votes
3answers
643 views

What do you call a machine learning system that keeps on learning?

As I understand it from this video lecture, there are three types of deep learning: Supervised Unsupervised Reinforcement All these can serve to train a neural network either only prior to its ...