Questions tagged [deep-learning]

For questions related to deep learning, which refers to a subset of machine learning methods based on artificial neural networks (ANNs) with multiple hidden layers. The adjective deep thus refers to the number of layers of the ANNs. The expression deep learning was apparently introduced (although not in the context of machine learning or ANNs) in 1986 by Rina Dechter in the paper "Learning while searching in constraint-satisfaction-problems".

Filter by
Sorted by
Tagged with
81
votes
8answers
14k views

Do scientists know what is happening inside artificial neural networks?

Do scientists or research experts know from the kitchen what is happening inside complex "deep" neural network with at least millions of connections firing at an instant? Do they understand the ...
32
votes
3answers
21k views

Why is Lisp such a good language for AI?

I've heard before from computer scientists and from researchers in the area of AI that that Lisp is a good language for research and development in artificial intelligence. Does this still apply, with ...
2
votes
1answer
526 views

Is back propagation applied for each data point or for a batch of data points?

I am new to deep learning and trying to understand the concept of back propagation. I have a doubt on when the back propagation is applied. Assume that I have a training data set of 1000 images for ...
22
votes
6answers
11k views

What is the purpose of an activation function in neural networks?

It is said that activation functions in neural networks help introduce non-linearity. What does this mean? What does non-linearity mean in this context? How does the introduction of this non-...
41
votes
8answers
33k views

In a CNN, does each new filter have different weights for each input channel, or are the same weights of each filter used across input channels?

My understanding is that the convolutional layer of a convolutional neural network has four dimensions: input_channels, filter_height, filter_width, number_of_filters. Furthermore, it is my ...
4
votes
2answers
243 views

How to estimate the capacity of a neural network?

Is it possible to estimate the capacity of a neural network model? If so, what are the techniques involved?
4
votes
1answer
228 views

What is a graph neural network?

What is a graph neural network (GNN)? How is a GNN different from a NN? How exactly is a GNN related to graphs? What are the components of a GNN? What are the inputs and outputs of GNNs? How can ...
59
votes
3answers
39k views

How can neural networks deal with varying input sizes?

As far as I can tell, neural networks have a fixed number of neurons in the input layer. If neural networks are used in a context like NLP, sentences or blocks of text of varying sizes are fed to a ...
20
votes
5answers
20k views

Can a neural network be used to predict the next pseudo random number?

Is it possible to feed a neural network the output from a random number generator and expect it learn the hashing (or generator) function, so that it can predict what will be the next generated pseudo-...
5
votes
2answers
2k views

Are neural networks statistical models?

By reading the abstract of Neural Networks and Statistical Models paper it would seem that ANNs are statistical models. In contrast Machine Learning is not just glorified Statistics. I am looking ...
11
votes
1answer
6k views

How can policy gradients be applied in the case of multiple continuous actions?

Trusted Region Policy Optimization (TRPO) and Proximal Policy Optimization (PPO) are two cutting edge policy gradients algorithms. When using a single continuous action, normally, you would use some ...
5
votes
3answers
4k views

CPU preferences and specifications for a multi GPU deep-learning setup [closed]

For a multi (4xTitan Xp) GPU deep learning setup what kind of CPU is preferable? Specifically I am comparing: Intel Xeon E5-2620 with 8x2.1GHz 20MB L3 Cache Intel Xeon E5-1620K with 4x3.5Ghz 10MB L3 ...
6
votes
2answers
398 views

What is experience replay in laymen's terms?

I've been reading Google's DeepMind Atari paper and I'm trying to understand the concept of "experience replay". Experience replay comes up in a lot of other reinforcement learning papers (...
2
votes
2answers
179 views

Concrete Example for Q Learning

I am not sure if I understood the q learning algorithms correctly. Therefore I would give a concrete example and ask if someone can tell me how to update the q value correctly. First I initialized ...
2
votes
2answers
2k views

Does fp32 & fp64 performance of GPU affect deep learning model training? [closed]

I am purchasing Titan RTX GPU. Everything seems fine with that except float32 & float64 performance which seems lower vis-a-vis some of its counter parts. I wanted to understand if single ...
52
votes
11answers
8k views

What are some well-known problems where neural networks don't do very well?

What are some well-known cases, problems or real-world applications where neural networks don't do very well? Specification: I'm looking for specific regression tasks (with accessible data-sets) ...
16
votes
3answers
28k views

How do I handle large images when training a CNN?

Suppose that I have 10K images of sizes $2400 \times 2400$ to train a CNN. How do I handle such large image sizes without downsampling? Here are a few more specific questions. Are there any ...
13
votes
3answers
4k views

How to implement a constrained action space in reinforcement learning?

I'm coding a reinforcement learning model with a PPO agent thanks to the very good Tensorforce library, built on top of Tensorflow. The first version was very simple and I'm now diving into a more ...
15
votes
2answers
1k views

When is deep learning overkill?

For example, for classifying emails as spam, is it worthwhile - from a time/accuracy perspective - to apply deep learning (if possible) instead of another machine learning algorithm? Will deep ...
15
votes
3answers
14k views

What is the concept of Tensorflow Bottlenecks?

What is the concept and how does one calculate Bottleneck values? How do these values help image classification? Please explain in simple words.
11
votes
5answers
2k views

Why are deep neural networks and deep learning insufficient to achieve general intelligence?

Everything related to Deep Learning (DL) and deep(er) networks seems "successful", at least progressing very fast, and cultivating the belief that AGI is at reach. This is popular imagination. DL is a ...
6
votes
3answers
4k views

What is non-Euclidean data?

What is non-Euclidean data? Where does this type of data arises? Apparently, graphs and manifolds are non-Euclidean data. Why exactly is that the case? What is the difference between non-Euclidean and ...
19
votes
3answers
24k views

Understanding GAN loss function

I'm struggling to understand the GAN loss function as provided in Understanding Generative Adversarial Networks (a blog post written by Daniel Seita). In the standard cross-entropy loss, we have an ...
11
votes
1answer
246 views

What are the state-of-the-art results on the generalization ability of deep learning methods?

I've read a few classic papers on different architectures of deep CNNs used to solve varied image-related problems. I'm aware there's some paradox in how deep networks generalize well despite ...
10
votes
3answers
14k views

Measuring Object size using Deep Neural Network

I have a large dataset of vehicles with the ground truth of their lengths (Over 100k samples). Is it possible to train a deep network to measure/estimate vehicle length ? I haven't seen any papers ...
11
votes
2answers
4k views

Which layer in a CNN consumes more training time: convolution layers or fully connected layers?

In a convolutional neural network, which layer consumes more training time: convolution layers or fully connected layers? We can take AlexNet architecture to understand this. I want to see the time ...
8
votes
2answers
830 views

Why use a recurrent neural network over a feedforward neural network for sequence prediction?

If recurrent neural networks (RNNs) are used to capture prior information, couldn't the same thing be achieved by a feedforward neural network (FFNN) or multi-layer perceptron (MLP) where the inputs ...
6
votes
2answers
290 views

Is reinforcement learning using shallow neural networks still deep reinforcement learning?

Often times I see the term deep reinforcement learning to refer to RL algorithms that use neural networks, regardless of whether or not the networks are deep. For example, PPO is often considered a ...
3
votes
2answers
876 views

Each training run for DDQN agent takes 2 days, and still ends up with -13 avg score, but OpenAi baseline DQN needs only an hour to converge to +18?

Status: For a few weeks now, I have been working on a Double DQN agent for the PongDeterministic-v4 environment, which you can find here. A single training run ...
6
votes
3answers
1k views

Board/Card Game AI - Questions concerning state/action space - Deep Reinforcement Learning

Ok, I now know how a machine can learn to play to play Atari games (Breakout): Playing Atari with Reinforcement Learning With the same technique it is even possible to play FPS games (Doom): Playing ...
4
votes
2answers
829 views

How good is AI in math?

Currently, AI is advancing fast in deep learning: Entire human chess knowledge learned and surpassed by DeepMind's AlphaZero in four hours. As a layman, I'm taking this as a quite powerful searching ...
4
votes
1answer
79 views

Is it possible to combine two neural networks trained on different tasks into one that knows both tasks?

I'm relatively new to artificial intelligence and neural networks. Let's say I have two different fully trained neural networks. The first one is trained for mathematical addition and the second one ...
4
votes
3answers
266 views

Can someone direct me to a sites and/or videos that can bring an absolute beginner up to speed with AI?

To start, I'm not a programmer/computer scientist/et al... - I work in Finance and have, through my job, self-thought myself VBA for excel and outlook and would consider myself as being in the upper ...
3
votes
1answer
135 views

In novelty search, are the novel structures or behaviour of the neural network rewarded?

I have been reading a lot lately about some very promising work coming out of Uber's AI Labs using mutation algorithms enhanced with novelty search to evolve deep neural nets. See the paper Safe ...
2
votes
1answer
47 views

Should I remove the units of a neural network or increase dropout?

When adding dropout to a neural network, we are randomly removing a fraction of the connections (setting those weights to zero for that specific weight update iteration). If the dropout probability is ...
10
votes
3answers
223 views

What is a deep neural network?

What is the definition of a deep neural network? Why are they so popular or important?
9
votes
2answers
758 views

What benefits can be got by applying Graph Convolutional Neural Network instead of ordinary CNN?

What benefits can we got by applying Graph Convolutional Neural Network instead of ordinary CNN? I mean if we can solve a problem by CNN, what is the reason should we convert to Graph Convolutional ...
8
votes
3answers
2k views

How can an AI freely make decisions on a network?

Say a Deep Neural Net is created using Keras or Tensorflow. Usually when you want to make a prediction the user would invoke model.predict.... However, how would ...
5
votes
3answers
579 views

Use of machine learning for analyzing companies enlisted in stock market

Can current trends and tools, in the field of machine learning, replicate the complexity of financial market? If yes, then what are the tools available in this domain. Q. I am trying to build a model ...
4
votes
1answer
78 views

Video engagement analysis with deep learning

I am trying to rank video scenes/frames based on how appealing they are for a viewer. Basically, how "interesting" or "attractive" a scene inside a video can be for a viewer. My final goal is to ...
4
votes
1answer
437 views

How would you encode your input vector/matrix from a sequence of moves in game like tasks to train an AI? e.g. Chess AI?

I've seen data sets for classification / regressions tasks in domains such as credit default detection, object identification in an image, stock price prediction etc. All of these data sets could ...
4
votes
1answer
845 views

Dataset containing images of varying dimensions and orientations

I am new to deep learning. I have a dataset of images of varying dimensions of a certain object. A few images of the object are also in varying orientations. The objective is to learn the features ...
4
votes
1answer
68 views

Aesthetics analysis with deep learning

I'm trying to score video scenes in terms of aesthetics and cinematography features. Basically, how "interesting" a scene or video frame can be for a viewer. Simpler, how attractive a scene is. My ...
2
votes
1answer
304 views

AI chatbot design

I am looking to train a chatbot that can help me relieve stress and deal with my negative emotions. I would like for the chatbot to be like the ones that pass the Turing test, remain professional, yet ...
2
votes
1answer
143 views

How does a batch normalization layer work?

I understood that we normalize to input features in order to bring them on the same scale so that weights won't be learned in arbitrary fashion and training would be faster. Then I studied about ...
2
votes
0answers
108 views

Is there a dataset for the detection of bomb explosions?

I would like to train a deep neural network to recognize bomb explosions. I was wondering if there is an open visual dataset for bomb explosion? Alternatively, if you know a good deep architecture or ...
2
votes
1answer
101 views

Why is this ResNet50 misclassifying objects?

I'm new to Deep Learning, and I have some conceptual problems. I followed a simple tutorial here, and trained a model in Keras to do image classification on 10 classes of logos. I prepared 10 classes ...
2
votes
1answer
125 views

Why do we need convolutional neural networks instead of feed-forward neural networks?

Why do we need convolutional neural networks instead of feed-forward neural networks? What is the significance of a CNN? Even a feed-forward neural network will able to solve the image classification ...
1
vote
1answer
94 views

How many layers exists in my neural network?

I have a neural network model defined as below. How many layers exist there? Not sure which ones to count when we are asked about the number. ...
0
votes
1answer
41 views

Use deep learning to rank video scenes

I'm new to machine learning and especially, deep learning. Given a video (and it's subtitle), I need to generate a 10-second summary out of this video. How can I use ML and DL to produce the most ...