Questions tagged [deep-learning]

For questions related to deep learning, which refers to a subset of machine learning methods based on artificial neural networks (ANNs) with multiple hidden layers. The adjective deep thus refers to the number of layers of the ANNs. The expression deep learning was apparently introduced (although not in the context of machine learning or ANNs) in 1986 by Rina Dechter in the paper "Learning while searching in constraint-satisfaction-problems".

Filter by
Sorted by
Tagged with
3 votes
1 answer
829 views

What is regression layer in a spatial transformer?

I came across this line while reading the original paper on Spatial Transformers by Deepmind in the last paragraph of Sec 3.1: The localisation network function floc() can take any form, such as a ...
  • 73
4 votes
1 answer
224 views

How to deal with padded inputs in a fully connected feed forward network?

I have a fully connected network that takes in a variable-length input padded with 0. However, the network doesn't seem to be learning and I am guessing that the high number of zeros in the input ...
  • 189
3 votes
1 answer
933 views

How to evaluate the goodness of images generated by GANs?

As we all know, there has been tons of GAN variants featuring different aspects of the image generation task such as stability, resolution or the ability to manipulate images. However, it is still ...
3 votes
2 answers
737 views

ML model that is most suited to analyse Google Analytics data

Google Analytics allows me to collect data about every web-session. For simplicity, let's assume for each user, we collect the number of pages and time spent on site for each session: ...
  • 31
5 votes
1 answer
678 views

In YOLO, when is $\mathbb{1}_{i j}^{\mathrm{obj}} = 1$, and what are the ground-truth labels for $x_i$ and $y_i$?

I'm trying to implement a custom version of the YOLO neural network. Originally, it was described in the paper You Only Look Once: Unified, Real-Time Object Detection (2016). I have some problems ...
  • 276
37 votes
5 answers
22k views

What is the purpose of an activation function in neural networks?

It is said that activation functions in neural networks help introduce non-linearity. What does this mean? What does non-linearity mean in this context? How does the introduction of this non-...
3 votes
2 answers
170 views

Facial recognition and classifying unknowns with neural networks

As far as I understand, neural networks aren't good at classifying 'unknowns', i.e. objects that do not belong to a learned class. But how do face detection/recognition approaches usually determine ...
  • 227
2 votes
1 answer
740 views

Periodic Pattern in Validation Loss Curve

I am currently trying to solve a regression problem using neural networks. I want to detect movement patterns in images over time (video) and output a continuous value. During the training process I ...
6 votes
2 answers
445 views

Should the actor or actor-target model be used to make predictions after training is complete (DDPG)?

The situation I am referring to the paper T. P. Lillicrap et al, "Continuous control with deep reinforcement learning" where they discuss deep learning in the context of continuous action spaces ("...
  • 161
2 votes
0 answers
364 views

Deep NN architecture for predicting a matrix from two matrices

Recently my friend asked me a question: having two input matrices X and Y (each size NxD) where D >> N, and ground truth matrix Z of size DxD, what deep architecture shall I use to learn a deep model ...
4 votes
1 answer
46 views

How do we perform object classification given images from a camera that captures images at 15 FPS?

I've been working with vanilla feedforward neural networks and have been researching the convolutional neural network literature. If a camera is capturing a video at a rate of 15 frames per second, is ...
  • 295
2 votes
2 answers
119 views

What is easier or more efficient to summarize voice or text? [DP/RN]

If possible consider the relationship between implementation difficulty and accuracy in voice examples or simply chat conversations. And currently, what are the directions on algorithms like Deep ...
2 votes
1 answer
63 views

Does augmenting data changes the distribution of augmented data?

When we augment data for training are we also changing the distribution of data and if its a different distribution why do we use it to train a model for original distribution ?
  • 205
8 votes
2 answers
5k views

Why should the number of neurons in a hidden layer be a power of 2?

I have read somewhere on the web (I lost the reference) that the number of units (or neurons) in a hidden layer should be a power of 2 because it helps the learning algorithm to converge faster. Is ...
  • 205
1 vote
0 answers
52 views

How to calculate Adaptive gradient?

In the FaceNet paper there mentions an gradient algorithm called 'AdaGrad'(Adaptive Gradient) referenced to this paper which has apparently been used to calculate the gradient of the Triplet Loss ...
3 votes
1 answer
198 views

In novelty search, are the novel structures or behaviour of the neural network rewarded?

I have been reading a lot lately about some very promising work coming out of Uber's AI Labs using mutation algorithms enhanced with novelty search to evolve deep neural nets. See the paper Safe ...
  • 145
9 votes
1 answer
4k views

How does weight normalization work?

I was reading the paper Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks about improving the learning of an ANN using weight normalization. They ...
  • 145
4 votes
1 answer
864 views

Is it possible to train a neural network to identify only one type of object?

I am new to neural networks. Is it possible to train a neural network to identify only one type of object? For instance, a table from a large set of images, where the neural network should be able to ...
9 votes
5 answers
2k views

Can prior knowledge be encoded in deep neural networks?

I was reading Gary Marcus's a Critical Appraisal of Deep Learning. One of his criticisms is that neural networks don't incorporate prior knowledge in tackling a problem. My question is: have there ...
  • 1,146
3 votes
2 answers
3k views

Why is no activation function used at the final layer of super-resolution models?

I'm trying to implement some image super-resolution models on medical images. After reading a set of papers, I found that none of the existing models use any activation function for the last layer. ...
  • 321
4 votes
0 answers
319 views

What are some interesting recent papers that synthesize symbolic AI with Deep Learning?

A lot of people seem to be under the impression that combining GOFAI and contemporary AI will make models more general. I'm particularly interested in reasoning through analogy or case-based reasoning....
  • 316
5 votes
2 answers
176 views

Can variations in microphones used in training set and test set impact the accuracy of speech recognition models?

If I train a speech recognition model using data collected from N different microphones, but deploy it on an unseen (test) microphone - does it impact the accuracy of the model? While I understand ...
3 votes
0 answers
225 views

What is the relation between the definition of learnability of Vapnik and Gold and learnability of neural networks?

Gold showed that a language can be learned only if it contains a finite set of sentences. We know that deep neural networks can implement any function. Does this contradict the Gold's result? What ...
0 votes
1 answer
391 views

The connection between number of layer of DNN and computational complexity of it

number of layer of DNN and computational complexity of it are correlated after optimization, but how to estimate it before designing DNN?
0 votes
1 answer
1k views

What is the purpose of the GAN?

The Generative Adversarial Network (GAN) is composed of a generator $G$ and a discriminator $D$. How do these two components interact? What is the intuition behind the GAN, its purpose, and how it is ...
1 vote
0 answers
105 views

Modelling odd-even distinction of an integer with neural networks

Will it be possible to model the problem of odd-even distinction of an integer (not binary string representation) using neural networks?
  • 119
2 votes
0 answers
69 views

Continuous ground truth in supervised (metric) learning?

I am writing my thesis in the field of (deep) metric learning (DML). I am training a network in the fashion of contrastive / triplet Siamese networks to learn similarity and dissimilarity of inputs. ...
  • 171
5 votes
3 answers
12k views

Can I do deep learning with the 1060 or the 1070 ti? [closed]

Before I start, I want to let you know that I am completely new to the field of deep learning! Since I need a new graphics card either way (gaming you know) I am thinking about buying the GTX 1060 ...
8 votes
3 answers
5k views

How can 3 same size CNN layers in different ordering output different receptive field from the input layer?

Below is a quote from CS231n: Prefer a stack of small filter CONV to one large receptive field CONV layer. Suppose that you stack three 3x3 CONV layers on top of each other (with non-linearities in ...
  • 401
6 votes
1 answer
3k views

How are the kernels initialized in a convolutional neural network?

I am currently learning about CNNs. I am confused about how filters (aka kernels) are initialized. Suppose that we have a $3 \times 3$ kernel. How are the values of this filter initialized before ...
  • 401
4 votes
2 answers
1k views

What layers to use in a Neural Network for card game

I am currently writing an engine to play a card game and I would like for an ANN to learn how to play the game. The game is currently playable, and I believe for this game a deep-recurrent-Q-network ...
1 vote
0 answers
64 views

How do stacked denoising autoencoders work

I've been studying a recommender system which uses a collaborative deep learning approach and Bayesian learning. It has the following NN representation : I need to know the working of stacked ...
  • 303
4 votes
1 answer
301 views

Semantic Segmentation how to upsampling

Many of the architectures that do semantic segmentation like SegNet, DilatedNet (Yu and Koltun), DeepLab, etc. do not work on high resolution images. For such benchmarks like Cityscapes, what is a ...
  • 169
2 votes
1 answer
192 views

What is a heavy node in neural networks?

I was watching a documentary on Netflix about AlphaGo, and at one point (~1:10:16 from the end), one of the programmers uses the term "heavy node," which I assume has to do with neural networks. I did ...
1 vote
0 answers
675 views

How to teach an AI to race optimally in a racing game?

I play a racing game called Need For Madness ( some gameplay: https://www.youtube.com/watch?v=NC5uFZ-t0A8 ). NFM is a racing game, where the player can choose different cars and race and crash the ...
0 votes
1 answer
62 views

Histopathological image vs. natural image

What is the difference between a histopathological image and a natural image when training a neural network?
1 vote
0 answers
179 views

Deep Learning Approaches for Color Enhancement Testing

I'm a student, and currently into image processing project and coding using OpenCV. Recently, I watched Sebastian Thrun from Udacity in TedTalks talked about AlphaGo and I'm totally interested in the ...
  • 121
3 votes
1 answer
59 views

Creating a classifier for simpler classifiers trained on few training samples

Suppose I have a classification problem with a stream of training-samples constantly arriving over time. I cannot keep all training-samples in memory, but I still want to train a classifier that will ...
4 votes
1 answer
705 views

How would you encode your input vector/matrix from a sequence of moves in game like tasks to train an AI? e.g. Chess AI?

I've seen data sets for classification / regressions tasks in domains such as credit default detection, object identification in an image, stock price prediction etc. All of these data sets could ...
  • 63
6 votes
1 answer
249 views

Is one big network faster than several small ones?

The basis of my question is that a CNN that does great on MNIST is far smaller than a CNN that does great on ImageNet. Clearly, as the number of potential target classes increases, along with image ...
  • 474
18 votes
2 answers
11k views

How to implement an "unknown" class in multi-class classification with neural networks?

For example, I need to detect classes for MNIST data. But I want to have not 10 classes for digits, but also I want to have 11th class "not a digit", so that any letter, any other type of ...
26 votes
2 answers
28k views

What are "bottlenecks" in neural networks?

What are "bottlenecks" in the context of neural networks? This term is mentioned, for example, in this TensorFlow article, which also uses the term "bottleneck values". How does ...
1 vote
0 answers
344 views

Natural language processing with a continuous dependent variable

I have a large number of observations. Each observation contains: dependent variable: a scores ranging from 0 - 100 independent variable: a large article I want to know which words or phrases ...
4 votes
1 answer
706 views

Is there any value given to each chess piece in AlphaZero? [closed]

Recently, DeepMind's AlphaZero chess algorithm did better than the prior best chess software Stockfish. I read the paper Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning ...
  • 151
1 vote
1 answer
135 views

How can I design the input layer of a feed-forward neural network to be trained with a medical dataset with three features?

I am building a feed-forward neural network with two hidden layers, which I will train with a medical dataset, which consists of both data, such as age and sex, and images of x-ray scans ($1024 \times ...
7 votes
2 answers
2k views

How do I create an AI for a two-players board game?

Goal I want to create an artificial intelligence to compete against other players in a board game. Game explanation I have a board game similar to 'snakes and ladders'. You have to get to a final ...
  • 119
7 votes
1 answer
1k views

Why is the target $r + \gamma \max_{a'} Q(s', a'; \theta_i^-)$ in the loss function of the DQN architecture?

In the paper Human-level control through deep reinforcement learning, the DQN architecture is presented, where the loss function is as follows $$ L_i(\theta_i) = \mathbb{E}_{(s, a, r, s') \sim U(D)} \...
  • 653
1 vote
3 answers
171 views

Can we make an algorithm which can solve any high school (science) problem using ML and DL?

I heard that your ML model's quality depends directly on the quality and the quantity of data you use. So I was thinking that can question answers be used as data to train an algorithm which can ...
4 votes
2 answers
936 views

How good is AI in math?

Currently, AI is advancing fast in deep learning: Entire human chess knowledge learned and surpassed by DeepMind's AlphaZero in four hours. As a layman, I'm taking this as a quite powerful searching ...
  • 167
2 votes
1 answer
2k views

What fast loss convergence indicates on a CNN?

I'm training two CNNs (AlexNet e GoogLeNet) in two differents DL libraries (Caffe e Tensorflow). The networks was implemented by dev teams of each libraries (here and here) I reduced the original ...