Questions tagged [deep-learning]

For questions related to deep learning, which refers to a subset of machine learning methods based on artificial neural networks (ANNs) with multiple hidden layers. The adjective deep thus refers to the number of layers of the ANNs. The expression deep learning was apparently introduced (although not in the context of machine learning or ANNs) in 1986 by Rina Dechter in the paper "Learning while searching in constraint-satisfaction-problems".

728 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
15 votes
1 answer
282 views

Will parameter sweeping on one split of data followed by cross validation discover the right hyperparameters?

Let's call our dataset splits train/test/evaluate. We're in a situation where we require months of data. So we prefer to use the evaluation dataset as infrequently as possible to avoid polluting our ...
13 votes
2 answers
556 views

Can some one help me understand this paragraph from Nvidia's progressive GAN paper?

In the paper Progressive growing of gans for improved quality, stability, and variation (ICLR, 2018) by Nvidia researchers, the authors write Furthermore, we observe that mode collapses traditionally ...
  • 401
10 votes
0 answers
313 views

Extending FaceNet’s triplet loss to object recognition

FaceNet uses a novel loss metric (triplet loss) to train a model to output embeddings (128-D from the paper), such that any two faces of the same identity will have a small Euclidean distance, and ...
9 votes
0 answers
237 views

Why is my GAN more unstable with bigger networks?

I am working with generative adversarial networks (GANs) and one of my aims at the moment is to reproduce samples in two dimensions that are distributed according to a circle (see animation). When ...
  • 134
6 votes
1 answer
102 views

Can training a model on a dataset composed by real images and drawings hurt the training process of a real-world application model?

I'm training a multi-label classifier that's supposed to be tested on underwater images. I'm wondering if feeding the model drawings of a certain class plus real images can affect the results badly. ...
  • 91
5 votes
0 answers
63 views

Origins of the name of convolutional neural networks

Convolutional neural networks (CNNs) contain convolutional layers. In modern deep learning libraries such as Tensorflow and PyTorch among others, convolutional layers are implemented by using the ...
  • 520
5 votes
0 answers
68 views

It is possible to use deep learning to give approximate solutions to NP-hard graph theory problems?

It is possible to use deep learning to give approximate solutions to NP-hard graph theory problems? If we take, for example, the travelling salesman problem (or the dominating set problem). Let's say ...
  • 171
5 votes
2 answers
489 views

What's the difference between architectures and backbones?

In the paper "ForestNet: Classifying Drivers of Deforestation in Indonesia using Deep Learning on Satellite Imagery", the authors talk about using: Feature Pyramid Networks (as the ...
5 votes
0 answers
127 views

Why Pixel RNN (Row LSTM) can capture triangular contexts?

I'm reading the paper Pixel Recurrent Neural Network. I have a question about Row LSTM. Why Row LSTM can capture triangular contexts? In this paper, the kernel of the one-dimensional convolution ...
5 votes
0 answers
277 views

How to correctly implement self-play with DQN?

I have an environment where an agent faces an equal opponent, and while I've achieved OK performance implementing DQN and treating the opponent as a part of the environment, I think performance would ...
  • 51
5 votes
0 answers
1k views

What are the differences between Yolo v1 and CenterNet?

I recently read a new paper (late 2019) about a one-shot object detector called CenterNet. Apart from this, I'm using Yolo (V3) one-shot detector, and what surprised me is the close similarity between ...
  • 298
4 votes
0 answers
63 views

Training and inference for highly-context-sensitive information

What is the best way to train / do inference when the context matters highly as to what the inferred result should be? For example in the image below all people are standing upright, but because of ...
  • 91
4 votes
2 answers
129 views

What kind of output should be used for predicting angles in DNNs?

I am building a model which predicts angles as output. What are the different kinds of outputs that can be used to predict angles? For example, output the angle in radians cyclic nature of the ...
  • 189
4 votes
0 answers
470 views

What is the difference between GAT and GaAN?

I was looking at two papers Graph Attention Networks (GAT) by Petar Veličković and GaAN: Gated Attention Networks for Learning on Large and Spatiotemporal Graphs by Jiani Zhang. I'm trying to ...
  • 1,108
4 votes
1 answer
287 views

How did the OpenAI 5 for Dota concatenate units?

I am no expert in the field of AI so I apologize if this is a simple/easy question. I was trying to implement a network similar to OpenAI's for another game and I noticed that I did not fully ...
4 votes
0 answers
170 views

What characteristics make it difficult for a Neural Network to approximate a function?

What are the characteristics which make a function difficult for the Neural Network to approximate? Intuitively, one might think uneven functions might be difficult to approximate, but uneven ...
user avatar
4 votes
0 answers
55 views

What is the motivation for row-wise convolution and folding in Kalchbrenner et al. (2014)?

I was reading the paper by Kalchbrenner et al. titled A Convolutional Neural Network for Modelling Sentences and I am struggling to understand their definition of convolutional layer. First, let's ...
4 votes
0 answers
325 views

What are the ways to calculate the error rate of a deep Convolutional Neural Network, when the network produces different results using the same data?

I am new to the object recognition community. Here I am asking about the broadly accepted ways to calculate the error rate of a deep CNN when the network produces different results using the same data....
4 votes
1 answer
1k views

Which other loss functions for hierarchical multi-label classification could I use?

I am looking to try different loss functions for a hierarchical multi-label classification problem. So far, I have been training different models or submodels like multilayer perceptron (MLP) branch ...
  • 143
4 votes
4 answers
811 views

Use Machine/Deep Learning to Guess a String

I want to be able to input a block of text and then have it guess a string within a predefined range (i.e. a string that starts with three letters and ends with five numbers like "XXX12345", etc). ...
4 votes
0 answers
320 views

What are some interesting recent papers that synthesize symbolic AI with Deep Learning?

A lot of people seem to be under the impression that combining GOFAI and contemporary AI will make models more general. I'm particularly interested in reasoning through analogy or case-based reasoning....
  • 326
3 votes
2 answers
70 views

How to instruct Mask RCNN to identify objects too close to each other?

I have been trying to train a Mask RCNN model to identify individual poker chips in a stack. No matter what property I change, the end results look like the following image. I was guessing the issue ...
  • 31
3 votes
0 answers
126 views

Are there neural networks with (hard) constraints on the weights?

I don't know too much about Deep Learning, so my question might be silly. However, I was wondering whether there are NN architectures with some hard constraints on the weights of some layers. For ...
  • 173
3 votes
0 answers
49 views

Why might the convolution be inappropriate when the task involves incorporating information from very distant locations in the input?

When I am reading about convolutional neural networks, I have encountered the following sentence from the textbook(page 341) that says about the limitation of the usage of the convolution in CNNs. ...
  • 187
3 votes
0 answers
53 views

How are Ground truth provided to each Pyramid map in RetinaNet or YOLOv3 Paper? How is the mapping of Feature Pyramids done to Ground Truth

SO the YOLO V3 and RetinaNet both uses the Feature pyramids which look something like this: (except b and e which have one ...
  • 253
3 votes
0 answers
287 views

How can I use Monte Carlo Dropout in a pre-trained CNN model?

In Monte Carlo Dropout (MCD), I know that I should enable dropout during training and testing, then get multiple predictions for the same input $x$ by performing multiple forward passes with $x$, then,...
  • 31
3 votes
0 answers
287 views

Stack of Planes as the Action Space Representation for AlphaZero (Chess)

I have a question regarding the action space of the policy network used in AlphaZero. From the paper: We represent the policy π(a|s) by a 8 × 8 × 73 stack of planes encoding a probability ...
  • 117
3 votes
0 answers
32 views

If random rotations are included in the data augmentation process, how are the new bounding boxes calculated?

When studying bounding box-based detectors, it's not clear to me if data augmentation includes adding random rotations. If random rotations are added, how is the new bounding box calculated?
3 votes
0 answers
631 views

What is the justification for Kaiming He initialization?

I've been trying to understand where the formulas for Xavier and Kaiming He initialization come from. My understanding is that these initialization schemes come from a desire to keep the gradients ...
  • 181
3 votes
0 answers
49 views

Enforcing sparsity constraints that make use of spatial contiguity

I have a deep learning network that outputs grayscale image reconstructions. In addition to good reconstruction performance (measured through mean squared error or some other measure like psnr), I ...
3 votes
1 answer
110 views

Is batch learning with gradient descent equivalent to "rehearsal" in incremental learning?

I am learning about incremental learning and read that rehearsal learning is retraining with old data. In essence, isn't this the exact same thing as batch learning (with stochastic gradient descent)? ...
3 votes
0 answers
70 views

Would a different learning rate for every neuron and layer mitigate or solve the vanishing gradient problem?

I'm interested in using the sigmoid (or tanh) activation function instead of RELU. I'm aware of RELU advantages on faster computation and no vanishing gradient problem. But about vanishing gradient, ...
3 votes
1 answer
429 views

What are the keys and values of the attention model for the encoder and decoder in the "Attention Is All You Need" paper?

I have recently encountered the paper on NLP. It is very new to me and I am still unable to see how that works. I have used all the resources over there from the original paper to Youtube videos and ...
  • 253
3 votes
0 answers
66 views

Where does reinforcement learning actually show up in Deepmind's game engines?

From the brief research I've done on the topic, it appears that the way Deepmind's Alphazero or Muzero makes decisions is through Monte Carlo tree searches, where in the randomized simulations allows ...
3 votes
0 answers
67 views

Simplification of expected reward under the limit in continuous tasks

I was reading the average reward setting for continuous tasks from rich sutton's book (page 202, 2nd edition). There he perform a simplification over the expected reward under the limit approaching to ...
3 votes
1 answer
264 views

How should I define the loss function for a multi-object detection problem?

I'm trying to create a text recognition project using CNN. I need help regarding the text detection task. I have the training images and bounding box details for them. But I'm unable to figure out ...
  • 31
3 votes
0 answers
207 views

Understanding the results of "Visualizing and Understanding Convolutional Networks"

I am trying to understand the results of the paper Visualizing and Understanding Convolutional Networks, in particular the following image: What are these 3x3 blocks and their 9 cells representing? ...
3 votes
0 answers
41 views

What is the difference between "out-of-distribution (generalisation)" and "(meta)-transfer learning"?

I'm trying to develop a better understanding of the concept of "out-of-distribution" (generalization) in the context of Bengio's "Moving from System 1 DL to System 2 DL" and the concept of "(meta)-...
3 votes
0 answers
456 views

Can Bert be used to extract embedding for large categorical features?

I've lot of training data points (i.e in millions) and I've around few features but the issue with that is all the features are categorical data with 1 million+ categories in each. So, I couldn't use ...
  • 149
3 votes
0 answers
82 views

Why don't the neural networks inside LSTM cells contain hidden layers?

I watched a video explaining how LSTM cells have very rudimentary feed-forward neural networks, basically a 2 layer input-output with no hidden layers. Why don't LSTM cells have more complex neural ...
3 votes
0 answers
112 views

How can I do hyperparameter optimization for a CNN-LSTM neural network?

I have built a CNN-LSTM neural network with 2 inputs and 2 outputs in Keras. I trained the network with model.fit_generator() (and not ...
  • 31
3 votes
0 answers
46 views

Rarely predict minority class imbalanced datasets

I have a dataset in which class A has 99.8%, class B 0.1% and class C 0.1%. If I train my model on this dataset, it predicts always class A. If I do oversampling, it predicts the classes evenly. I ...
3 votes
0 answers
70 views

How does the memory mechanism (reading and writing) work in a neural Turing machine?

In neural Turing machine (NTM), reading memory is represented as \begin{align} r_t \leftarrow \sum\limits_i^R w_t(i) \mathcal{M}_t(i) \tag{2} \end{align} and writing to memory is represented as ...
  • 976
3 votes
0 answers
48 views

Can sequence-to-sequence models be used to convert source code from one programming language to another?

Sequence-to-sequence models have achieved good performance in natural language translation. Could these models also be applied to convert source code written in one programming language to source code ...
3 votes
1 answer
137 views

Tabular Datasets where Deep neural networks outperforms XGBoost

Are there (complex) tabular datasets where deep neural networks (e.g. more than 3 layers) outperform traditional methods such as XGBoost by a large margin? I'd prefer tabular datasets rather than ...
  • 31
3 votes
0 answers
704 views

How does deepfake technology work with multiple people in a single frame?

I was watching this video from corridor crew, according to them, they have used deepfake technology to create this video. I myself have never made a deepfake videos, but I have enough knowledge in the ...
  • 976
3 votes
0 answers
117 views

Is there a deep learning-based architecture for digit localisation?

I'm new to object detectors and segmentation. I want to localize digits on a plate as fast as possible. All images of the dataset are normalized to $300 \times 60$. There are different approaches to ...
3 votes
0 answers
280 views

Image to image regression in tensorflow

I am working on an image to image regression task which requires me to develop a deep learning model that takes in a sequence of 5 images and return another image. The sequence of 5 images and the ...
  • 61
3 votes
0 answers
90 views

Feasibility of using machine learning to obtain self-consistent solutions

I am a physicist and I don't have much background on machine learning or deep learning except taking a couple of courses on statistics. In physics, we often simulate a model by means of two-way ...
3 votes
0 answers
58 views

How do deep fakes get the right encoding for both people?

Deep fakes work by using a single encoder but then having a different decoder for different people. But I wondered what if the encoder encodes say "closed eyes" of person A as the same code for "...
  • 2,106

1
2 3 4 5
15