Questions tagged [deep-learning]

For questions related to deep learning, which refers to a subset of machine learning methods based on artificial neural networks (ANNs) with multiple hidden layers. The adjective deep thus refers to the number of layers of the ANNs. The expression deep learning was apparently introduced (although not in the context of machine learning or ANNs) in 1986 by Rina Dechter in the paper "Learning while searching in constraint-satisfaction-problems".

774 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
16 votes
1 answer
350 views

Will parameter sweeping on one split of data followed by cross validation discover the right hyperparameters?

Let's call our dataset splits train/test/evaluate. We're in a situation where we require months of data. So we prefer to use the evaluation dataset as infrequently as possible to avoid polluting our ...
Philipp Cannons's user avatar
11 votes
0 answers
380 views

Extending FaceNet’s triplet loss to object recognition

FaceNet uses a novel loss metric (triplet loss) to train a model to output embeddings (128-D from the paper), such that any two faces of the same identity will have a small Euclidean distance, and ...
Benedict Aaron Tjandra's user avatar
7 votes
2 answers
131 views

Can training a model on a dataset composed by real images and drawings hurt the training process of a real-world application model?

I'm training a multi-label classifier that's supposed to be tested on underwater images. I'm wondering if feeding the model drawings of a certain class plus real images can affect the results badly. ...
user's user avatar
  • 101
7 votes
0 answers
2k views

What are the differences between Yolo v1 and CenterNet?

I recently read a new paper (late 2019) about a one-shot object detector called CenterNet. Apart from this, I'm using Yolo (V3) one-shot detector, and what surprised me is the close similarity between ...
Louis Lac's user avatar
  • 318
6 votes
1 answer
96 views

It is possible to use deep learning to give approximate solutions to NP-hard graph theory problems?

It is possible to use deep learning to give approximate solutions to NP-hard graph theory problems? If we take, for example, the travelling salesman problem (or the dominating set problem). Let's say ...
Jake B.'s user avatar
  • 181
6 votes
0 answers
170 views

Why Pixel RNN (Row LSTM) can capture triangular contexts?

I'm reading the paper Pixel Recurrent Neural Network. I have a question about Row LSTM. Why Row LSTM can capture triangular contexts? In this paper, the kernel of the one-dimensional convolution ...
musako's user avatar
  • 181
6 votes
0 answers
351 views

How to correctly implement self-play with DQN?

I have an environment where an agent faces an equal opponent, and while I've achieved OK performance implementing DQN and treating the opponent as a part of the environment, I think performance would ...
Pell000's user avatar
  • 61
5 votes
0 answers
804 views

What is the justification for Kaiming He initialization?

I've been trying to understand where the formulas for Xavier and Kaiming He initialization come from. My understanding is that these initialization schemes come from a desire to keep the gradients ...
Jack M's user avatar
  • 252
5 votes
0 answers
68 views

Training and inference for highly-context-sensitive information

What is the best way to train / do inference when the context matters highly as to what the inferred result should be? For example in the image below all people are standing upright, but because of ...
g491's user avatar
  • 101
5 votes
2 answers
808 views

What's the difference between architectures and backbones?

In the paper "ForestNet: Classifying Drivers of Deforestation in Indonesia using Deep Learning on Satellite Imagery", the authors talk about using: Feature Pyramid Networks (as the ...
codinggirl123's user avatar
4 votes
0 answers
85 views

What are the advantages of GANs over Diffusion Models in image generation?

Diffusion Models have recently gained popularity in the field of image generation, with widely used products such as Stable Diffusion employing this approach and yielding impressive results. While ...
David's user avatar
  • 61
4 votes
2 answers
141 views

Why are some Neural Networks more forgiving on Quantization?

I know this might be a bit general question and concerning a rather active research field, much beyond my expertise, but I do believe there're some answers. The use of NN parameters quantization can ...
edmz's user avatar
  • 91
4 votes
0 answers
2k views

Should batch normalisation be applied before or after ReLU?

I know that there has been some discussion about this (e.g. here and here), but I can't seem to find consensus. The crucial thing that I haven't seen mentioned in these discussions is that applying ...
Kris's user avatar
  • 171
4 votes
1 answer
455 views

How should I define the loss function for a multi-object detection problem?

I'm trying to create a text recognition project using CNN. I need help regarding the text detection task. I have the training images and bounding box details for them. But I'm unable to figure out ...
h4x's user avatar
  • 41
4 votes
0 answers
268 views

Understanding the results of "Visualizing and Understanding Convolutional Networks"

I am trying to understand the results of the paper Visualizing and Understanding Convolutional Networks, in particular the following image: What are these 3x3 blocks and their 9 cells representing? ...
Andreas K.'s user avatar
4 votes
0 answers
63 views

Can sequence-to-sequence models be used to convert source code from one programming language to another?

Sequence-to-sequence models have achieved good performance in natural language translation. Could these models also be applied to convert source code written in one programming language to source code ...
pfds2222's user avatar
4 votes
2 answers
248 views

What kind of output should be used for predicting angles in DNNs?

I am building a model which predicts angles as output. What are the different kinds of outputs that can be used to predict angles? For example, output the angle in radians cyclic nature of the ...
Yashas's user avatar
  • 189
4 votes
0 answers
581 views

What is the difference between GAT and GaAN?

I was looking at two papers Graph Attention Networks (GAT) by Petar Veličković and GaAN: Gated Attention Networks for Learning on Large and Spatiotemporal Graphs by Jiani Zhang. I'm trying to ...
razvanc92's user avatar
  • 1,108
4 votes
1 answer
467 views

How did the OpenAI 5 for Dota concatenate units?

I am no expert in the field of AI so I apologize if this is a simple/easy question. I was trying to implement a network similar to OpenAI's for another game and I noticed that I did not fully ...
Isamu Isozaki's user avatar
4 votes
0 answers
204 views

What characteristics make it difficult for a Neural Network to approximate a function?

What are the characteristics which make a function difficult for the Neural Network to approximate? Intuitively, one might think uneven functions might be difficult to approximate, but uneven ...
user avatar
4 votes
0 answers
59 views

What is the motivation for row-wise convolution and folding in Kalchbrenner et al. (2014)?

I was reading the paper by Kalchbrenner et al. titled A Convolutional Neural Network for Modelling Sentences and I am struggling to understand their definition of convolutional layer. First, let's ...
Tomasz Garbus's user avatar
4 votes
0 answers
347 views

What are the ways to calculate the error rate of a deep Convolutional Neural Network, when the network produces different results using the same data?

I am new to the object recognition community. Here I am asking about the broadly accepted ways to calculate the error rate of a deep CNN when the network produces different results using the same data....
Daqi Dong's user avatar
4 votes
1 answer
2k views

Which other loss functions for hierarchical multi-label classification could I use?

I am looking to try different loss functions for a hierarchical multi-label classification problem. So far, I have been training different models or submodels like multilayer perceptron (MLP) branch ...
Skinish's user avatar
  • 153
4 votes
4 answers
1k views

Use Machine/Deep Learning to Guess a String

I want to be able to input a block of text and then have it guess a string within a predefined range (i.e. a string that starts with three letters and ends with five numbers like "XXX12345", etc). ...
TreHoffman's user avatar
4 votes
0 answers
353 views

What are some interesting recent papers that synthesize symbolic AI with Deep Learning?

A lot of people seem to be under the impression that combining GOFAI and contemporary AI will make models more general. I'm particularly interested in reasoning through analogy or case-based reasoning....
Daniel's user avatar
  • 326
3 votes
0 answers
51 views

Why policy gradient theorem has two different forms?

I have been studying policy gradients recently but found different expositions from different sources, which greatly confused me. From the book "Reinforcement Learning: an Introduction (Sutton &...
Yuxiang Wei's user avatar
3 votes
0 answers
20 views

Why does training converges when the norm of gradient increases?

This is from deep learning book by Ian Goodfellow and Yoshua Bengio and Aaron Courville. When training converges well, I thought the gradient should be at local minima. But the book says it often does ...
tesio's user avatar
  • 195
3 votes
0 answers
126 views

Best calculus books for Deep Learning

Recommend some calculus books for Deep Learning and neural networks. I know what is integration, differentiation, derivates, limits on a based level. I would like to understand on deep level the ...
Dan Il's user avatar
  • 31
3 votes
0 answers
786 views

What is the difference between prompt tuning and prefix tuning?

I read prompt tuning and prefix tuning are two effective mechanisms to leverage frozen language models to perform downstream tasks. What is the difference between the two and how they work really? ...
Exploring's user avatar
  • 293
3 votes
1 answer
290 views

For which problem sizes is Deep Q-Learning suitable and why?

I am wondering for which problem sizes a Deep Q-Learning algorithm is most appropriate. For example, whether it is particularly suited for low complexity problems or not for high complexity problems. ...
user avatar
3 votes
0 answers
253 views

Are there neural networks with (hard) constraints on the weights?

I don't know too much about Deep Learning, so my question might be silly. However, I was wondering whether there are NN architectures with some hard constraints on the weights of some layers. For ...
Onil90's user avatar
  • 173
3 votes
0 answers
49 views

Why might the convolution be inappropriate when the task involves incorporating information from very distant locations in the input?

When I am reading about convolutional neural networks, I have encountered the following sentence from the textbook(page 341) that says about the limitation of the usage of the convolution in CNNs. ...
satya's user avatar
  • 187
3 votes
0 answers
63 views

How are Ground truth provided to each Pyramid map in RetinaNet or YOLOv3 Paper? How is the mapping of Feature Pyramids done to Ground Truth

SO the YOLO V3 and RetinaNet both uses the Feature pyramids which look something like this: (except b and e which have one ...
Deshwal's user avatar
  • 253
3 votes
0 answers
425 views

How can I use Monte Carlo Dropout in a pre-trained CNN model?

In Monte Carlo Dropout (MCD), I know that I should enable dropout during training and testing, then get multiple predictions for the same input $x$ by performing multiple forward passes with $x$, then,...
lebebop's user avatar
  • 31
3 votes
0 answers
443 views

Stack of Planes as the Action Space Representation for AlphaZero (Chess)

I have a question regarding the action space of the policy network used in AlphaZero. From the paper: We represent the policy π(a|s) by a 8 × 8 × 73 stack of planes encoding a probability ...
sb3's user avatar
  • 137
3 votes
0 answers
40 views

If random rotations are included in the data augmentation process, how are the new bounding boxes calculated?

When studying bounding box-based detectors, it's not clear to me if data augmentation includes adding random rotations. If random rotations are added, how is the new bounding box calculated?
FourierFlux's user avatar
3 votes
0 answers
59 views

Enforcing sparsity constraints that make use of spatial contiguity

I have a deep learning network that outputs grayscale image reconstructions. In addition to good reconstruction performance (measured through mean squared error or some other measure like psnr), I ...
Jane Sully's user avatar
3 votes
1 answer
159 views

Is batch learning with gradient descent equivalent to "rehearsal" in incremental learning?

I am learning about incremental learning and read that rehearsal learning is retraining with old data. In essence, isn't this the exact same thing as batch learning (with stochastic gradient descent)? ...
JobHunter69's user avatar
3 votes
0 answers
81 views

Would a different learning rate for every neuron and layer mitigate or solve the vanishing gradient problem?

I'm interested in using the sigmoid (or tanh) activation function instead of RELU. I'm aware of RELU advantages on faster computation and no vanishing gradient problem. But about vanishing gradient, ...
Rogelio Triviño's user avatar
3 votes
1 answer
765 views

What are the keys and values of the attention model for the encoder and decoder in the "Attention Is All You Need" paper?

I have recently encountered the paper on NLP. It is very new to me and I am still unable to see how that works. I have used all the resources over there from the original paper to Youtube videos and ...
Deshwal's user avatar
  • 253
3 votes
0 answers
80 views

Where does reinforcement learning actually show up in Deepmind's game engines?

From the brief research I've done on the topic, it appears that the way Deepmind's Alphazero or Muzero makes decisions is through Monte Carlo tree searches, where in the randomized simulations allows ...
Amar Srivastava's user avatar
3 votes
0 answers
71 views

Simplification of expected reward under the limit in continuous tasks

I was reading the average reward setting for continuous tasks from rich sutton's book (page 202, 2nd edition). There he perform a simplification over the expected reward under the limit approaching to ...
Swakshar Deb's user avatar
3 votes
0 answers
45 views

What is the difference between "out-of-distribution (generalisation)" and "(meta)-transfer learning"?

I'm trying to develop a better understanding of the concept of "out-of-distribution" (generalization) in the context of Bengio's "Moving from System 1 DL to System 2 DL" and the concept of "(meta)-...
maxcompression's user avatar
3 votes
0 answers
915 views

How does adding noise to the action in DDPG help in learning?

I can't understand how playing with the action generated by the actor network in DDPG by adding the noise term helps in exploration.
Ahmad Fares's user avatar
3 votes
0 answers
542 views

Can Bert be used to extract embedding for large categorical features?

I've lot of training data points (i.e in millions) and I've around few features but the issue with that is all the features are categorical data with 1 million+ categories in each. So, I couldn't use ...
user_12's user avatar
  • 149
3 votes
0 answers
97 views

Why don't the neural networks inside LSTM cells contain hidden layers?

I watched a video explaining how LSTM cells have very rudimentary feed-forward neural networks, basically a 2 layer input-output with no hidden layers. Why don't LSTM cells have more complex neural ...
Snowybluesky's user avatar
3 votes
0 answers
1k views

Why isn't there a model playing FPS like CoD or Battlefield already existing?

Assuming we had an unlimited time to train a model and a very powerful machine to use our model in real-time (hello quantum computer), I'd like to know why no one could achieve to build an AI able to ...
politinsa's user avatar
  • 131
3 votes
0 answers
156 views

How can I do hyperparameter optimization for a CNN-LSTM neural network?

I have built a CNN-LSTM neural network with 2 inputs and 2 outputs in Keras. I trained the network with model.fit_generator() (and not ...
Yining's user avatar
  • 31
3 votes
0 answers
53 views

Rarely predict minority class imbalanced datasets

I have a dataset in which class A has 99.8%, class B 0.1% and class C 0.1%. If I train my model on this dataset, it predicts always class A. If I do oversampling, it predicts the classes evenly. I ...
Johnny P.'s user avatar
3 votes
0 answers
78 views

How does the memory mechanism (reading and writing) work in a neural Turing machine?

In neural Turing machine (NTM), reading memory is represented as \begin{align} r_t \leftarrow \sum\limits_i^R w_t(i) \mathcal{M}_t(i) \tag{2} \end{align} and writing to memory is represented as ...
Eka's user avatar
  • 1,036

1
2 3 4 5
16