Questions tagged [deep-learning]

For questions related to deep learning, which refers to a subset of machine learning methods based on artificial neural networks (ANNs) with multiple hidden layers. The adjective deep thus refers to the number of layers of the ANNs. The expression deep learning was apparently introduced (although not in the context of machine learning or ANNs) in 1986 by Rina Dechter in the paper "Learning while searching in constraint-satisfaction-problems".

632 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
3
votes
0answers
38 views

What are the benefits of using the state information that maintains the graph structure?

When you applying a graph structured data to the graph convolution network, what are the benefits of using the state information that maintains the graph structure?
3
votes
1answer
63 views

How to make a distinction between item feature and environment feature?

My data is stock data with features such as stocks' closing prices.I am curious to know if I can put the economy feature such as 'national interest rate' or 'unemployment rate' besides each stocks' ...
3
votes
2answers
170 views

How to fix time dimension in time varying data-sets using deep learning model for classification?

Dataset Description I am working on famous ABIDE Autism Datasets. The dataset is very big in a sense that it has more than 1000 subjects containing half of them as autisitic and other half as ...
3
votes
0answers
170 views

Should noise (such as OU) be decreased over time in actor / critic algorithms?

In most of RL algorithms I saw, there is a coefficient that reduces actions exploration over time, to help convergence. But in Actor-Critic, or other algorithms (A3C, DDPG, ...) used in continuous ...
3
votes
0answers
208 views

DQN Agent not learning anymore - what can I do to fix this?

I am trying to use Deep-Q-Learning to learn an ANN which controls a 7-DOF robotic arm. The robotic arm must avoid an obstacle and reach a target. I have implemented a number of state-of-art ...
3
votes
0answers
172 views

How does bidirectional encoding allow the predicted word to indirectly “see itself”?

Before the release of BERT, we used to say that it is not possible to train bidirectional models by simply conditioning each word on its previous and next words, since this would allow the word that's ...
3
votes
0answers
352 views

How to encode Azul game state as NN input

Question to NN practicioners. I'd like to encode Azul board game state as an input to NN, let's focus on 2-player variant for a while. There are 5 round "Factories" on the table (7 on picture, ignore ...
3
votes
0answers
48 views

What is meant by “model discriminability for local patches within the receptive field”?

In the Abstract section of the paper Network In Network, what does the authors actually mean to say?
3
votes
1answer
490 views

Alternative to sliding window neural network (was: Object detect (or) image classification at specific locations in the frame)

Recent advances in Deeplearning and dedicated hardware has made it possible to detect images with a much better accuracy than ever. Neural networks are the gold standard for computer vision ...
3
votes
0answers
473 views

Mountain car problem with images - not converging

I'm trying to find the optimal policy for the mountain car problem using deep Q learning with images as input, however, I cannot find a way to get my Q function to give me good solutions (I followed ...
3
votes
1answer
549 views

Periodic Pattern in Validation Loss Curve

I am currently trying to solve a regression problem using neural networks. I want to detect movement patterns in images over time (video) and output a continuous value. During the training process I ...
3
votes
0answers
213 views

What is the relation between the definition of learnability of Vapnik and Gold and learnability of neural networks?

Gold showed that a language can be learned only if it contains a finite set of sentences. We know that deep neural networks can implement any function. Does this contradict the Gold's result? What ...
3
votes
0answers
40 views

What is the “question” when using Dynamic Memory Networks to do part-of-speech tagging and sentiment analysis?

In the paper Ask Me Anything: Dynamic Memory Networks for Natural Language Processing the authors described a Dynamic Memory Network in the context of question answering. Then, they also tested the ...
3
votes
0answers
96 views

How is the distribution of the state related to the distribution of the units in Boltzmann machines?

I'm trying to understand Boltzmann machines. Tutorials explain it with two formulas. Logistic function for the probability of single units: $$p(unit=1)=\frac{1}{1+e^{-\sum_{x}wx } }$$ and, when the ...
2
votes
0answers
21 views

What are the metrics to be used for unsupervised monocular depth estimation in computer vision?

I am currently replicating the results of this paper. In this paper they have not mentioned how they are evaluating the results as no ground truth is available for comparison. Same goes for other ...
2
votes
0answers
42 views

Are there any new weight initialization techniques for DNN published after 2015?

Considering weights initialization in my personal projects, I always used some standard techniques such as: Glorot (also known as Xavier) initialization (2010). Mertens initialization (2010). He ...
2
votes
0answers
26 views

Why does batch norm standardize with sample mean/variance, when it also learns parameters to scale the mean/variance?

Batch norm is a normalizing layer that is shown to help deep networks learn faster and with higher generalization accuracy. It normalizes the activations of the previous layer to a mean $\beta$ and ...
2
votes
0answers
47 views

How can I use Monte Carlo Dropout in a pre-trained CNN model?

In Monte Carlo Dropout (MCD), I know that I should enable dropout during training and testing, then get multiple predictions for the same input $x$ by performing multiple forward passes with $x$, then,...
2
votes
0answers
38 views

How are nested bounding boxes handled in object detection (and in particular in the case of the SSD)?

The basic approach to non-maximum-suppression makes sense, but I am kind of confused about how you handle nested bounding boxes. Suppose you have two predicted boxes, with one completely enclosing ...
2
votes
0answers
19 views

Interpretation of Inner Product in a two-tower model

I have seen at quite a few places the use of two-tower architecture. This(Fig 6) is one of the examples. Each tower computes embedding of a concept which is orthogonal to the concepts in the rest of ...
2
votes
0answers
70 views

How to handle long sequences with transformers?

I have a time series sequence with 10 million steps. In step $t$, I have a 400 dimensional feature vector $X_t$ and a scalar value $y_t$ which I want to predict during inference time and I know during ...
2
votes
0answers
22 views

Why does the loss stops reducing after a point in this Transformer Model?

Context I was making a Transformer Model to convert English Sentences to German Sentences. But the loss stops reducing after some time. Code ...
2
votes
0answers
64 views

Why should the weight updates be proportional to input?

I'm reading the book Grokking Deep Learning. Regarding weight updates during training, it has the following code and explanation: ...
2
votes
0answers
34 views

How to define loss function for Discriminator in GANs?

To train the discriminator network in GANs we set the label for the true samples as $1$ and $0$ for fake ones. Then we use binary cross-entropy loss for training. Since we set the label $1$ for true ...
2
votes
0answers
25 views

What is the difference between text-based image retrieval and natural language object retrieval?

I'm working on creating a model that locates the object in the scene (2D image or 3D scene) using a natural language query. I came across this paper on natural language object retrieval, which ...
2
votes
1answer
64 views

Is batch learning with gradient descent equivalent to “rehearsal” in incremental learning?

I am learning about incremental learning and read that rehearsal learning is retraining with old data. In essence, isn't this the exact same thing as batch learning (with stochastic gradient descent)? ...
2
votes
0answers
38 views

Handling a Large Discrete Action Space in Deep Q Learning

I am attempting to solve a timetabling problem using deep Q learning. It could be thought of as a resource allocation problem to obtain some certificate of 'optimality'. However, how to define and ...
2
votes
0answers
19 views

How is the data labelled in order to train a region proposal network?

I don't get how the training of the RPN works. From the forward propagation, I have $W \times H \times k$ outputs from the RPN. How is the training data labeled such that I can use the loss function ...
2
votes
0answers
32 views

Combine DQN with the Average Reward setting

I have to deal with a non-episodic task, where there is addittionally a continuous state space and more specifically in each time step there is always a new state that has never been seen before. I ...
2
votes
0answers
93 views

What is the memory complexity of the memory-efficient attention in Reformer?

When I read the paper, Reformer: The Efficient Transformer, I cannot get the same complexity of the memory-efficient method in Table 1 (p. 5), which summarizes time/memory complexity of scaled dot-...
2
votes
0answers
49 views

What is meant by “arranging the final features of CNN in a grid” and how to do it?

In the paper What You Get Is What You See: A Visual Markup Decompiler, the authors have proposed a method to extract the features from the CNN and then arrange those extracted features in a grid to ...
2
votes
0answers
63 views

What is a heatmap in the CornerNet paper?

I have been working on understanding how CornerNet works, but I couldn't figure out a few parts about the architecture. First, the authors mention that there are 3 distinct parts to be predicted as a ...
2
votes
0answers
76 views

Inaccurate masks with Mask-RCNN: Stairs effect and sudden stops

I've been using matterport's Mask R-CNN to train on a custom dataset. However, there seem to be some parameters that i failed to correctly define because on practically all of the images, the bottom ...
2
votes
0answers
80 views

Should batch normalisation be applied before or after ReLU?

I know that there has been some discussion about this (e.g. here and here), but I can't seem to find consensus. The crucial thing that I haven't seen mentioned in these discussions is that applying ...
2
votes
0answers
40 views

In a DQN, can Prioritized Experience Replay actually perform worse than a regular Experience Replay?

I've written a Double DQN-based stock trading bot using mainly time series stock data. I've recently upgraded my Experience Replay(ER) code with a version of Prioritized Experience Replay (PER) ...
2
votes
0answers
44 views

How to understand the matrices used in the Attention layer?

Attention-scoring mechanism seems to be a commonly-used component in various seq2seq models, and I was reading about the original "Location-based Attention" in Bahadanau well-known paper at https://...
2
votes
0answers
37 views

Incorporating domain knowledge into recurrent network

I am currently trying to solve a classification task with a recurrent artificial neural network (RNN). Situation There are up to 350 inputs (X) mapped on one categorical output (y)(13 differnt ...
2
votes
0answers
36 views

Can you find another reason for sample inefficiency of model-free on-policy Deep Reinforcement Learning?

The following mindmap gives an overview of multiple reasons for sample inefficiency. The list is definitely not complete. Can you see another reason not mentioned so far? Some related links: ...
2
votes
0answers
26 views

How does sampling works in case of imbalanced image datasets?

I am solving a problem of image classification of the image dataset for 3 classes. Dataset is highly imbalanced. How will sampling (either over- or under-sampling) work in that case? Should I remove (...
2
votes
0answers
27 views

How is visual attention mechanism different from a two branch convolutional neural network?

I am doing some research on the visual attention mechanism in remote sensing domain (where the features learnt from one layer are highlighted using the attention mask derived from another layer). From ...
2
votes
1answer
58 views

Does L1/L2 Regularization help reach an optimum result faster?

I understand that L1 and L2 regularization helps to prevent overfitting. My question is then, does that mean they also help a neural network learn faster as a result? The way I'm thinking is that ...
2
votes
1answer
87 views

Can you explain me this CNN architecture?

I am starting to get my head around convolutional neural networks, and I have been working with the CIFAR-10 dataset and some research papers that used it. In one of these papers, they mention a ...
2
votes
0answers
48 views

Can a typical supervised learning problem be solved with reinforcement learning methods?

Let's say I want to teach a neural to classify images, and, for some reason, I insist on using reinforcement learning rather than supervised learning. I have a dataset of images and their matching ...
2
votes
0answers
47 views

Is there any application of topology to deep learning?

Is there any application of topology (as in math discipline) to deep learning? If so, what are some examples?
2
votes
1answer
59 views

How should I define the loss function for a multi-object detection problem?

I'm trying to create a text recognition project using CNN. I need help regarding the text detection task. I have the training images and bounding box details for them. But I'm unable to figure out ...
2
votes
1answer
48 views

A model for each sub-problem vs one model for the whole problem

Let's say one wants to use a neural net to learn some function $g(x)$. Let's say that we know that $g$ is a combination of two functions (or two sub-problems), $g(x)=f_2(f_1(x))$, and that we have two ...
2
votes
0answers
32 views

Single label classification into hierarchical categories using a neural network

I am working on a classification problem into progressive classes. In other words, there is some hierarchy of categories in such a way, that A < B < C, e.g. low, medium, high, very high. What ...
2
votes
0answers
38 views

Creating Dataset for Image Classification

I want to develop a CNN model to identify 24 hand signs in American Sign Language. I created a custom dataset that contains 3000 images for each hand sign i.e. 72000 images in the entire dataset. For ...
2
votes
0answers
33 views

Possible approaches to dealing with unbalanced dataset and highly biased deep learning algorithm

I have an extremely unbalanced video dataset for a two class video classification problem.All my videos in my current video dataset is $40$ second long with $900p$ resolution.However the dataset is ...
2
votes
0answers
45 views

Is Mean Squared Error Loss function a good loss function for continuous variables $0 < x < 1$

Suppose I am utilising a neural network to predict the next state, $s'$ based on the current $(s, a)$ pairs. all my neural network inputs are between 0 and 1 and the loss function for this network ...

1
2
3 4 5
13