# Questions tagged [deep-learning]

For questions related to deep learning, which refers to a subset of machine learning methods based on artificial neural networks (ANNs) with multiple hidden layers. The adjective deep thus refers to the number of layers of the ANNs. The expression deep learning was apparently introduced (although not in the context of machine learning or ANNs) in 1986 by Rina Dechter in the paper "Learning while searching in constraint-satisfaction-problems".

1,414 questions
Filter by
Sorted by
Tagged with
44 views

### Why is the mean used to compute the expectation in the GAN loss?

From Goodfellow et al. (2014), we have the adversarial loss:  \min_G \, \max_D V (D, G) = \mathbb{E}_{x∼p_{data}(x)} \, [\log \, D(x)] \\ \quad\quad\quad\quad\quad\quad\quad + \, \mathbb{E}_{z∼p_z(...
15 views

### How can I decrease the time to compute the mask in the Mask-RCNN for human body detection?

I am using Mask-RCNN to detect human bodies in photos, to get a rough approximation of the ratio of their heights to the length of their chests. I want to decrease the time for making the mask of the ...
15 views

### Are there any deep learning tools so that can be used to estimate long-tail distributions?

Are there any deep learning tools so that can be used to estimate long-tail distributions? update: my context is there are is there is a long-tail distribution (probability distribution) in my data of ...
84 views

### How can we teach a neural net to make arbitrary data associations?

Let's say I have pairs of keys and values of the form $(x_1, y_1), \dots, (x_N, y_N)$. Then I give a neural net a key and a value, $(x_i, y_i)$. For example, $x_i$ could be $4$ and $y_i$ could be $3$, ...
16 views

### Why it is reshaped the last layers of VGG_UNet segmentation model?

I want to do a multiclass segmentation task using deep learning (in python). Here, is a summary of vgg_unet model that is mainly collected from GitHub. So, in my dataset 8 labels are available. So, at ...
26 views

### Does a better discriminator in GANs mean better sample generation by the generator?

Since the discriminator defines how the generator is updated, then building a discriminator with a higher number of parameters/more layers should lead to a better quality of generated samples. So, ...
24 views

### Does a constant low validation loss mean great model accuracy?

This is more of a conceptual question that I am asking here. I am using an encoder-decoder model with Bidirectional LSTMs used as a time series regression problem. I train it with a sufficiently low ...
37 views

### How to take the optimal batch_size for training a model?

I have an image dataset, which is composed of 113695 images for training and 28424 images for validation. Now, when I use ImageDataGenerator and ...
67 views

### How to determine the number of hidden layers and units of a deep auto-encoder?

I am using a deep autoencoder for my problem. However, the way I choose the number of hidden layers and hidden units in a hidden layer is still based on my feeling. The size of the model that ...
10 views

### Generative Adversarial Network with two images in input

I am doing an internship project regarding deep learning, and it is a totally new topic for me as I have never studied machine learning in the bachelor's degree courses. I have to implement a GAN that ...
176 views

### How should I choose the target's update frequency in DQN?

I have been dealing with a problem that I'm trying to solve with DQN. A general question that I have is regarding the target's update frequency. How should it change? Depending on what factor do we ...
32 views

### Why does my loss value of autoencoder in PyTorch is negative?

I am trying to implement SDNE, a algorithm uses deep auto encoder to map a graph to latent representation d dimension. The idea is kind of simple, SDNE uses the ...
36 views

### How can I implement 2D CNN filter with channelwise-bound kernel weights?

I would like to bind kernel parameters through channels/feature-maps for each filter. In a conv2d operation, each filter consists of HxWxC parameters I would like to have filters that have HxW ...
148 views

### What is the scope of real-world deep learning applications in 2020?

2015 was a milestone year for AI--"deep learning" was validated in a very public way with AlphaGo. However, at the time, the question was raised: "What else is deep learning good for?&...
16 views

### Transfer Learning: Finetune a model with a splitted dataset?

Lets say I want to fine-tune a model. I have a pretrained ResNet model and on top of this model I add some extra layers. And lets say I have a dataset of 10,000 images. The recommended way would be: ...
60 views

### In DQN, when do the parameters in the Neural Network update based on the reward received?

I'm aware that we back-propagate after computing the loss between: The Neural Network Q values and the Target Network Q values However, all this is doing is updating the parameters of the Neural ...
44 views

### Why are Target Networks used in Deep Q-Learning as opposed to the Expected Value equation?

I understand we use a target network because it helps resolve issues regarding stability, however, that's not what I'm here to ask. What I would like to understand is why a target network is used as a ...
73 views

### What is Precision@K for link prediction in graph embedding meaning?

I am trying to re-implement the SDNE algorithm for graph embedding by PyTorch. I get stuck at some issues about evaluation metric Precision@K. precision@k is a metric which gives equal weight to the ...
47 views

### What is the representational capacity of a learning algorithm? [duplicate]

The definition I see for representational capacity is "the family of functions the learning algorithm can choose from when varying the parameters in order to reduce a training objective." (...
15 views

### How does YOLO handle non-class objects?

I have been reading more about computer vision and I'm bothered by YOLO and similar deep learning architectures. The thing I am confused on is how non-class image sections are dealt with, in ...
14 views

### Understanding graphs of the mean square error: relationships between val loss and train loss

I am currently working with some models aimed at predicting time series (89 days for training, 22 for testing), including a CNN LSTM and a convLSTM. When training these models, I had the following ...
895 views

### What is the computational complexity of the forward pass of a convolutional neural network?

How do I determine the computational complexity (big-O notation) of the forward pass of a convolutional neural network? Let's assume for simplicity that we use zero-padding such that the input size ...
70 views

### What is the state-of-the-art algorithm for neural style transfer?

I've read the paper A Neural Algorithm of Artistic Style by Gatys et. al. and I find the application of neural style transfer very fun. I also read that Exploring the structure of a real-time, ...
41 views

### Would a different learning rate for every neuron and layer mitigate or solve the vanishing gradient problem?

I'm interested in using the sigmoid (or tanh) activation function instead of RELU. I'm aware of RELU advantages on faster computation and no vanishing gradient problem. But about vanishing gradient, ...
31 views

### Keras model accuracy not improving beyond threshold

I am currently working on a public project for the National Weather Model. We are experimenting with using a recurrent neural network to replace the output of a quadratic formula that is in use. The ...
31 views

### How to add voice inflections to an existing SV2TTS voice cloning implementation?

I've been playing with the SV2TTS voice cloning implementation provided on Github: https://github.com/CorentinJ/Real-Time-Voice-Cloning It was straight forward to use as a TTS engine with clearly ...
258 views

### How to generate labels for self-supervised training?

I've been reading a lot lately about self-supervised learning and I didn't understand very well how to generate the desired label for a given image. Let's say that I have an image classification task, ...
61 views

### How can I find a specific word in an audio file?

I'm trying to train and use a neural network to detect a specific word in an audio file. The input of the neural network is an audio of 2-3 seconds duration, and the neural network must determine ...
53 views

### How to make sense of label propagation formula in graph neural networks?

In the label propagation algorithm in section 3.2.3, we know the label of some nodes and we want to predict the label for the rest of the nodes whose labels we don't know. The update formula for this ...
20 views

### What are some solutions for dealing with time series data that are recorded at uneven intervals?

Let's say I have a time series data which is a bunch of observations that occur at different time stamps and intervals. For example, my observations come from a camera located at a traffic ...
27 views

### Is Bayesian NN vs adding random data more accurate?

I’m trying to train a classifier to recognize if people are wearing seatbelts. What if the person submitted a picture unrelated to a seatbelt classifier? Would I create an image label that is full of ...
34 views

### Why do we use the Target Network for action evaluation in Double deep Q networks

Is there any specific reason as to why The target Network is used for evaluation and The online network Is used for selection, what would be the difference if both roles were switched, our online ...
69 views

### Can AlphaZero considered as Multi-Agent Deep Reinforcement Learning?

Can AlphaZero considered as Multi-Agent Deep Reinforcement Learning? I could not find a clear answer on this. I would say yes it is Multi Agent Learning, as there are two Agents playing against each ...
88 views

### How to use text as an input for a neural network - regression problem? How many likes/claps an article will get

I am trying to predict the number of likes an article or a post will get using a NN. I have a dataframe with ~70,000 rows and 2 columns: "text" (predictor - strings of text) and "likes&...
38 views

### Having trouble understanding how Double deep Q networks work

I’ve looked at various articles and I’m still very confused, I understand the normal double Q learning about having two Action value estimates that use two different set of samples But coming to ...
54 views

### How to handle images that don’t pertain to image classifier at all?

I am trying to create a CNN model that classifies if a person is wearing a seatbelt or not to verify they drive safely. I know to get images of people wearing seatbelts and people not wearing ...
45 views

### The are some fundamental learning theories for developing an AI that imitates human behavior

Most if not all AI systems are not to imitate human, but to finally out-perform human. Examples include using AI to play a game, classification problems, auto-driving, and goal-oriented chatbots. ...
37 views

### What is the reason for implementing position-wise Feed Forward Network in Transformer?

Does anyone know the reason why there's a FCNN after the self-attention layer? Or at least some intuition for it?
103 views

### Is gradient descent scale invariant or not?

I know we should scale the input and output (assuming regression task) before we feed it to the neural network. Then the gradient descent will give the better minima much faster. But I have subtle ...
35 views

### Is it a good idea to train a neural network to classify images without base-hypothesis?

I'm a relative beginner in deep-learning (understand by that, I'm doing my first kaggle competition right now, and I have loads to learn still) and I was just wondering something. Let's say you have ...
86 views

### Who first coined the term “deep learning”?

AFAIK, deep learning became popular in 2012 with the victory of ImageNet Competition - Large Scale Visual Recognition Challenge 2012 where winners of this contest actually used deep learning ...
88 views

### How to train an LSTM with varying length input?

I have a dataset where each of the training instances is different in the length and the data is sequential. So, I design an LSTM but I am thinking about how to train the LSTM. In fixed-length data, ...
89 views

### What is the memory complexity of the memory-efficient attention in Reformer?

When I read the paper, Reformer: The Efficient Transformer, I cannot get the same complexity of the memory-efficient method in Table 1 (p. 5), which summarizes time/memory complexity of scaled dot-...
82 views

### How large should the corpus be to optimally retrain the GPT-2 model?

I just started working with the GPT-2 models and want to retrain one on a pretty narrow topic, so I have problems finding training material. How large should the corpus be to optimally retrain the GPT-...
70 views

### What is the difference between artificial neural network (ANN) and deep learning?

I have read many mixed definitions around these two terms. For example, is it right to say deep learning is any ANN with more than two hidden layers? What are formal definitions for these two?
80 views

### Combining clustering and deep learning for computer vision

Is there any recent work on combining clustering approaches (k-means, or gaussian mixture or PGM) with deep learning for computer vision? In particular I'm interested in if anyone has used the first ...
93 views

### Why L2 loss is more commonly used in Neural Networks than other loss functions?

Why L2 loss is more commonly used in Neural Networks than other loss functions? What is the reason to L2 being a default choice in Neural Networks?
65 views

### In what situations ELUs should be used instead of RELUs?

I always use RELUs actication functions when I need to and I understand limitations of ELUs. So in what situation do I need to consider ELUs over RELUs?
Problem (Sketch): I'm interested in a particular formulation of the transfer-learning problem, which, given a trained network $f$ seeks to learn a new network $g$ whose last few layers behave very ...
### Why can we use a network to estimate $Q_\pi(s, a)$ in Actor-Critic Method?
According to deep Q learning, we want to learn $Q^*(s,a)$, which is the optimal action-value function. It does make sense because we assume there is only one optimal function so the algorithm will ...