Questions tagged [deep-learning]

For questions related to deep learning, which refers to a subset of machine learning methods based on artificial neural networks (ANNs) with multiple hidden layers. The adjective deep thus refers to the number of layers of the ANNs. The expression deep learning was apparently introduced (although not in the context of machine learning or ANNs) in 1986 by Rina Dechter in the paper "Learning while searching in constraint-satisfaction-problems".

Filter by
Sorted by
Tagged with
3
votes
1answer
44 views

Why is the mean used to compute the expectation in the GAN loss?

From Goodfellow et al. (2014), we have the adversarial loss: $$ \min_G \, \max_D V (D, G) = \mathbb{E}_{x∼p_{data}(x)} \, [\log \, D(x)] \\ \quad\quad\quad\quad\quad\quad\quad + \, \mathbb{E}_{z∼p_z(...
0
votes
0answers
15 views

How can I decrease the time to compute the mask in the Mask-RCNN for human body detection?

I am using Mask-RCNN to detect human bodies in photos, to get a rough approximation of the ratio of their heights to the length of their chests. I want to decrease the time for making the mask of the ...
0
votes
0answers
15 views

Are there any deep learning tools so that can be used to estimate long-tail distributions?

Are there any deep learning tools so that can be used to estimate long-tail distributions? update: my context is there are is there is a long-tail distribution (probability distribution) in my data of ...
1
vote
1answer
84 views

How can we teach a neural net to make arbitrary data associations?

Let's say I have pairs of keys and values of the form $(x_1, y_1), \dots, (x_N, y_N)$. Then I give a neural net a key and a value, $(x_i, y_i)$. For example, $x_i$ could be $4$ and $y_i$ could be $3$, ...
0
votes
0answers
16 views

Why it is reshaped the last layers of VGG_UNet segmentation model?

I want to do a multiclass segmentation task using deep learning (in python). Here, is a summary of vgg_unet model that is mainly collected from GitHub. So, in my dataset 8 labels are available. So, at ...
1
vote
1answer
26 views

Does a better discriminator in GANs mean better sample generation by the generator?

Since the discriminator defines how the generator is updated, then building a discriminator with a higher number of parameters/more layers should lead to a better quality of generated samples. So, ...
0
votes
0answers
24 views

Does a constant low validation loss mean great model accuracy?

This is more of a conceptual question that I am asking here. I am using an encoder-decoder model with Bidirectional LSTMs used as a time series regression problem. I train it with a sufficiently low ...
2
votes
2answers
37 views

How to take the optimal batch_size for training a model?

I have an image dataset, which is composed of 113695 images for training and 28424 images for validation. Now, when I use ImageDataGenerator and ...
1
vote
1answer
67 views

How to determine the number of hidden layers and units of a deep auto-encoder?

I am using a deep autoencoder for my problem. However, the way I choose the number of hidden layers and hidden units in a hidden layer is still based on my feeling. The size of the model that ...
0
votes
0answers
10 views

Generative Adversarial Network with two images in input

I am doing an internship project regarding deep learning, and it is a totally new topic for me as I have never studied machine learning in the bachelor's degree courses. I have to implement a GAN that ...
3
votes
1answer
176 views

How should I choose the target's update frequency in DQN?

I have been dealing with a problem that I'm trying to solve with DQN. A general question that I have is regarding the target's update frequency. How should it change? Depending on what factor do we ...
0
votes
0answers
32 views

Why does my loss value of autoencoder in PyTorch is negative?

I am trying to implement SDNE, a algorithm uses deep auto encoder to map a graph to latent representation d dimension. The idea is kind of simple, SDNE uses the ...
0
votes
1answer
36 views

How can I implement 2D CNN filter with channelwise-bound kernel weights?

I would like to bind kernel parameters through channels/feature-maps for each filter. In a conv2d operation, each filter consists of HxWxC parameters I would like to have filters that have HxW ...
2
votes
1answer
148 views

What is the scope of real-world deep learning applications in 2020?

2015 was a milestone year for AI--"deep learning" was validated in a very public way with AlphaGo. However, at the time, the question was raised: "What else is deep learning good for?&...
0
votes
0answers
16 views

Transfer Learning: Finetune a model with a splitted dataset?

Lets say I want to fine-tune a model. I have a pretrained ResNet model and on top of this model I add some extra layers. And lets say I have a dataset of 10,000 images. The recommended way would be: ...
1
vote
1answer
60 views

In DQN, when do the parameters in the Neural Network update based on the reward received?

I'm aware that we back-propagate after computing the loss between: The Neural Network Q values and the Target Network Q values However, all this is doing is updating the parameters of the Neural ...
1
vote
1answer
44 views

Why are Target Networks used in Deep Q-Learning as opposed to the Expected Value equation?

I understand we use a target network because it helps resolve issues regarding stability, however, that's not what I'm here to ask. What I would like to understand is why a target network is used as a ...
0
votes
1answer
73 views

What is Precision@K for link prediction in graph embedding meaning?

I am trying to re-implement the SDNE algorithm for graph embedding by PyTorch. I get stuck at some issues about evaluation metric Precision@K. precision@k is a metric which gives equal weight to the ...
1
vote
1answer
47 views

What is the representational capacity of a learning algorithm? [duplicate]

The definition I see for representational capacity is "the family of functions the learning algorithm can choose from when varying the parameters in order to reduce a training objective." (...
0
votes
0answers
15 views

How does YOLO handle non-class objects?

I have been reading more about computer vision and I'm bothered by YOLO and similar deep learning architectures. The thing I am confused on is how non-class image sections are dealt with, in ...
1
vote
0answers
14 views

Understanding graphs of the mean square error: relationships between val loss and train loss

I am currently working with some models aimed at predicting time series (89 days for training, 22 for testing), including a CNN LSTM and a convLSTM. When training these models, I had the following ...
1
vote
1answer
895 views

What is the computational complexity of the forward pass of a convolutional neural network?

How do I determine the computational complexity (big-O notation) of the forward pass of a convolutional neural network? Let's assume for simplicity that we use zero-padding such that the input size ...
1
vote
0answers
70 views

What is the state-of-the-art algorithm for neural style transfer?

I've read the paper A Neural Algorithm of Artistic Style by Gatys et. al. and I find the application of neural style transfer very fun. I also read that Exploring the structure of a real-time, ...
3
votes
0answers
41 views

Would a different learning rate for every neuron and layer mitigate or solve the vanishing gradient problem?

I'm interested in using the sigmoid (or tanh) activation function instead of RELU. I'm aware of RELU advantages on faster computation and no vanishing gradient problem. But about vanishing gradient, ...
1
vote
0answers
31 views

Keras model accuracy not improving beyond threshold

I am currently working on a public project for the National Weather Model. We are experimenting with using a recurrent neural network to replace the output of a quadratic formula that is in use. The ...
1
vote
0answers
31 views

How to add voice inflections to an existing SV2TTS voice cloning implementation?

I've been playing with the SV2TTS voice cloning implementation provided on Github: https://github.com/CorentinJ/Real-Time-Voice-Cloning It was straight forward to use as a TTS engine with clearly ...
1
vote
1answer
258 views

How to generate labels for self-supervised training?

I've been reading a lot lately about self-supervised learning and I didn't understand very well how to generate the desired label for a given image. Let's say that I have an image classification task, ...
3
votes
0answers
61 views

How can I find a specific word in an audio file?

I'm trying to train and use a neural network to detect a specific word in an audio file. The input of the neural network is an audio of 2-3 seconds duration, and the neural network must determine ...
1
vote
0answers
53 views

How to make sense of label propagation formula in graph neural networks?

In the label propagation algorithm in section 3.2.3, we know the label of some nodes and we want to predict the label for the rest of the nodes whose labels we don't know. The update formula for this ...
0
votes
0answers
20 views

What are some solutions for dealing with time series data that are recorded at uneven intervals?

Let's say I have a time series data which is a bunch of observations that occur at different time stamps and intervals. For example, my observations come from a camera located at a traffic ...
0
votes
0answers
27 views

Is Bayesian NN vs adding random data more accurate?

I’m trying to train a classifier to recognize if people are wearing seatbelts. What if the person submitted a picture unrelated to a seatbelt classifier? Would I create an image label that is full of ...
0
votes
0answers
34 views

Why do we use the Target Network for action evaluation in Double deep Q networks

Is there any specific reason as to why The target Network is used for evaluation and The online network Is used for selection, what would be the difference if both roles were switched, our online ...
4
votes
1answer
69 views

Can AlphaZero considered as Multi-Agent Deep Reinforcement Learning?

Can AlphaZero considered as Multi-Agent Deep Reinforcement Learning? I could not find a clear answer on this. I would say yes it is Multi Agent Learning, as there are two Agents playing against each ...
3
votes
1answer
88 views

How to use text as an input for a neural network - regression problem? How many likes/claps an article will get

I am trying to predict the number of likes an article or a post will get using a NN. I have a dataframe with ~70,000 rows and 2 columns: "text" (predictor - strings of text) and "likes&...
0
votes
0answers
38 views

Having trouble understanding how Double deep Q networks work

I’ve looked at various articles and I’m still very confused, I understand the normal double Q learning about having two Action value estimates that use two different set of samples But coming to ...
0
votes
2answers
54 views

How to handle images that don’t pertain to image classifier at all?

I am trying to create a CNN model that classifies if a person is wearing a seatbelt or not to verify they drive safely. I know to get images of people wearing seatbelts and people not wearing ...
2
votes
1answer
45 views

The are some fundamental learning theories for developing an AI that imitates human behavior

Most if not all AI systems are not to imitate human, but to finally out-perform human. Examples include using AI to play a game, classification problems, auto-driving, and goal-oriented chatbots. ...
0
votes
0answers
37 views

What is the reason for implementing position-wise Feed Forward Network in Transformer?

Does anyone know the reason why there's a FCNN after the self-attention layer? Or at least some intuition for it?
2
votes
1answer
103 views

Is gradient descent scale invariant or not?

I know we should scale the input and output (assuming regression task) before we feed it to the neural network. Then the gradient descent will give the better minima much faster. But I have subtle ...
0
votes
0answers
35 views

Is it a good idea to train a neural network to classify images without base-hypothesis?

I'm a relative beginner in deep-learning (understand by that, I'm doing my first kaggle competition right now, and I have loads to learn still) and I was just wondering something. Let's say you have ...
1
vote
1answer
86 views

Who first coined the term “deep learning”?

AFAIK, deep learning became popular in 2012 with the victory of ImageNet Competition - Large Scale Visual Recognition Challenge 2012 where winners of this contest actually used deep learning ...
0
votes
1answer
88 views

How to train an LSTM with varying length input?

I have a dataset where each of the training instances is different in the length and the data is sequential. So, I design an LSTM but I am thinking about how to train the LSTM. In fixed-length data, ...
2
votes
0answers
89 views

What is the memory complexity of the memory-efficient attention in Reformer?

When I read the paper, Reformer: The Efficient Transformer, I cannot get the same complexity of the memory-efficient method in Table 1 (p. 5), which summarizes time/memory complexity of scaled dot-...
2
votes
1answer
82 views

How large should the corpus be to optimally retrain the GPT-2 model?

I just started working with the GPT-2 models and want to retrain one on a pretty narrow topic, so I have problems finding training material. How large should the corpus be to optimally retrain the GPT-...
1
vote
1answer
70 views

What is the difference between artificial neural network (ANN) and deep learning?

I have read many mixed definitions around these two terms. For example, is it right to say deep learning is any ANN with more than two hidden layers? What are formal definitions for these two?
1
vote
0answers
80 views

Combining clustering and deep learning for computer vision

Is there any recent work on combining clustering approaches (k-means, or gaussian mixture or PGM) with deep learning for computer vision? In particular I'm interested in if anyone has used the first ...
2
votes
1answer
93 views

Why L2 loss is more commonly used in Neural Networks than other loss functions?

Why L2 loss is more commonly used in Neural Networks than other loss functions? What is the reason to L2 being a default choice in Neural Networks?
1
vote
2answers
65 views

In what situations ELUs should be used instead of RELUs?

I always use RELUs actication functions when I need to and I understand limitations of ELUs. So in what situation do I need to consider ELUs over RELUs?
1
vote
0answers
46 views

Reference for Transfer Learning via Final Layers of a Neural Network

Problem (Sketch): I'm interested in a particular formulation of the transfer-learning problem, which, given a trained network $f$ seeks to learn a new network $g$ whose last few layers behave very ...
0
votes
1answer
54 views

Why can we use a network to estimate $Q_\pi(s, a)$ in Actor-Critic Method?

According to deep Q learning, we want to learn $Q^*(s,a)$, which is the optimal action-value function. It does make sense because we assume there is only one optimal function so the algorithm will ...

1
3 4
5
6 7
29