Questions tagged [deep-learning]

For questions related to deep learning, which refers to a subset of machine learning methods based on artificial neural networks (ANNs) with multiple hidden layers. The adjective deep thus refers to the number of layers of the ANNs. The expression deep learning was apparently introduced (although not in the context of machine learning or ANNs) in 1986 by Rina Dechter in the paper "Learning while searching in constraint-satisfaction-problems".

Filter by
Sorted by
Tagged with
3
votes
1answer
231 views

How should I choose the target's update frequency in DQN?

I have been dealing with a problem that I'm trying to solve with DQN. A general question that I have is regarding the target's update frequency. How should it change? Depending on what factor do we ...
0
votes
0answers
37 views

Why does my loss value of autoencoder in PyTorch is negative?

I am trying to implement SDNE, a algorithm uses deep auto encoder to map a graph to latent representation d dimension. The idea is kind of simple, SDNE uses the ...
0
votes
1answer
37 views

How can I implement 2D CNN filter with channelwise-bound kernel weights?

I would like to bind kernel parameters through channels/feature-maps for each filter. In a conv2d operation, each filter consists of HxWxC parameters I would like to have filters that have HxW ...
2
votes
1answer
149 views

What is the scope of real-world deep learning applications in 2020?

2015 was a milestone year for AI--"deep learning" was validated in a very public way with AlphaGo. However, at the time, the question was raised: "What else is deep learning good for?&...
0
votes
0answers
16 views

Transfer Learning: Finetune a model with a splitted dataset?

Lets say I want to fine-tune a model. I have a pretrained ResNet model and on top of this model I add some extra layers. And lets say I have a dataset of 10,000 images. The recommended way would be: ...
1
vote
1answer
70 views

In DQN, when do the parameters in the Neural Network update based on the reward received?

I'm aware that we back-propagate after computing the loss between: The Neural Network Q values and the Target Network Q values However, all this is doing is updating the parameters of the Neural ...
1
vote
1answer
51 views

Why are Target Networks used in Deep Q-Learning as opposed to the Expected Value equation?

I understand we use a target network because it helps resolve issues regarding stability, however, that's not what I'm here to ask. What I would like to understand is why a target network is used as a ...
0
votes
1answer
90 views

What is Precision@K for link prediction in graph embedding meaning?

I am trying to re-implement the SDNE algorithm for graph embedding by PyTorch. I get stuck at some issues about evaluation metric Precision@K. precision@k is a metric which gives equal weight to the ...
1
vote
1answer
48 views

What is the representational capacity of a learning algorithm? [duplicate]

The definition I see for representational capacity is "the family of functions the learning algorithm can choose from when varying the parameters in order to reduce a training objective." (...
2
votes
1answer
38 views

How does YOLO handle non-class objects?

I have been reading more about computer vision and I'm bothered by YOLO and similar deep learning architectures. The thing I am confused about is how non-class image sections are dealt with. In ...
1
vote
0answers
14 views

Understanding graphs of the mean square error: relationships between val loss and train loss

I am currently working with some models aimed at predicting time series (89 days for training, 22 for testing), including a CNN LSTM and a convLSTM. When training these models, I had the following ...
1
vote
1answer
1k views

What is the computational complexity of the forward pass of a convolutional neural network?

How do I determine the computational complexity (big-O notation) of the forward pass of a convolutional neural network? Let's assume for simplicity that we use zero-padding such that the input size ...
0
votes
1answer
108 views

What is the state-of-the-art algorithm for neural style transfer?

I've read the paper A Neural Algorithm of Artistic Style by Gatys et. al. and I find the application of neural style transfer very fun. I also read that Exploring the structure of a real-time, ...
3
votes
0answers
41 views

Would a different learning rate for every neuron and layer mitigate or solve the vanishing gradient problem?

I'm interested in using the sigmoid (or tanh) activation function instead of RELU. I'm aware of RELU advantages on faster computation and no vanishing gradient problem. But about vanishing gradient, ...
1
vote
0answers
39 views

Keras model accuracy not improving beyond threshold

I am currently working on a public project for the National Weather Model. We are experimenting with using a recurrent neural network to replace the output of a quadratic formula that is in use. The ...
0
votes
0answers
37 views

How to add voice inflections to an existing SV2TTS voice cloning implementation?

I've been playing with the SV2TTS voice cloning implementation provided on Github: https://github.com/CorentinJ/Real-Time-Voice-Cloning It was straight forward to use as a TTS engine with clearly ...
1
vote
1answer
303 views

How to generate labels for self-supervised training?

I've been reading a lot lately about self-supervised learning and I didn't understand very well how to generate the desired label for a given image. Let's say that I have an image classification task, ...
3
votes
1answer
77 views

How can I find a specific word in an audio file?

I'm trying to train and use a neural network to detect a specific word in an audio file. The input of the neural network is an audio of 2-3 seconds duration, and the neural network must determine ...
1
vote
0answers
53 views

How to make sense of label propagation formula in graph neural networks?

In the label propagation algorithm in section 3.2.3, we know the label of some nodes and we want to predict the label for the rest of the nodes whose labels we don't know. The update formula for this ...
0
votes
0answers
21 views

What are some solutions for dealing with time series data that are recorded at uneven intervals?

Let's say I have a time series data which is a bunch of observations that occur at different time stamps and intervals. For example, my observations come from a camera located at a traffic ...
0
votes
0answers
27 views

Is Bayesian NN vs adding random data more accurate?

I’m trying to train a classifier to recognize if people are wearing seatbelts. What if the person submitted a picture unrelated to a seatbelt classifier? Would I create an image label that is full of ...
0
votes
0answers
35 views

Why do we use the Target Network for action evaluation in Double deep Q networks

Is there any specific reason as to why The target Network is used for evaluation and The online network Is used for selection, what would be the difference if both roles were switched, our online ...
3
votes
1answer
77 views

Can AlphaZero considered as Multi-Agent Deep Reinforcement Learning?

Can AlphaZero considered as Multi-Agent Deep Reinforcement Learning? I could not find a clear answer on this. I would say yes it is Multi Agent Learning, as there are two Agents playing against each ...
3
votes
1answer
89 views

How to use text as an input for a neural network - regression problem? How many likes/claps an article will get

I am trying to predict the number of likes an article or a post will get using a NN. I have a dataframe with ~70,000 rows and 2 columns: "text" (predictor - strings of text) and "likes&...
0
votes
0answers
38 views

Having trouble understanding how Double deep Q networks work

I’ve looked at various articles and I’m still very confused, I understand the normal double Q learning about having two Action value estimates that use two different set of samples But coming to ...
0
votes
2answers
55 views

How to handle images that don’t pertain to image classifier at all?

I am trying to create a CNN model that classifies if a person is wearing a seatbelt or not to verify they drive safely. I know to get images of people wearing seatbelts and people not wearing ...
1
vote
1answer
45 views

The are some fundamental learning theories for developing an AI that imitates human behavior

Most if not all AI systems are not to imitate human, but to finally out-perform human. Examples include using AI to play a game, classification problems, auto-driving, and goal-oriented chatbots. ...
0
votes
0answers
41 views

What is the reason for implementing position-wise Feed Forward Network in Transformer?

Does anyone know the reason why there's a FCNN after the self-attention layer? Or at least some intuition for it?
1
vote
1answer
109 views

Is gradient descent scale invariant or not?

I know we should scale the input and output (assuming regression task) before we feed it to the neural network. Then the gradient descent will give the better minima much faster. But I have subtle ...
0
votes
0answers
35 views

Is it a good idea to train a neural network to classify images without base-hypothesis?

I'm a relative beginner in deep-learning (understand by that, I'm doing my first kaggle competition right now, and I have loads to learn still) and I was just wondering something. Let's say you have ...
1
vote
1answer
95 views

Who first coined the term “deep learning”?

AFAIK, deep learning became popular in 2012 with the victory of ImageNet Competition - Large Scale Visual Recognition Challenge 2012 where winners of this contest actually used deep learning ...
0
votes
1answer
91 views

How to train an LSTM with varying length input?

I have a dataset where each of the training instances is different in the length and the data is sequential. So, I design an LSTM but I am thinking about how to train the LSTM. In fixed-length data, ...
2
votes
0answers
91 views

What is the memory complexity of the memory-efficient attention in Reformer?

When I read the paper, Reformer: The Efficient Transformer, I cannot get the same complexity of the memory-efficient method in Table 1 (p. 5), which summarizes time/memory complexity of scaled dot-...
1
vote
1answer
93 views

How large should the corpus be to optimally retrain the GPT-2 model?

I just started working with the GPT-2 models and want to retrain one on a pretty narrow topic, so I have problems finding training material. How large should the corpus be to optimally retrain the GPT-...
1
vote
1answer
80 views

What is the difference between artificial neural network (ANN) and deep learning?

I have read many mixed definitions around these two terms. For example, is it right to say deep learning is any ANN with more than two hidden layers? What are formal definitions for these two?
1
vote
0answers
80 views

Combining clustering and deep learning for computer vision

Is there any recent work on combining clustering approaches (k-means, or gaussian mixture or PGM) with deep learning for computer vision? In particular I'm interested in if anyone has used the first ...
1
vote
1answer
96 views

Why L2 loss is more commonly used in Neural Networks than other loss functions?

Why L2 loss is more commonly used in Neural Networks than other loss functions? What is the reason to L2 being a default choice in Neural Networks?
1
vote
2answers
65 views

In what situations ELUs should be used instead of RELUs?

I always use RELUs actication functions when I need to and I understand limitations of ELUs. So in what situation do I need to consider ELUs over RELUs?
1
vote
0answers
47 views

Reference for Transfer Learning via Final Layers of a Neural Network

Problem (Sketch): I'm interested in a particular formulation of the transfer-learning problem, which, given a trained network $f$ seeks to learn a new network $g$ whose last few layers behave very ...
0
votes
1answer
56 views

Why can we use a network to estimate $Q_\pi(s, a)$ in Actor-Critic Method?

According to deep Q learning, we want to learn $Q^*(s,a)$, which is the optimal action-value function. It does make sense because we assume there is only one optimal function so the algorithm will ...
4
votes
1answer
85 views

What activation functions are currently popular?

I am not asking what activation function is better. I want to know what activation functions are more used in research or deployment. Also, are they used in combination? e.g. ReLU, ELUs, etc. I'd ...
1
vote
0answers
40 views

What is a multi channel supervised classifier?

I came across a paper that describes its model architecture in the following way. Our TRIL network is a two-channel network jointly trained to predict the expert’s action given state and the system’s ...
0
votes
0answers
45 views

Possible reasons that validation recall is fluctuating across different epochs but the precision is stable?

I'm training a deep learning model. After each epoch I measure the performance of the model on validation set. Here is how the performance looks like while training: It's a binary classification task ...
3
votes
1answer
61 views

How are training hyperparameters determined for large models?

When training a relatively small DL model, which takes several hours to train, I typically start with some starting points from literature and then use a trial-and-error or grid-search approach to ...
2
votes
0answers
49 views

What is meant by “arranging the final features of CNN in a grid” and how to do it?

In the paper What You Get Is What You See: A Visual Markup Decompiler, the authors have proposed a method to extract the features from the CNN and then arrange those extracted features in a grid to ...
1
vote
0answers
67 views

Can I think graph convolution as 2D convolution like images?

Kipf et al described in his paper that we can write graph convolution operation like this: $$H_{t+1} = AH_tW_t$$ where, $A$ is the normalized adjacency matrix, $H_t$ is the embedded representation of ...
1
vote
0answers
49 views

Which neural network should I use to distinguish between different types of defects?

I want to teach a neural network to distinguish between different types of defects. For that, I generated images of fake-defects. The images of the fake-defect types are attached. I tried many ...
0
votes
0answers
59 views

What could be the possible strategy and Deep Learning method that MathPix might be using for LaTex detection?

I want to build an open Source OCR just like MathPix. There is already a model to extract LaTex from the image by Harverd NLP's im2markup but the problem is that their data has been trained and tested ...
3
votes
1answer
197 views

Why does every neuron in hidden layers of a multi-layer perceptron typically have the same activation function? [duplicate]

Why does every neuron in a hidden layer of a multi-layer perceptron (MLP) typically have the same activation function as every other neuron in the same or other hidden layers (so I exclude the output ...
3
votes
2answers
75 views

How can I predict the true label for data with incomplete features based on the trained model with data with more features?

Suppose I have a model that was trained with a dataset that contains the features (f1, f2, f3, f4, f5, f6). However, my test dataset does not contain all features ...

1
4 5
6
7 8
29