Questions tagged [deep-learning]

For questions related to deep learning, which refers to a subset of machine learning methods based on artificial neural networks (ANNs) with multiple hidden layers. The adjective deep thus refers to the number of layers of the ANNs. The expression deep learning was apparently introduced (although not in the context of machine learning or ANNs) in 1986 by Rina Dechter in the paper "Learning while searching in constraint-satisfaction-problems".

Filter by
Sorted by
Tagged with
1
vote
0answers
5 views

Computational complexity of the forward pass of a CONVOLUTIONAL neural network?

How do I determine the computational complexity (big-O notation) of the forward pass of a CONVOLUTIONAL neural network? Let's assume for simplicity that we use zero-padding such that the input size ...
0
votes
0answers
15 views

What is the state-of-the-art algorithm for neural style transfer?

I've read the paper A Neural Algorithm of Artistic Style by Gatys et. al. and I find the application of neural style transfer very fun. I also read that Exploring the structure of a real-time, ...
2
votes
0answers
27 views

Would a different learning rate for every neuron and layer mitigate or solve the vanishing gradient problem?

I'm interested in using the sigmoid (or tanh) activation function instead of RELU. I'm aware of RELU advantages on faster computation and no vanishing gradient problem. But about vanishing gradient, ...
1
vote
0answers
13 views

Keras model accuracy not improving beyond threshold

I am currently working on a public project for the National Weather Model. We are experimenting with using a recurrent neural network to replace the output of a quadratic formula that is in use. The ...
0
votes
0answers
13 views

How to add voice inflections to an existing SV2TTS voice cloning implementation?

I've been playing with the SV2TTS voice cloning implementation provided on Github: https://github.com/CorentinJ/Real-Time-Voice-Cloning It was straight forward to use as a TTS engine with clearly ...
1
vote
1answer
32 views

How to generate labels for self-supervised training?

I've been reading a lot lately about self-supervised learning and I didn't understand very well how to generate the desired label for a given image. Let's say that I have an image classification task, ...
2
votes
0answers
55 views

How can I find a specific word in an audio file?

I'm trying to train and use a neural network to detect a specific word in an audio file. The input of the neural network is an audio of 2-3 seconds duration, and the neural network must determine ...
1
vote
0answers
46 views

How to make sense of label propagation formula in graph neural networks?

In the label propagation algorithm in section 3.2.3, we know the label of some nodes and we want to predict the label for the rest of the nodes whose labels we don't know. The update formula for this ...
0
votes
0answers
18 views

What are some solutions for dealing with time series data that are recorded at uneven intervals?

Let's say I have a time series data which is a bunch of observations that occur at different time stamps and intervals. For example, my observations come from a camera located at a traffic ...
0
votes
0answers
22 views

Is Bayesian NN vs adding random data more accurate?

I’m trying to train a classifier to recognize if people are wearing seatbelts. What if the person submitted a picture unrelated to a seatbelt classifier? Would I create an image label that is full of ...
0
votes
0answers
29 views

Why do we use the Target Network for action evaluation in Double deep Q networks

Is there any specific reason as to why The target Network is used for evaluation and The online network Is used for selection, what would be the difference if both roles were switched, our online ...
3
votes
1answer
36 views

Can AlphaZero considered as Multi-Agent Deep Reinforcement Learning?

Can AlphaZero considered as Multi-Agent Deep Reinforcement Learning? I could not find a clear answer on this. I would say yes it is Multi Agent Learning, as there are two Agents playing against each ...
2
votes
1answer
53 views

How to use text as an input for a neural network - regression problem? How many likes/claps an article will get

I am trying to predict the number of likes an article or a post will get using a NN. I have a dataframe with ~70,000 rows and 2 columns: "text" (predictor - strings of text) and "likes&...
0
votes
0answers
36 views

Having trouble understanding how Double deep Q networks work

I’ve looked at various articles and I’m still very confused, I understand the normal double Q learning about having two Action value estimates that use two different set of samples But coming to ...
0
votes
1answer
32 views

If I'm trying to build a binary image classifier, how do I handle images that aren't related to the classifier?

I am trying to create an image classifier to classify whether a person is wearing a seatbelt or not. Now, say a person tries to put through an image of a frog (random). How should I handle the image ...
0
votes
1answer
39 views

How to handle images that don’t pertain to image classifier at all?

I am trying to create a CNN model that classifies if a person is wearing a seatbelt or not to verify they drive safely. I know to get images of people wearing seatbelts and people not wearing ...
1
vote
1answer
29 views

The are some fundamental learning theories for developing an AI that imitates human behavior

Most if not all AI systems are not to imitate human, but to finally out-perform human. Examples include using AI to play a game, classification problems, auto-driving, and goal-oriented chatbots. ...
0
votes
0answers
21 views

What is the reason for implementing position-wise Feed Forward Network in Transformer?

Does anyone know the reason why there's a FCNN after the self-attention layer? Or at least some intuition for it?
0
votes
1answer
39 views

Is gradient descent scale invariant or not?

I know we should scale the input and output (assuming regression task) before we feed it to the neural network. Then the gradient descent will give the better minima much faster. But I have subtle ...
0
votes
0answers
33 views

Is it a good idea to train a neural network to classify images without base-hypothesis?

I'm a relative beginner in deep-learning (understand by that, I'm doing my first kaggle competition right now, and I have loads to learn still) and I was just wondering something. Let's say you have ...
1
vote
1answer
60 views

Who first coined the term “deep learning”?

AFAIK, deep learning became popular in 2012 with the victory of ImageNet Competition - Large Scale Visual Recognition Challenge 2012 where winners of this contest actually used deep learning ...
0
votes
1answer
66 views

How to train an LSTM with varying length input?

I have a dataset where each of the training instances is different in the length and the data is sequential. So, I design an LSTM but I am thinking about how to train the LSTM. In fixed-length data, ...
2
votes
0answers
73 views

What is the memory complexity of the memory-efficient attention in Reformer?

When I read the paper, Reformer: The Efficient Transformer, I cannot get the same complexity of the memory-efficient method in Table 1 (p. 5), which summarizes time/memory complexity of scaled dot-...
1
vote
0answers
53 views

How large should the corpus be to optimally retrain the GPT-2 model?

I just started working with the GPT-2 models and want to retrain one on a pretty narrow topic, so I have problems finding training material. How large should the corpus be to optimally retrain the GPT-...
1
vote
1answer
68 views

What is the difference between artificial neural network (ANN) and deep learning?

I have read many mixed definitions around these two terms. For example, is it right to say deep learning is any ANN with more than two hidden layers? What are formal definitions for these two?
1
vote
0answers
79 views

Combining clustering and deep learning for computer vision

Is there any recent work on combining clustering approaches (k-means, or gaussian mixture or PGM) with deep learning for computer vision? In particular I'm interested in if anyone has used the first ...
2
votes
1answer
83 views

Why L2 loss is more commonly used in Neural Networks than other loss functions?

Why L2 loss is more commonly used in Neural Networks than other loss functions? What is the reason to L2 being a default choice in Neural Networks?
1
vote
2answers
62 views

In what situations ELUs should be used instead of RELUs?

I always use RELUs actication functions when I need to and I understand limitations of ELUs. So in what situation do I need to consider ELUs over RELUs?
1
vote
0answers
45 views

Reference for Transfer Learning via Final Layers of a Neural Network

Problem (Sketch): I'm interested in a particular formulation of the transfer-learning problem, which, given a trained network $f$ seeks to learn a new network $g$ whose last few layers behave very ...
0
votes
1answer
47 views

Why can we use a network to estimate $Q_\pi(s, a)$ in Actor-Critic Method?

According to deep Q learning, we want to learn $Q^*(s,a)$, which is the optimal action-value function. It does make sense because we assume there is only one optimal function so the algorithm will ...
3
votes
1answer
75 views

What activation functions are currently popular?

I am not asking what activation function is better. I want to know what activation functions are more used in research or deployment. Also, are they used in combination? e.g. ReLU, ELUs, etc. I'd ...
1
vote
0answers
39 views

What is a multi channel supervised classifier?

I came across a paper that describes its model architecture in the following way. Our TRIL network is a two-channel network jointly trained to predict the expert’s action given state and the system’s ...
0
votes
0answers
43 views

Possible reasons that validation recall is fluctuating across different epochs but the precision is stable?

I'm training a deep learning model. After each epoch I measure the performance of the model on validation set. Here is how the performance looks like while training: It's a binary classification task ...
2
votes
1answer
52 views

How are training hyperparameters determined for large models?

When training relatively small DL model which takes several hours to train, I typically start with some starting points from literature and then use trial-and-error or grid-search approach to tune up ...
2
votes
0answers
46 views

What is meant by “arranging the final features of CNN in a grid” and how to do it?

In the paper What You Get Is What You See: A Visual Markup Decompiler, the authors have proposed a method to extract the features from the CNN and then arrange those extracted features in a grid to ...
1
vote
0answers
51 views

Can I think graph convolution as 2D convolution like images?

Kipf et al described in his paper that we can write graph convolution operation like this: $$H_{t+1} = AH_tW_t$$ where, $A$ is the normalized adjacency matrix, $H_t$ is the embedded representation of ...
1
vote
0answers
47 views

Which neural network should I use to distinguish between different types of defects?

I want to teach a neural network to distinguish between different types of defects. For that, I generated images of fake-defects. The images of the fake-defect types are attached. I tried many ...
0
votes
0answers
31 views

What could be the possible strategy and Deep Learning method that MathPix might be using for LaTex detection?

I want to build an open Source OCR just like MathPix. There is already a model to extract LaTex from the image by Harverd NLP's im2markup but the problem is that their data has been trained and tested ...
3
votes
2answers
69 views

How can I predict the true label for data with incomplete features based on the trained model with data with more features?

Suppose I have a model that was trained with a dataset that contains the features (f1, f2, f3, f4, f5, f6). However, my test dataset does not contain all features ...
0
votes
0answers
50 views

Can I train a neural network with multiple datasets (e.g. 25)?

I want to create a neural network that I can train with many datasets (e.g. 20 - 25 datasets). Can I use transfer learning for this? Or is there a better approach than this?
5
votes
1answer
83 views

Can GANs be used to generate something other than images?

AFAIK, GANs are used for generating/synthesizing near-perfect human faces (deepfakes), gallery arts, etc., but can GANs be used to generate something other than images?
1
vote
1answer
62 views

What should the output of a neural network that needs to classify in an unsupervised fashion XOR data be?

XOR data, without labels: [[0,0],[0,1],[1,0],[1,1]] I'm using this network for auto-classifying XOR data: ...
2
votes
1answer
77 views

Are tabular reinforcement learning methods obsolete (or getting obsolete)?

While learning RL, I came across some problems where the Q-matrix that I need to make is very very large. I am not sure if it is ever practical. Then I research and came to this conclusion that using ...
0
votes
0answers
32 views

Suggestion for an Enterprise search engine: What are all the way I can use machine learning/dnn/nlp to enhance elasticsearch results?

We are building an enterprise search engine for about 130,000 documents. It includes FAQs, Small articles to 3-4 page documents, all of them are stored in structured JSON. We were originally planning ...
2
votes
1answer
59 views

How does the target network in double DQNs find the maximum Q* value for each action?

I understand the fact that the neural network is used to take the states as inputs and it outputs the Q-value for state-action pairs. However, in order to compute this and update its weights, we need ...
0
votes
1answer
41 views

How do we make our outputs to have the same size as the true mask?

When we are doing multi-label segmentation tasks, our y_true (the mask) will be (w, h, 3), but, in our model, at the last layer, ...
0
votes
0answers
44 views

Predict probability of user making a conversion

My dear friends, In the past couple of years I read a lot about AI with JS and some libraries like TensorFlow. I have great interest in the subject but never used it on a serious project. However, ...
0
votes
0answers
37 views

Using a neural network in a microcontroller to recognize air-written letters and numbers

Recently I finished a project which combined stm32f030r8 microcontroller and MPU9250 sensor to create a system which would detect orientation on all 3 axis using a combination of accel data, gyro data ...
0
votes
1answer
39 views

Is there a way to get landmark features automatically learned by a neural network?

Is there a way to get landmark features automatically learned by a neural network without having to manually pre-label them in the images that are being fed into the network?
1
vote
1answer
44 views

Why would the learning rate curve go backwards?

I'm working on recognizing the numbers 3 and 7 using the MNIST data set. I'm using cnn_learner() function from fastai library. When I plotted the learning rate, the ...

1
2 3 4 5
26