Questions tagged [deep-learning]

For questions related to deep learning, which refers to a subset of machine learning methods based on artificial neural networks (ANNs) with multiple hidden layers. The adjective deep thus refers to the number of layers of the ANNs. The expression deep learning was apparently introduced (although not in the context of machine learning or ANNs) in 1986 by Rina Dechter in the paper "Learning while searching in constraint-satisfaction-problems".

Filter by
Sorted by
Tagged with
0
votes
1answer
27 views

How to split data into training validation and test set when the number of data in classes varies greatly?

I have 5 classes of pictures to classify: 0 -> ~3 200 (~800 initial number before interference and duplication) 1 -> ~9 000 (I reduced from ~90 000) 2 -> ~8 000 3 -> ~3 000 4 -> ~7 200 How to ...
1
vote
1answer
28 views

Is it harmful to set the learning rate of training a model to be too high if there is some decay function for the learning rate?

It is known that if $\alpha$ is set to high, then the cost function of the model may not converge. However, would a decaying of the learning rate provide some "tuning" of the $\alpha$ value during ...
1
vote
1answer
79 views

What are some applications of deep learning?

Deep-learning could be performed in three varieties supervised unsupervised reinforcement learning Deep learning in that matter also can be referred to as structured learning since deep learning is ...
0
votes
0answers
16 views

Separated LSTMs or a global one for cluster of related features

I have an $n$-dimensional time-series to apply LSTM to, $n$ is the number of features for each time point. These features can be clustered according to their concept, for example $n_1, ..., n_4$ are ...
1
vote
0answers
31 views

Overcome caveats on using Deep Learning for faster inference on limited performance availability

I am working in the field of Machine Vision, where accuracy and performance both play a major factor in deciding the approach towards a problem. Traditional rule based approaches work quite well in ...
2
votes
1answer
58 views

What is the difference between LSTM and fully connected LSTM?

I'm currently trying to understand the difference between a vanilla LSTM and a fully connected LSTM. In a paper I'm reading, the FC-LSTM gets introduced as FC-LSTM may be seen as a multivariate ...
3
votes
1answer
112 views

How do deepfakes work and how they might be dangerous?

Deepfakes (a portmanteau of "deep learning" and "fake") are synthetic media in which a person in an existing image or video is replaced with someone else's likeness. Nowadays most of the news ...
0
votes
1answer
33 views

Can a machine learning approach solve this constrained optimisation problem?

I had done with different classification, regression and clustering approaches for predictions of values, etc. I was wondering if there is a machine learning approach for distribution of a whole based ...
1
vote
0answers
37 views

How to compare SegNet, U-Net and EfficientNet?

SegNet and U-Net are created for segmentation problem and EfficientNet is created for classification problem. I have a task and it is saying that train these models on the same dataset and compare ...
0
votes
0answers
16 views

Studying the speech-generation model and have question about the confusing nature of model input and outputs

I am currently studying this model speech generation known as WaveNet model by Google. https://arxiv.org/pdf/1609.03499.pdf using the linked original paper and this implementation. I find the model ...
0
votes
0answers
62 views

TD-Leaf struggles at learning chess

I am currently working on implementing Giraffe chess algorithm. Following this paper, I designed a neural network similar to the one proposed by the author which I trained using TD-Leaf(lambda). The ...
1
vote
0answers
37 views

What is the reason for different learned features in upper and lower half in AlexNet?

I was reading AlexNet paper and the authors quoted the kernels on one GPU were "largely color agnostic," whereas the kernels on the other GPU were largely "color-specific." The upper GPU takes ...
1
vote
1answer
54 views

Can we use a pre trained Encoder (BERT, XLM ) with a Decoder (GPT, Transformer-XL) to build a Chatbot instead of Language Translation?

I was wondering if the BART or T5 models can do the task of generating sentences in English. Most of the models I have mentioned ...
0
votes
0answers
34 views

How GAN generator produce integer RGB colored picture?

For traditional neural networks, I know that we can't constraint the output to be strict integers. My question is what technique does GANs use to produce integer outputs, that can be then converted to ...
1
vote
0answers
40 views

Do you have to add a dense layer onto the final layer of an LSTM?

If my understanding of an LSTM is correct then the output from each LSTM unit is the hidden state from that layer. For the final layer if I wanted to predict e.g. a scalar real number, would I want to ...
1
vote
0answers
29 views

How to calibrate model's prediction given past images?

I want to predict how open is the mouth given a face image. It's a regression problem (0= mouth not open, 1=mouth completely open). And something between 0 and 1 is also allowed. ConvNet works fine ...
0
votes
0answers
35 views

Why in RL function approximators with recurrent structures can learn planning?

In the paper An Investigation of Model-Free Planning the authors use ConvLSTM to learn a planning function. In particular, for each input x_t at time-step ...
0
votes
1answer
32 views

Action recognition using video stream data

Recently, I am working on an action recognition project where my input data is from the video stream. I read some of the concepts like ConvLstm, Convolutional Lstm, etc. I am looking for someone who ...
0
votes
0answers
79 views

Should the network weights converge when training Deep Q networks?

I have two sets of data, one training and one test set. I use the train set to train the deep q network model variant. I also continuously evaluate the agent Q values obtained on the test set every ...
2
votes
1answer
125 views

What is the difference between one-shot learning, transfer learning and fine tuning?

Lately, there are lots of posts on one-shot learning. I tried to figure out what it is by reading some articles. To me, it looks like similar to transfer learning, in which we can use pre-trained ...
2
votes
1answer
59 views

In Batch Normalisation, are $\hat{\mu}$, $\hat{\sigma}$ the mean and stdev of the original mini-batch or of the input into the current layer?

In Batch Normalisation, are the sample mean and standard deviation we normalise by the mean/sd of the original data put into the network, or of the inputs in the layer we are currently BN'ing over? ...
1
vote
0answers
139 views

Can we force the initial state of a neural network to produce an “unknown” class?

Has anyone investigated ways to initialize a network so that everything is considered "unknown" at the start? When you consider the ways humans learn, if something doesn't fit a class well enough, it ...
3
votes
1answer
47 views

What's the right way of building a deep Q-network?

I'm new to RL and to deep q-learning and I have a simple question about the architecture of the neural network to use in an environment with a continous state space a discrete action space. I tought ...
0
votes
2answers
50 views

What does it mean to train a model?

We hear this many time for different problems Train a model to solve this problem! What do we really mean by training a model?
1
vote
0answers
28 views

In a DQN, can Prioritized Experience Replay actually perform worse than a regular Experience Replay?

I've written a Double DQN-based stock trading bot using mainly time series stock data. I've recently upgraded my Experience Replay(ER) code with a version of Prioritized Experience Replay (PER) ...
2
votes
1answer
33 views

How do LSTM or GRU gates learn to specialize in their desired tasks?

While I was studying the equations for the computation inside GRU and LSTM units, I realized that although the different gates have different Weight matrices, their overall structure is the same. They ...
3
votes
1answer
82 views

What are the keys and values of the attention model for the encoder and decoder in the “Attention Is All You Need” paper?

I have recently encountered the paper on NLP. It is very new to me and I am still unable to see how that works. I have used all the resources over there from the original paper to Youtube videos and ...
1
vote
0answers
50 views

Confusion about the proof that optimizing InfoNCE equals to maximizing mutual information

In the appendix of Representation Learning with Contrastive Predictive Coding, van den Oord et al. prove that optimizing InfoNCE is equivalent to maximize the mutual information between input image $...
1
vote
0answers
23 views

Optimal critic in WGAN

The Kantorovich-Rubinstein duality for the optimal transport problem implies that the Wasserstein distance between two distributions $\mu_1$ and $\mu_2$ can be computed as $$W(\mu_1,\mu_2)=\underset{f\...
2
votes
0answers
35 views

How to understand the matrices used in the Attention layer?

Attention-scoring mechanism seems to be a commonly-used component in various seq2seq models, and I was reading about the original "Location-based Attention" in Bahadanau well-known paper at https://...
2
votes
0answers
31 views

Why do we need recurrent neural networks instead of feed-forward neural networks? [duplicate]

Why do we need recurrent neural networks instead of feed-forward neural networks? What are the advantages of RNNs compared with FFNNs?
0
votes
0answers
16 views

Identifying and Labeling multiple letters in image

While I attempt to learn AI/ML I have taken on the task to create a Boggle solver. The idea is that a system could take an image of a Boggle arrangement of letters and identify the letters (and the ...
0
votes
0answers
24 views

LSTM for imbalanced panel data

The available tutorials are most focused on time series prediction. I am wondering how shall we prepare the input data when it is an imbalanced data? Here is how data looks like. ...
1
vote
1answer
32 views

What is the main contribution of the paper Disentangling by Factorising?

Considering the paper Disentangling by Factorising, in addition to introducing a new model for Disentangled Representation Learning, FactorVAE (see figure), what is the main theoretical contribution ...
1
vote
1answer
57 views

What is the intuition behind the Xavier initialization for deep neural networks?

The aim of weight initialization is to prevent layer activation outputs from exploding or vanishing during the course of a forward pass through a deep neural network I am really having trouble ...
1
vote
0answers
38 views

Why is this GAN not converging?

This GAN being trained with CelebA dataset doesn't seem to mode collapse, discriminator is not really over confident, and yet the quality is stuck on these rough Picasso-like generator images. Using ...
1
vote
0answers
27 views

Deep learning techniques with time-fixed, time-dependent and imaging data

I have a question about the use of deep learning techniques with time-fixed features and images (setting 1) and time-dependent features (setting 2). (I am pretty new to the deep learning world so ...
1
vote
0answers
28 views

Incorporating domain knowledge into recurrent network

I am currently trying to solve a classification task with a recurrent artificial neural network (RNN). Situation There are up to 350 inputs (X) mapped on one categorical output (y)(13 differnt ...
1
vote
0answers
29 views

Training Conditional DCGAN with GAN-CLS loss

I am trying to implement conditional GAN using GAN-CLS loss as described in paper: https://arxiv.org/abs/1605.05396 So, while training discriminator, I should I have three batches of data: [...
1
vote
1answer
64 views

How and when should we update the Q-target in deep Q-learning?

I have recently watched David silver's course, and started implementing the deep Q-learning algorithm. I thought I should make a switch between the Q-target and Q-current directly (meaning, every ...
1
vote
0answers
30 views

Can you find another reason for sample inefficiency of model-free on-policy Deep Reinforcement Learning?

The following mindmap gives an overview of multiple reasons for sample inefficiency. The list is definitely not complete. Can you see another reason not mentioned so far? Some related links: ...
3
votes
1answer
58 views

How would researchers determine the best deep learning model if every run of the code yields different results?

There are many factors that cause the results of ML models to be different for every run of the same piece of code. One factor could be different initialization of weights in the neural network. ...
1
vote
1answer
52 views

Can we increase the speed of training a reinforcement learning algorithm?

I am new in reinforcement learning. I started reading the PyTorch's documentation about the cart pole control. Whenever an agent fails, they restart the environment. When I run the code, the time in ...
2
votes
1answer
125 views

Why do we need convolutional neural networks instead of feed-forward neural networks?

Why do we need convolutional neural networks instead of feed-forward neural networks? What is the significance of a CNN? Even a feed-forward neural network will able to solve the image classification ...
2
votes
1answer
53 views

Learning policy where action involves discrete and continuous parameters

Typically it seems like reinforcement learning involves learning over either a discrete or a continuous action space. An example might be choosing from a set of pre-defined game actions in Gym Retro ...
3
votes
1answer
81 views

What is the intuition behind the attention mechanism?

Attention idea is one of the most influential ideas in deep learning. The main idea behind attention technique is that it allows the decoder to "look back” at the complete input and extracts ...
5
votes
1answer
128 views

Can deep learning be used to help mathematical research?

I am currently learning about deep learning and artificial intelligence and exploring his possibilities, and, as a mathematician at heart, I am inquisitive about how it can be used to solve problems ...
0
votes
1answer
35 views

What is the advantage of using Google's Coral over Nvidia's Xavier?

I was reading about the possibility of using Google's Coral for deep learning-based object detection and image classification. I heard it has a good speed in terms of frames/sec. I also read that ...
3
votes
1answer
81 views

How does publishing in the deep learning world work, with respect to journals and arXiv?

Let's say I implemented a new deep learning model that pushed some SOTA a little bit further, and I wrote a new paper about for publication. How does it work now? I pictured three options: Submit it ...
1
vote
1answer
40 views

Can I find a mapping that minimizes the maximum distance ratio of certain vectors?

Let's say we have several vector points. My goal is to distinguish the vectors, so I want to make them far from each other. Some of them are already far from each other, but some of them can be ...

1 2
3
4 5
26