Questions tagged [deep-learning]

For questions related to deep learning, which refers to a subset of machine learning methods based on artificial neural networks (ANNs) with multiple hidden layers. The adjective deep thus refers to the number of layers of the ANNs. The expression deep learning was apparently introduced (although not in the context of machine learning or ANNs) in 1986 by Rina Dechter in the paper "Learning while searching in constraint-satisfaction-problems".

Filter by
Sorted by
Tagged with
4
votes
1answer
15 views

How does the Generator in GAN's work?

After reading a lot of articles (for instance, this one - https://developers.google.com/machine-learning/gan/generator )... I've been wondering: How does the Generator in GAN's work? What is the ...
0
votes
1answer
22 views

Why is this deep Q agent constantly learning just one action?

I'm trying to implement deep q learning in the OpenAI's gym "Taxi-v3" environment. But my agent only learns to do one action in every state. What am I doning wrong? Here is the Github repository with ...
1
vote
0answers
21 views

What activation functions are better for what problems?

I’ve been reading about neural network architectures. In certain cases, people say that the sigmoid "more accurately reflects real-life" and, in other cases, functions like hard limits reflect "the ...
2
votes
0answers
27 views

How can I compare EEG data with accelerometer data in 1 algorithm?

I have frequency EEG data from fall and non-fall events and I am trying to incorporate it with accelerometer data that was collected at the same time. One approach is, of course, to use two separate ...
1
vote
0answers
18 views

Is there any paper that uses truncated neural networks?

Recently, I've found good success in truncated neural networks ie functions of the form $$ g=f1_{[-M,M]^d}, $$ where $f:\mathbb{R}^d\to\mathbb{R}^n$ is a feed-forward neural network and $1_{[-M,M]^d}$ ...
0
votes
1answer
18 views

What is the difference between batches in deep Q learning and supervised learning?

How is the batch loss calculated in both DQNs and simple classifiers? From what I understood, in a classifier, a common method is that you sample a mini-batch, calculate the loss for every example, ...
1
vote
1answer
30 views

Why does PyTorch use a different formula for the cross-entropy?

In my understanding, the formula to calculate the cross-entropy is $$ H(p,q) = - \sum p_i \log(q_i) $$ But in PyTorch nn.CrossEntropyLoss is calculated using this ...
3
votes
2answers
98 views

What's the function that SGD takes to calculate the gradient?

I'm struggling to fully understand the stochastic gradient descent algorithm. I know that gradient descent allows you to find the local minimum of a function. What I don't know is what exactly that ...
1
vote
0answers
16 views

Training dataset for convolutional neural network classification - will images captured on the ground be useful for training aerial imagery?

I am an agronomy graduate student looking to classify crops from weeds using convolutional neural networks (CNNs). The basic idea that I am wanting to get into involves separating crops from weeds ...
2
votes
0answers
12 views

word2vec implementation in Tensorflow 2.0

I want to implement word2vec using tensorflow 2.0 I have prepared dataset according to the skip-gramm model and I have got approx. 18 million observations(target and context words). I have used the ...
3
votes
1answer
33 views

How can a DQN backpropagate its loss?

I'm currently trying to take the next step in deep learning. I managed so far to write my own basic feed-forward network in python without any frameworks (just numpy and pandas), so I think I ...
2
votes
2answers
78 views

Why are traditional ML models still used over deep neural networks?

I'm still on my first steps in the Data Science field. I played with some DL frameworks, like TensorFlow (pure) and Keras (on top) before, and know a little bit of some "classic machine learning" ...
3
votes
0answers
41 views

Why don't the neural networks inside LSTM cells contain hidden layers?

I watched a video explaining how LSTM cells have very rudimentary feed-forward neural networks, basically a 2 layer input-output with no hidden layers. Why don't LSTM cells have more complex neural ...
2
votes
0answers
47 views

Why isn't there a model playing FPS like CoD or Battlefield already existing?

Assuming we had an unlimited time to train a model and a very powerful machine to use our model in real-time (hello quantum computer), I'd like to know why no one could achieve to build an AI able to ...
0
votes
0answers
29 views

How to Layer based Feature extraction?

I have read that in deep networks you can engineer each layer for a particular purpose with regards to feature learning. I'm wondering how that is actually done and how it is trained? In addition ...
0
votes
0answers
27 views

What's the mathematical relationship between number of trainable parameters and size of training set?

Let's say that I have a pre-trained model where the training set used to pretrain the model is very different from my training set. Let's say I unfreeze layers that have X trainable parameters. What ...
1
vote
0answers
15 views

What's the difference between semi-supervised VAEs and conditional VAEs?

Can someone explain the difference? I'm assuming the difference is just that the neural nets representing the encoder and decoder are trained in a semi-supervised manner in semi-supervised VAE, which ...
5
votes
1answer
200 views

What is the mathematical definition of an activation function?

What is the mathematical definition of an activation function to be used in a neural network? So far I did not find a precise one, summarizing which criterions (e.g. monotonicity, differentiability, ...
1
vote
0answers
20 views
1
vote
1answer
54 views

What is a cascaded convolutional neural network?

For a project I am doing, I found the paper Face Alignment in Full Pose Range: A 3D Total Solution. It is using a cascaded convolutional neural network, but I wasn't able to find the original paper ...
4
votes
1answer
33 views

Can training a model on a dataset composed by real images and drawings hurt the training process of a real-world application model?

I'm training a multi-label classifier that's supposed to be tested on underwater images. I'm wondering if feeding the model drawings of a certain class plus real images can affect the results badly. ...
4
votes
2answers
422 views

Effect of batch size and number of GPUs on model accuracy

I have a data set which was split using a fixed random seed and I am going to use 80% of data for training and rest on validation. Here are my GPU and batch size configurations use ...
3
votes
0answers
27 views

How can I do hyperparameter optimization for a CNN-LSTM neural network?

I have built a CNN-LSTM neural network with 2 inputs and 2 outputs in Keras. I trained the network with model.fit_generator() (and not ...
0
votes
0answers
26 views

Can BERT convert paragraph to vectors (doc2vec embedding) for classification tasks using another model? [closed]

I've heard the term "BERT embeddings" used a lot. Is this similar to Word2Vec or Doc2Vec embeddings used in NLP? I need to convert articles of text into one single vector as an input for my model. I ...
2
votes
0answers
32 views

How can I extract the reason of the legal compensation from a court report?

I'm working on a project (court-related). At a certain point, I have to extract the reason of the legal compensation. For instance, let's take these sentences (from a court report) Order mister X ...
4
votes
2answers
56 views

Does summing up word vectors destroy their meaning?

For example, I have a paragraph which I want to classify in a binary manner. But because the inputs have to have a fixed length, I need to ensure that every paragraph is represented by a uniform ...
1
vote
0answers
37 views

Is it a good idea to apply reinforcement learning to dots and boxes? [closed]

I am currently in college, and trying to learn reinforcement learning by myself. My primary goal is building an agent that play games such as dots and boxes. I have sufficient highschool maths ...
4
votes
1answer
83 views

How to formalize learning in terms of information theory?

Consider the following game on a MNIST dataset: There are 60000 images. You can pick any 1000 images and train your Neural Network without access to the rest of images. Your final result is ...
1
vote
1answer
54 views

Can we calculate mean recall and precision

I'm evaluating the accuracy in detecting objects for my image data set using three deep learning algorithms. I have selected a sample of 30 images. To measure the accuracy, I manually count the number ...
2
votes
1answer
25 views

How can a system recognize if two strings have the same or similar meaning?

How can a system recognize if two strings have the same or similar meaning? For example, consider the following two strings Wikipedia provides good information. Wikipedia is a good source of ...
3
votes
0answers
24 views

Rarely predict minority class imbalanced datasets

I have a dataset in which class A has 99.8%, class B 0.1% and class C 0.1%. If I train my model on this dataset, it predicts always class A. If I do oversampling, it predicts the classes evenly. I ...
2
votes
1answer
42 views

Are CNN, LSTM, GRU and transformer AGI or computational intelligence tools?

Will CNN, LSTM, GRU and transformer be better classified as Computational Intelligence (CI) tools or Artificial General Intelligence (AGI) tools? The term CI arose back when some codes like neural ...
4
votes
2answers
48 views

Is there any classifier that works best in general for NLP based projects?

I've written a program to analyse a given piece of text from a website and make conclusary classifications as to its validity. The code basically vectorizes the description (taken from the HTML of a ...
3
votes
2answers
44 views

How to use LSTM to generate a paragraph

A LSTM model can be trained to generate text sequences by feeding the first word. After feeding the first word, the model will generate a sequence of words (a sentence). Feed the first word to get the ...
2
votes
1answer
63 views

The reasoning behind the number of filters in the convolution layer

Let's assume an extreme case in which the kernel of the convolution layer takes only values 0 or 1. To capture all possible patterns in input of $C$ number of channels, we need $2^{C*K_H*K_W}$ filters,...
1
vote
0answers
30 views

Sample size for the evaluation of Deep Learning Models

I'm evaluating the performance and accuracy in detecting objects for my data set using three deep learning algorithms. In total there are 24,085 images. I measure the performance in terms of time ...
0
votes
1answer
43 views

How is the loss value calculated in order to compute the gradient?

The gradient descent step is the following \begin{align} \mathbf{W}_i = \mathbf{W}_{i-1} - \alpha * \nabla L(\mathbf{W}_{i-1}) \end{align} were $L(\mathbf{W}_{i-1})$ is the loss value, $\alpha$ the ...
0
votes
0answers
19 views

Reasoning behind $Zero$ validation accuracy in the following ResNet50 model for classification

I have written this code to classify Cats and dogs using Resnet50. Actually while studying I came to the conclusion that Transfer learning gives very good accuracy for deep learning models, but I ...
1
vote
0answers
16 views

Hinton's Capsule network 16 dimensions

As you may know, Hinton's Capsule Network has been around for about 2 years now. https://arxiv.org/abs/1710.09829 Much ado has been made about how the Capsules output a vector (magnitude = ...
2
votes
1answer
32 views

Accuracy scores in a Deep Learning project

I'm using three pre-trained deep learning models to detect vehicles and count from an image data set. The vehicles belong to one of these classes ['car', 'truck', 'motorcycle', 'bus']. So, for a ...
1
vote
0answers
14 views

Number of weights in historical to cutting edge deployment of deep networks [closed]

In cutting edge deployment of deep networks for different architectures (such as $CNN$, $QRNN$ etc) what is the historical trend of current limits of trainability possible computationally? By this I ...
2
votes
1answer
38 views

How to make DNN learn multiplication/division?

A single neuron with 2 weights and identity activation can learn addition/subtraction as the 2 weights will converge to 1 and 1 (addition), or 1 and -1 (subtraction). However, for multiplication and ...
1
vote
0answers
19 views

Is there any app / API which generates meaningful information?

I was wondering if there is any application / API which given a word comes up with legible and meaningful information related to it, and if possible relates it to any recent happenings or development ...
2
votes
0answers
23 views

YOLO 9000 about Better Stronger

In this paper, YOLO has three features compared to YOLO v1. This question is about Better and Faster. In the Better section, there are many techniques such as Batch Norm, Anchor Box and so on. In the ...
3
votes
0answers
37 views

How does the memory mechanism (reading and writing) work in a neural Turing machine?

In neural Turing machine (NTM), reading memory is represented as \begin{align} r_t \leftarrow \sum\limits_i^R w_t(i) \mathcal{M}_t(i) \tag{2} \end{align} and writing to memory is represented as ...
1
vote
0answers
25 views

How to handle a high dimensional video (large number of frames per video) data for training a video classification network

I have a video dataset as follows. Dataset size: 1k videos Frames per video: 4k (average) and 8k (maximum) Labels: Each video has one label. So the size of my input will be (N, 8000, 64, 64, 3) ...
4
votes
1answer
72 views

What are some new deep learning models for learning latent representation of data?

I know that autoencoders are one type of deep neural networks that can learn the latent representation of data. I guess there should be several other models like autoencoders. What are some new deep ...
2
votes
1answer
34 views

How does a batch normalization layer work?

I understood that we normalize to input features in order to bring them on the same scale so that weights won't be learned in arbitrary fashion and training would be faster. Then I studied about ...
4
votes
0answers
49 views

Training and inference for highly-context-sensitive information

What is the best way to train / do inference when the context matters highly as to what the inferred result should be? For example in the image below all people are standing upright, but because of ...
1
vote
0answers
20 views

Why the error rates in table3 and table4 are differenct in the paper “deep residual learning for image recognition”

Why are the error rates in table 3 and table 4 are different in the paper Deep Residual Learning for Image Recognition (2015). They are both error rates on the validation sets by single model. ...