Questions tagged [deep-learning]

For questions related to deep learning, which refers to a subset of machine learning methods based on artificial neural networks (ANNs) with multiple hidden layers. The adjective deep thus refers to the number of layers of the ANNs. The expression deep learning was apparently introduced (although not in the context of machine learning or ANNs) in 1986 by Rina Dechter in the paper "Learning while searching in constraint-satisfaction-problems".

644 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
1
vote
0answers
48 views

How to calculate Adaptive gradient?

In the FaceNet paper there mentions an gradient algorithm called 'AdaGrad'(Adaptive Gradient) referenced to this paper which has apparently been used to calculate the gradient of the Triplet Loss ...
1
vote
0answers
103 views

Modelling odd-even distinction of an integer with neural networks

Will it be possible to model the problem of odd-even distinction of an integer (not binary string representation) using neural networks?
1
vote
0answers
58 views

How do stacked denoising autoencoders work

I've been studying a recommender system which uses a collaborative deep learning approach and Bayesian learning. It has the following NN representation : I need to know the working of stacked ...
1
vote
0answers
634 views

How to teach an AI to race optimally in a racing game?

I play a racing game called Need For Madness ( some gameplay: https://www.youtube.com/watch?v=NC5uFZ-t0A8 ). NFM is a racing game, where the player can choose different cars and race and crash the ...
1
vote
0answers
268 views

Tensorflow: Can't overfit training data with batch size > 1

I coded a small RNN network with Tensorflow to return the total energy consumption given some parameters. There seem to be a problem in my code. It can't overfit the training data when I use a batch ...
1
vote
0answers
169 views

Deep Learning Approaches for Color Enhancement Testing

I'm a student, and currently into image processing project and coding using OpenCV. Recently, I watched Sebastian Thrun from Udacity in TedTalks talked about AlphaGo and I'm totally interested in the ...
1
vote
0answers
231 views

Natural language processing with a continuous dependent variable

I have a large number of observations. Each observation contains: dependent variable: a scores ranging from 0 - 100 independent variable: a large article I want to know which words or phrases ...
1
vote
0answers
186 views

Load a pretrained DQN model with tensorflow

I have replicated the NN that can play Breakout, but I don't know how use the pre-trainded checkpoints from DQN-tensorflow devsisters (github): https://github.com/devsisters/DQN-tensorflow/issues/39 ...
1
vote
0answers
99 views

Infer dependent variables to produce output aligned to trained data

Hypothetical example, say I wanted: P(gender,ethnicity|age,hair); so that the input would aligned to a trained dataset of: ...
1
vote
1answer
52 views

What is the status of the capsule networks?

What is the status of the capsule networks? I got an impression that capsule networks turned out not to be so useful in applications more complicated than the MNIST (at least according to this reddit ...
1
vote
2answers
107 views

Does replacing 3x3 filters with 3x1 and 1x3 filters improve the performance?

Recently I have come up with a VGG16 model for my binary classification task. I have relatively simple signal images Therefore (maybe?) other deeper models like ...
1
vote
2answers
150 views

How do I perform object detection if there is only one type of object?

How do I do object detection (or identify the location of an object) if there is only one kind of object, and they are more of less similar size, but the picture does not look like standard scenes (it ...
1
vote
2answers
67 views

CNN Pooling layers unhelpful when location important?

I'm trying to use a CNN to analyse statistical images. These images are not 'natural' images (cats, dogs, etc) but images generated by visualising a dataset. The idea is that these datasets hopefully ...
0
votes
0answers
13 views

Reward firstly increase, but after more episodes, start decrease, and weights diverges

I'm making a simple deep Q learning algorithm, with cartpole-v1 env. Like you can see in chart, after many episodes the reward decrease, some possible reasons? The exploration vs axplotation algorithm ...
0
votes
0answers
8 views

Choosing the size of the network for Neural Collaborative Filtering (NCF)?

I've been working on Neural Collaborative Filtering (NCF) recently to build a recommender system. After doing some hyperparameter tuning with various sizes for embedding and dense layers sizes, from ...
0
votes
0answers
5 views

Can I get some advices on inferencing people from upwards using Yolov5?

I'm trying to inference people from upwards and count them using Yolov5. I know the controversy between yolov5 and yolov4, but for me, Yolov5 is more easier and reliable to use, also the setup. I have ...
0
votes
0answers
20 views

Why integrated gradients don't explain samples correctly?

I have a linear tabular dataset made of floats. The dataset follows a simple rule like: ...
0
votes
0answers
22 views

How does Keras `BatchNormalization` work?

I have read some articles and watched some videos by Andrew Ng stating that it makes more sense to use batch normalization before applying the activation function. ...
0
votes
0answers
12 views

Monotonically increasing Siamese neural network

I want to design a Siamese neural network for which there are n inputs which are all positive and there is one output which is also positive. How can I enforce the condition that the input/output ...
0
votes
0answers
11 views

How can I prune BERT layers

I would like to finetune BERT on SQuAD and then evaluate the output from each layer (so from using 1 layer to using all 12). I know you can prune heads using Huggingface but was wondering how could ...
0
votes
0answers
12 views

Things to consider while adding custom function to generator output in GAN

I am training a GAN model (DCGAN) to generate 128x128 images. Now, I wish to add a function which will take the generator output, perform some pre-defined operations on it, and return the modified ...
0
votes
0answers
7 views

Should I L-2 Normalise outputs in Siamese Neural Neural Network for distance computation for Triplet Loss or not?

I am building a Siamese Neural Network for Images (CNN) which uses the FaceNet's Triplet Loss as its loss function. I found a good Implementation here where we build a model and the outputs from the ...
0
votes
0answers
15 views

Adversarial Attacks and interpolation methods

I am attacking a model. The model is a simple CNN and PGD is used. The model runs on 112x112 ImageNet dataset. So I first load images as 224x224 and use interpolation function to downsample it to ...
0
votes
4answers
46 views

Oversampling of Balanced Dataset

I am trying to add more data points in my (almost) balanced dataset for training my neural network. I have come across techniques such as SMOTE or Random Over Sampling but they work best for ...
0
votes
0answers
15 views

Can anybody just confirm whether or not my understanding of depthwise separable convolutions is correct?

I just need a simple Yes/No confirmation or to debunk my understanding of the difference between the normal convolutions and depthwise seperable convs. I have read quite a few articles and watched a ...
0
votes
1answer
15 views

What algorithm to use to classify data by spatial relations?

Let's assume I have dataset of image-like 2D samples where values can be divided into few discrete levels (for example 1, 2, 3 and 4) like in the image below, where each color maps different value, ...
0
votes
0answers
15 views

How do we get from entropy to KL divergence in this paper?

I'm reading through Regularizing Neural Networks By Penalizing Confident Output Distributions and I'm stuck on the equation in section 3.2. It's not clear to me at all that the self-entropy of the ...
0
votes
0answers
20 views

Backpropagation - what does rate of change calculated from the partial derivatives actually relate to?

I understand conceptually how backpropagation works according to the chain rule, and I understand that partial derivatives calculate the rate of change of a function containing multiple variables with ...
0
votes
1answer
34 views

Saving Computational Time with DNN Pruning

I am following this tutorial on pruning. It seems particularly attractive to me as it is said that if we reduce the number of parameters of the model we will reduce the execution time, which is ...
0
votes
0answers
9 views

Why is my LSTM model predicting accurately for only a few values and showing drastic aberration later?

I am training an LSTM model using stock data for time series forecasting and the results are a little confusing to me. This is the prediction I get after 5 epochs. And this after 100 epochs. Why the ...
0
votes
0answers
19 views

Can a siamese neural network distinguish expected from unexpected changes?

Please redirect me to another stack exchange if this isn't the appropriate forum. I am interested in finding a neural network architecture that can detect distance between two inputs. As an example, ...
0
votes
0answers
16 views

PyTorch: LSTM error while trying to update the hidden state

I am trying to train an LSTM while keeping its hidden state (LSTM stateful) until the moment when I am going to start a new epoch(episode). But here it's come an interesting situation because I am ...
0
votes
0answers
12 views

How to make CNN to recognize whole picture, not just the details?

In my current project I use CNNs to analize plots (CNN autoencoders for feature extraction and KMeans for clusterization) and I get the feeling that these networks, can extract only features that are ...
0
votes
1answer
19 views

How does high entropy targets relate to less variance of the gradient between training cases?

I've been trying to understand the Distilling the Knowledge in a Neural Network paper by Hinton et al. But I cannot fully understand this: When the soft targets have high entropy, they provide much ...
0
votes
0answers
21 views

How can AI algorithms be used in regards to cryptocurrency and token mining?

I am new to AI and lack the knowledge of it's capabilities. A question popped in my head, in regards to Blockchains and the mining of cryptocurrencies and tokens, how can machine learning algorithms ...
0
votes
0answers
17 views

How do the trainable projection layer used in PRADO and pQRNN work?

Trainable projection layers are said to be a very powerful thing but after reading: https://www.aclweb.org/anthology/D19-1506.pdf https://arxiv.org/pdf/2101.08890.pdf I don't understand how it works....
0
votes
0answers
24 views

Do Gradient Descent and Natural Gradient solve the same problem?

I am troubled by natural gradient methods. If we have a function f(x) we wish to minimize, gradient descent minimizes f(x) of course, but what does the natural gradient do? I found on https://...
0
votes
0answers
15 views

Dealing with huge peak in data distribution

I am trying to predict a continuous value using a deep neural network. I have about 100,000 samples, where input is a sequence of RNA, and output is a continuous metric determining the quality of the ...
0
votes
0answers
17 views

Multi dimensional LSTM modeling in KERAS

I have a database of time series signals with multiple features and Im trying to build a model to predict whether or not two samples are related to each other. For example : a database of 1000 sample ...
0
votes
0answers
11 views

Controlling mutual information in latent variables

Recently, I read some interesting papers on mutual information (MI) estimation in high dimensional variables using neural networks [Belghazi et al., 2018][Poole et al., 2019]. These methods besides ...
0
votes
0answers
8 views

Is it possible to get this loss with spikes, when training an LSTM with the cross-entropy on a multi-class classification problem of a time series?

The main question here will be "should I look for a bug?" My setup is a time series multiclass classification task, labeled per frame. I am using an LSTM, feeding inputs and using ...
0
votes
0answers
15 views

How to have a DNN output a classification for each user at once?

I have a Reinforcement Learning environment with an agent that allocates power values to different users. To do so, I have thought of implementing a deep neural network like the one shown in the ...
0
votes
0answers
32 views

Is vectorizing backpropagation feasible?

Does it make sense to have the backpropagation of a neural network layer happen all at once if the learning rate is lowered? This would mean the new weights of that layer would be independent of each ...
0
votes
0answers
15 views

Adding BERT embeddings in BiLSTM embedding layer

I am want to use BERT embeddings in the BiLSTM embedding layer instead of Word2Vec or FastText Embeddings. There is any code to do that?
0
votes
0answers
33 views

What are the best optimizations I can add to my neural network?

I am making an artificial neural network from scratch (without nn libraries) in python. So, as you can guess, its extremely unoptimized and slow. For this neural ...
0
votes
0answers
15 views

Mixed precision training - why we're fine with doing point wise operations in FP32

I'm starting to learn more about mixed-precision training, and I'm in particular confused about point-wise operations. In the original article (link), the authors mention, citing: Point-wise ...
0
votes
0answers
25 views

Why is my siamese network learning very well in e.g. 1 out of every 5 runs?

Why is my siamese network learning very well in e.g. 1 out of every 5 runs? The rest of the time it's not learning and maintains an accuracy of 0.5. Any explanations? Is the contrastive loss taken in ...
0
votes
0answers
19 views

How does BERT answer questions?

I have been trying to understand how the BERT model works. Specifically, I am trying to understand how it picks up answers to questions on a given passage. I have tried following this blog post and, ...
0
votes
0answers
11 views

Handling multi scale data input to neural networks

This is a theoretical question. Let's assume the simplest setup of binary classification of a time series. The input every frame is a multi channel sample of many sensors. Each sensor is completely ...
0
votes
0answers
27 views

Is it possible to use GPT-3 with voice cloning to have GPT-3 respond in a custom voice?

I'm curious about three things here. Is it possible to use GPT-3 along with voice cloning to have GPT-3 either respond in a custom voice or possibly create an app that can do the same sort of thing? ...

1
9
10
11 12 13