Questions tagged [deep-learning]

For questions related to deep learning, which refers to a subset of machine learning methods based on artificial neural networks (ANNs) with multiple hidden layers. The adjective deep thus refers to the number of layers of the ANNs. The expression deep learning was apparently introduced (although not in the context of machine learning or ANNs) in 1986 by Rina Dechter in the paper "Learning while searching in constraint-satisfaction-problems".

Filter by
Sorted by
Tagged with
0
votes
0answers
3 views

How to predict multiple set of coordinates (of bounding boxes) for signboards text localization through neural network?

I am creating a signboard translation application from scratch. I have images of signboards where there are multiple texts and I have the corresponding set of coordinates of bounding boxes for ...
2
votes
0answers
15 views

Why is my GAN more unstable with bigger networks?

I am working with generative adversarial networks (GANs) and one of my aims at the moment is to reproduce samples in two dimensions that are distributed according to a circle (see animation). When ...
0
votes
0answers
13 views

How does the embeddings work in vision transformer from paper?

I get the part from paper where image is split into P say 16x16 (smaller images) patches and then you have to ...
0
votes
1answer
23 views

Factors that causing totally different outcomes from an exactly same model and datasets

Here is a model that trains time series data in (batch, step, features) way. I have kept the random state for train test split function the same. Every parameter below the same, running the model ...
1
vote
0answers
15 views

Underfitting a single batch: Can't cause autoencoder to overfit multi-sample batches of 1d data. How to debug?

TL;DR I am unable to overfit batches with multiple samples using autoencoder. Fully connected decoder seems to handle more samples per batch than conv decoder, but then also fails when number of ...
1
vote
1answer
33 views

Aren't scores in the Wasserstein GAN probabilities?

I am quite new to GAN and I am reading about WGAN vs DCGAN. Relating to the Wasserstein GAN (WGAN), I read here Instead of using a discriminator to classify or predict the probability of generated ...
0
votes
0answers
19 views

Is using a LSTM, CNN or any other neural network model on top of a Transformer(using hidden states) overkill?

I have recently come across transformers, I am new to Deep Learning. I have seen a paper using CNN and BiLSTM on top of a transformer, the paper uses a transformer(XLM-R) for sentiment analysis in ...
0
votes
1answer
37 views

Is the working of RNNs, LSTM and GRU sequential or parallel?

You take any blog or any example and all they tell you about is the given picture below. It has 4 different matrices and 3 of whose weights are shared. So, I'm wondering how is this achieved in ...
0
votes
0answers
13 views

How to deal with dynamically changing input tensor in neural networks without padding?

I have a dataset about the monitored health/growth of a community of people. The dataset has tensor shaped (batch_size, features, person, window), where: person==...
0
votes
0answers
36 views

What would be the vectorized version of the gradient and loss of the cross-entropy formula?

I am looking for the vectorized version of these formulas: The cross-entropy loss $$L(D) = \sum^N_{n=1}\sum^K_{k=1}t_{kn} \ln(y_{\vec{w_k}}(\vec{x_n}))$$ The gradient of the cross-entropy $$\nabla ...
0
votes
0answers
30 views

Alpha Zero does not converge for Connect 6, a game with huge branching factor - why?

I have a problem with applying alpha zero self-play to a game (Connect 6) with a huge branching factor (30,000 on average). I have implemented the MCTS as described but I found that during the MCTS ...
2
votes
1answer
49 views

What do the variables in the cross-correlation formula mean?

I understand what cross-correlation does given a kernel and an input image, but the formula confuses me a little. Given here in Goodfellow's Deep Learning (page 329), I can't quite understand what $m$ ...
1
vote
0answers
14 views
0
votes
0answers
11 views

Nonrealtime tracking for object detection (with or without deep features)

I've been doing a search for tracking systems based on bounding box for several days (like Sort, DeepSort, Motpy, ...). But neither is nonrealtime. My problem is based on the tracking of multiple ...
-1
votes
0answers
10 views

How to take an average of features extraction from sequence of images? [closed]

I have a 5D sequence of images from feature extraction, but I need to take average among all sequence features so that can fit to 2DCNN layer, how to do that in TensorFlow? Thanks in advance.
2
votes
0answers
31 views

Are there any new weight initialization techniques for DNN published after 2015?

Considering weights initialization in my personal projects, I always used some standard techniques such as: Glorot (also known as Xavier) initialization (2010). Mertens initialization (2010). He ...
0
votes
0answers
17 views

Keras 1D CNN always predicts the same result even if accuracy is high on training set

The validation accuracy of my 1D CNN is stuck on 0.5 and that's because I'm always getting the same prediction out of a balanced data set. At the same time my training accuracy keeps increasing and ...
0
votes
1answer
23 views

Why won't my model train with CTC loss?

I am trying to train an LSTM using CTC loss, but the loss does not decrease when I train it. I have created a minimal example of my issue by creating training data where the network simply has to copy ...
1
vote
1answer
37 views

What happens if there is no activation function in some layers of a neural network?

What if I don't apply an activation function on some layers in a neural network. How will it affect the model? Take for instance the following code snippet: ...
2
votes
1answer
50 views

How to compute the gradient of the cross-entropy loss function with respect to the parameters with softmax activation function?

I've seen plenty of examples of people doing Sigmoid + MSE backpropagation implementations, yet I do not seem to understand how to implement backpropagation as stated in the title in the case of multi-...
3
votes
2answers
43 views

Is pre-processing used in deep learning?

I'm new to deep learning. I wanted to know: do we use pre-processing in deep learning? Or it is only used in machine learning. I searched for it and its methods on the internet, but I didn't find a ...
0
votes
1answer
17 views

Is it possible to modify or replace the basic network of YOLO?

I have an idea to adapt YOLO algorithm to my application, the original YOLO algorithm is for image classifications, which have 24 convolutional layers with output class of 1000, is it possible to ...
1
vote
0answers
28 views

Are monotonically increasing functions easier to learn?

A monotonically increasing function is a function that as x gets bigger so does its output. So, if plotted, it will never go down. Although the outputs might stay constant. Logically this seems like ...
2
votes
0answers
21 views

Why does batch norm standardize with sample mean/variance, when it also learns parameters to scale the mean/variance?

Batch norm is a normalizing layer that is shown to help deep networks learn faster and with higher generalization accuracy. It normalizes the activations of the previous layer to a mean $\beta$ and ...
1
vote
0answers
20 views

variational auto encoder loss goes down but does not reconstruct input. out of debugging ideas

My variational autoencoder seems to work for MNIST, but fails on slightly "harder" data. By "fails" I mean there are at least two apparent problems: Very poor reconstruction, for ...
1
vote
0answers
24 views

What is the definition of pre-training?

I want to pre-train a model (combined by two popular modules A and B, and both are large blocks), then fine-tune it on downstream tasks. What if for the weight initialization for pre-training, module ...
2
votes
0answers
39 views

How can I use Monte Carlo Dropout in a pre-trained CNN model?

In Monte Carlo Dropout (MCD), I know that I should enable dropout during training and testing, then get multiple predictions for the same input $x$ by performing multiple forward passes with $x$, then,...
0
votes
0answers
19 views

Which AI technique should I use for (key)point detection (in an image of a plantar pressure)?

I am relatively new to the field of AI. I have a problem that I would like to solve with AI, but I don't know which buzzwords I should use to search for solutions. I have a plantar pressure scan, like ...
0
votes
0answers
15 views

Is parameter sharing in AlBERT akin to repeated application of same function on input?

I read the AlBERT and saw that its architecture used "Parameter Sharing" among layers of the encoder. They mentioned that this was done to save model space, make fewer training parameters ...
0
votes
0answers
21 views

Is there a reference that describes Recurrent Neural Networks for NLP tasks?

I would like some references of works that try to understand the functioning of any kind of RNN in natural language processing tasks. They can be any work that tries to explain the functioning of the ...
1
vote
0answers
28 views

How is few-shot learning different from transfer learning?

To my understanding, transfer learning helps to incorporate data from other related datasets and achieve the task with less labelled data (maybe in 100s of images per category). Few-shot learning ...
0
votes
0answers
16 views

Interpreting coordinates of the function from the image using Grid Lines

I have a very different kind of problem at hand. I have a function plotted on the grid lines as shown below Now, I need to get the x and y coordinates for this particular function at regular ...
1
vote
0answers
27 views

Backpropagation implementation not applicable for other cases

I saw this implementation of backpropagation in MATLAB, where the loss function used is MSE, and the last layer's activation function was sigmoid. I denoted the portions of the formula for what I ...
2
votes
0answers
23 views

How do gradients are flown back into the Siamese network when branching is done?

I am curious about the working of a Siamese network. So, let us suppose I am using a triplet loss for my network and I have instantiated single CNN 3 times and there are 3 inputs to the network. So, ...
0
votes
0answers
25 views

Correct way to work with both categorical and continuous features together

I have a time series with both continuous and categorical features, and I want to do a prediction task. I will elaborate: The data is composed of 100Hz sampling of some voltages, kind of like an ecg ...
2
votes
0answers
27 views

How are nested bounding boxes handled in object detection (and in particular in the case of the SSD)?

The basic approach to non-maximum-suppression makes sense, but I am kind of confused about how you handle nested bounding boxes. Suppose you have two predicted boxes, with one completely enclosing ...
0
votes
2answers
49 views

Simple DQN too slow to train [closed]

I have been trying to solve the OpenAI lunar lander game with a DQN taken from this paper https://arxiv.org/pdf/2006.04938v2.pdf The issue is that it takes 12 hours to train 50 episodes so something ...
1
vote
0answers
26 views

How to derive compact convex set K and its diameter D to program Accelegrad algorithm in practice?

Given the original paper (https://arxiv.org/pdf/1809.02864.pdf), I would like to implement the Accelegrad algorithm for which I report the pseudocode of the paper: In the pseudocode, the authors ...
1
vote
1answer
44 views

Training a classifier on different datasets with different image conditions for different labels causes the model to infer using the background

I have an interesting problem related to training the model on two different datasets for the target feature on images taken on different conditions, which might affect the model's ability to ...
0
votes
1answer
41 views

How to deal with wrong labels in a classification problem? [closed]

I have a problem where I need to train a NN to classify images based on a 10k samples trainset where each sample has 2 labels - one correct and one wrong. What could be a good approach to this problem?...
0
votes
1answer
61 views

A way to leverage machine learning to reduce DFS/BFS search time on a tree graph?

I'm not very knowledgeable in this field but I'm wondering if any research or information already exists for the following situation: I have some data that may or may not look similar to each other. ...
3
votes
2answers
117 views

Mathematical Analysis of the Loss function of GAN

I was pondering on loss function of GAN and following thing turned out \begin{aligned} L(D, G) & = \mathbb{E}_{x \sim p_{r}(x)} [\log D(x)] + \mathbb{E}_{x \sim p_g(x)} [\log(1 - D(x)] \\ & =...
1
vote
0answers
35 views

Training speed in GPU vs CPU for LSTM

I was experimenting seq2seq model which was the bi-LSTM encoder/decoder with attention. When I compared the training times on GPU vs CPU while varying the batch size, I got CPU on the small batch ...
0
votes
0answers
15 views

Is there any solution to the problem of detecting whether a user is having trouble finding something while surfing a webpage?

While a user is navigating through a website, we need to detect whether the user is having trouble finding something in realtime. The output is used to trigger an event that should pop up a FAQ page ...
1
vote
1answer
27 views

Optimum Discriminator for label smoothed GAN

I was reading the paper called Improved Techniques for Training GANs. And, in the one-sided label smoothing part, they said that optimum discriminator with label smoothing is $$ D^*(x)=\frac{\alpha \...
0
votes
0answers
19 views

How to build a commercial Image-Image search engine using LSH / Near Duplicate or some other algo on more than 20M images

TL;DR: HOW DO I APPLY LSH WITH A DEEP LEARNING MODEL TO BUILD A IMAGE-IMAGE SEARCH ENGINE ON >20M IMAGES? I want to build a system where I am helping my ...
4
votes
1answer
47 views

What is a unified neural network model?

In many articles (for example, in the YOLO paper, this paper or this one), I see the term "unified" being used. I was wondering what the meaning of "unified" in this case is.
0
votes
1answer
33 views

Loss randomly changing, incorrect output (even for low loss) when trying to overfit on a single set of input and output

I am trying to make a neural network framework from scratch in C++ just for fun, and to test my backpropagation, I thought it would be an easy way to test the functionality if I give it one input - a ...
2
votes
2answers
30 views

Appropriate convolutional neural network architecture when the input consists of two distinct signals

I have a dataset consisting of a set of samples. Each sample consists of two distinct desctized signals S1(t), S2(t). Both signals are synchronous; however, they show different aspects of a phenomena. ...
1
vote
0answers
23 views

Papers on using symbolic methods as constraint on neural network?

Given a set of constraints on the input data, I am looking for papers that discuss using symbolic methods (decision trees, rule based, etc.) as a separate source of certainty in a classification task. ...

1
2 3 4 5
29