Questions tagged [deep-learning]

For questions related to deep learning, which refers to a subset of machine learning methods based on artificial neural networks (ANNs) with multiple hidden layers. The adjective deep thus refers to the number of layers of the ANNs. The expression deep learning was apparently introduced (although not in the context of machine learning or ANNs) in 1986 by Rina Dechter in the paper "Learning while searching in constraint-satisfaction-problems".

609 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
2
votes
0answers
559 views

Getting worse performance when training a pre-trained model with the existing class

I am training pre-trained SSD-InceptionV2-Coco to detect the "car", which is one of the classes in mscoco label. I train the model with ~50k sample from KITTI, 500k iteration with batch size 2. I ...
2
votes
0answers
34 views

Possible to use codebase snapshots as input in deep learning?

I'm trying to predict grades within a course at my university. At the moment I manually extracting features, but I'm curious if it's possible to somehow use my entire dataset with a deep learning ...
2
votes
0answers
371 views

Coding CGAN paper model in Keras

I was coding a CGAN model using Keras along with the paper (https://arxiv.org/pdf/1411.1784.pdf) and I wanted to try and match the models to exactly what the paper says. I knew the models presented in ...
2
votes
0answers
245 views

Deep NN architecture for predicting a matrix from two matrices

Recently my friend asked me a question: having two input matrices X and Y (each size NxD) where D >> N, and ground truth matrix Z of size DxD, what deep architecture shall I use to learn a deep model ...
2
votes
0answers
46 views

Loss of precision when encoding DNN weights

This question is related to the usage of NN in critical systems (those where a failure can cause life threatening situations - autopilots for example) and the need for formal guarantees on their ...
2
votes
0answers
120 views

Train, Validation and Test Split for Reporting Accuracy of Neural Model and BOW

I need to report accuracies of my neural model in a conference paper as compared to various baselines. What are the accepted standards for reporting accuracies in a fair manner? Neural Model: To be ...
2
votes
0answers
70 views

seq2seq vector to letters model

I'm looking to build a sequence-to-sequence model that takes in a 2048-long vector of 1s and 0s as my input and translating it to my known output of (a variable length) 1-20 long characters (ex. ...
2
votes
1answer
154 views

How do I classify an image that contains only polygons?

I have two closed polygons, drawn as connected straight black lines on a white background. I need to classify such images in to three forms Two separate polygons One polygon encloses the other The ...
1
vote
0answers
24 views

What are the benefits of Cross Stage Partial Connections over Residual Connections?

Cross Stage Partial Connections (CSPC) try to solve the next problems: Reduce the computations of the model in order to make it more suitable for edge devices. Reduce memory usage. Better ...
1
vote
0answers
12 views

Why identity mapping is so hard for deeper neural network as suggested by Resnet paper?

In resnet paper they said that a deeper network should not produce more error than its shallow counterpart since it can learn the identity map for the extra added layer. But empirical result shown ...
1
vote
0answers
41 views

Setting up a deep learning architecture for multi-dimensional data

The input data is thousands, millions of 4x1000 matrices. Each row consists of 3 small natural numbers (1000 combinations) and a corresponding real number between 0 and 1. The output is a 1x1000 ...
1
vote
0answers
30 views

How to restrain a model's outputs to a certain range without affecting its representative capacity?

CONTEXT I am trying to build a regression model that finds the optimal parameters for a given input. The data I am using are point clouds, with N points and ...
1
vote
0answers
17 views

Using numerical/categorical data and image data to detect objects

Let's say that I want to create a program capable of detecting lamps on some pictures. Those pictures can be, for instance, of a room, a street, etc. I would like to know if the following is possible: ...
1
vote
0answers
16 views

Dealing with bias in multi-channel auto encoders

The problem I have a multi-channel 1D signal I want to auto-encode. I am unable to resonstruct the input when the number of channels increases. Code I am using a convolutional encoder, and a ...
1
vote
0answers
24 views

Does the order of data augmentation and normalization matter?

What is the preferred order of data augmentation and normalization? Is it the former followed by the latter?
1
vote
0answers
18 views

Why does the relativistic discriminator increase the probability that generated data are real and decrease the probability that real data are real?

I was reading the ESRGAN whitepaper, where I came across this line: Relativistic discriminator [2] is developed not only to increase the probability that generated data are real but also to ...
1
vote
0answers
10 views

How to predict multiple set of coordinates (of bounding boxes) for signboards text localization through neural network?

I am creating a signboard translation application from scratch. I have images of signboards where there are multiple texts and I have the corresponding set of coordinates of bounding boxes for ...
1
vote
0answers
17 views

Underfitting a single batch: Can't cause autoencoder to overfit multi-sample batches of 1d data. How to debug?

TL;DR I am unable to overfit batches with multiple samples using autoencoder. Fully connected decoder seems to handle more samples per batch than conv decoder, but then also fails when number of ...
1
vote
0answers
16 views
1
vote
0answers
28 views

Are monotonically increasing functions easier to learn?

A monotonically increasing function is a function that as x gets bigger so does its output. So, if plotted, it will never go down. Although the outputs might stay constant. Logically this seems like ...
1
vote
0answers
21 views

variational auto encoder loss goes down but does not reconstruct input. out of debugging ideas

My variational autoencoder seems to work for MNIST, but fails on slightly "harder" data. By "fails" I mean there are at least two apparent problems: Very poor reconstruction, for ...
1
vote
0answers
26 views

What is the definition of pre-training?

I want to pre-train a model (combined by two popular modules A and B, and both are large blocks), then fine-tune it on downstream tasks. What if for the weight initialization for pre-training, module ...
1
vote
0answers
34 views

How is few-shot learning different from transfer learning?

To my understanding, transfer learning helps to incorporate data from other related datasets and achieve the task with less labelled data (maybe in 100s of images per category). Few-shot learning ...
1
vote
0answers
25 views

How do gradients are flown back into the Siamese network when branching is done?

I am curious about the working of a Siamese network. So, let us suppose I am using a triplet loss for my network and I have instantiated single CNN 3 times and there are 3 inputs to the network. So, ...
1
vote
0answers
26 views

How to derive compact convex set K and its diameter D to program Accelegrad algorithm in practice?

Given the original paper (https://arxiv.org/pdf/1809.02864.pdf), I would like to implement the Accelegrad algorithm for which I report the pseudocode of the paper: In the pseudocode, the authors ...
1
vote
0answers
22 views

Building a resume recommendation for a job post?

There are few challenges I am facing when building a resume recommendation for a particular job positing. Let's say we convert the resume into a vector on n-dimensions and job description also as an n-...
1
vote
0answers
51 views

Hairstyle Virtual Try On

I want to help people with cancer who are under chemotherapy, and generally people who have lost their hair to Virtually Try-On Toupees/Wigs on their head. VTO must support both the frontal and side ...
1
vote
0answers
31 views

Can I constrain my neurons in a neural network in according to the orders of the input?

I'm working with data that is ranked. So the inputs are 1,2,3 etc. This means the smaller numbers (ranks) are preferred to the larger ones. Hence the order is important. I want to estimate a number ...
1
vote
1answer
40 views

How to normalize for perceptual loss when training neural net from scratch?

Let's say we are training a new neural network from scratch. I calculate the mean and standard deviation of my dataset (assume I am training a fully convolutional neural net and my dataset is images) ...
1
vote
0answers
47 views

Stack of Planes as the Action Space Representation for AlphaZero (Chess)

I have a question regarding the action space of the policy network used in AlphaZero. From the paper: We represent the policy π(a|s) by a 8 × 8 × 73 stack of planes encoding a probability ...
1
vote
1answer
27 views

Why is Adam trapped in bad/suspicious local optima after the first few updates?

In the paper On the Variance of the Adaptive Learning Rate and Beyond, in section 2, the authors write To further analyze this phenomenon, we visualize the histogram of the absolute value of ...
1
vote
0answers
17 views

What is the intuition behind equations 10, 11 and 12 of the paper “Noise2Noise: Learning Image Restoration without Clean Data”?

Can anyone help me understand these functions described in the paper Noise2Noise: Learning Image Restoration without Clean Data I have read the portion A.4 in the appendix but need a more detailed and ...
1
vote
0answers
30 views

Why should variance(output) equal variance(input) in Xavier Initialisation?

In a lot of explanations online for Xavier Initialization, I see the following: With each passing layer, we want the variance to remain the same. This helps us keep the signal from exploding to a ...
1
vote
0answers
20 views

How significant is the decoder part of the capsule network?

Capsule Networks use an encoder-decoder structure, where the encoder part consists of the capsule layers (PrimiaryCaps and DigitCaps) and is also the part of the capsule network which performs the ...
1
vote
0answers
18 views

Object Detection as a means of Anomaly Detection

Is it possible to train an Object Detector (e.g. SSD), to detect when something is not in the image. Imagine an assembly line that transports some objects. Each object needs to have 5 screws. If the ...
1
vote
0answers
24 views

How does Google's 2016 GNMT architecture work?

I'm trying to read this paper describing Google's LSTM architecture for machine translation from 2016. However, I'm getting stuck as certain things are described too vaguely for me. This is a picture ...
1
vote
0answers
29 views

How is input defined for a biaxial lstm network for generating music?

I am reading Composing Music With Recurrent Neural Networks by Daniel D. Johnson. But I am really confused about the input passed to this network. If we pass notes of music along the time axis, then ...
1
vote
0answers
22 views

How can I improve the performance on unseen data for semantic segmentation using an auto-encoder?

I am using simple autoencoders for the task of semantic segmentation on the VOC2012 dataset. I am currently using a simple autoencoder based model. It is trained on adam optimizer with cross-entropy ...
1
vote
0answers
40 views

Where can I find pre-trained agents able to play games with multiple stages like exploration, dialog, combat?

My goal is to create an ML model to be able to classify different game stages, e.g., dialog with a non-player character, exploration, combat with enemy, in-game menu etc. In order to do that, I am ...
1
vote
0answers
35 views

In the MINE paper, why is $\hat{G}_B$ biased, and how does the exponential moving average reduce the bias?

While reading the Mutual Information Neural Estimation (MINE) paper [1] I came across section 3.2 Correcting the bias from the stochastic gradients. The proposed method requires the computation of the ...
1
vote
0answers
28 views

When training deep learning models for object detection in images, do you need a large number of images, or a large number of training samples?

I am training a deep learning model for object detection. The consensus is that the more images that you have, the better the results will be. All the tutorials that I have seen say that more images ...
1
vote
0answers
28 views

What is a “center loss”?

I have seen that a center loss is beneficial in computer vision, especially in face recognition. I have tried to understand this concept from the following material A Discriminative Feature Learning ...
1
vote
0answers
22 views

What exactly are deep learning primitives?

I came across the concept of "deep learning primitives" from the Nvidia talk Jetson AGX Xavier New Era Autonomous Machines (on slide 44). There doesn't seem to be a lot of articles in the ...
1
vote
0answers
29 views

WGAN-GP Loss formalization

I have to write the formalization of the loss function of my network, built following the WGAN-GP model. The discriminator takes 3 consecutive images as input (such as 3 consecutive frames of a video) ...
1
vote
0answers
21 views

REINFORCE Agent suddenly drops. How to verify if it's due to catastrophic forgetting?

I am using the default implementations of REINFORCE, DQN and c51 available from the tf.agents repo (links). As you can see, DQN manages to improve performance while REINFORCE seems to suffer from ...
1
vote
1answer
51 views

Single-Shot Learning for Object Re-Identification

I am looking for a way to re-identify/classify/recognize x real life objects (x < 50) with a camera. Each object should be presented to the AI only once for learning and there's always only one of ...
1
vote
0answers
92 views

What is the justification for Kaiming He initialization?

I've been trying to understand where the formulas for Xavier and Kaiming He initialization come from. My understanding is that these initialization schemes come from a desire to keep the gradients ...
1
vote
0answers
89 views

What is the computational complexity in terms of Big-O notation of a Gated Recurrent Unit Neural network?

I have been digging up of articles across the internet in context of computational complexity of GRU. Interestingly, I came across this article, http://cse.iitkgp.ac.in/~psraja/FNNs%20,RNNs%20,LSTM%...
1
vote
0answers
12 views

How Restricted Boltzman Machine (RBM) generates hand-written digit?

I am reading RBMs from this paper. In Fig1 they show an example of generating hand-written digit using RBMs. This is the figure they are showing: In the learning step first we sample $h$ from $h \sim ...
1
vote
0answers
22 views

Machine Learning Techniques for Objects Location/Orientation in Images

what Machine Learning tool can understand in which location and orientation a picture was taken from? That is from pictures of similar objects, say for example pictures of car interiors. So given a ...

1
3 4
5
6 7
13