# Questions tagged [deep-learning]

For questions related to deep learning, which refers to a subset of machine learning methods based on artificial neural networks (ANNs) with multiple hidden layers. The adjective deep thus refers to the number of layers of the ANNs. The expression deep learning was apparently introduced (although not in the context of machine learning or ANNs) in 1986 by Rina Dechter in the paper "Learning while searching in constraint-satisfaction-problems".

1,136 questions
Filter by
Sorted by
Tagged with
0answers
24 views

### Why do we need recurrent neural networks instead of feed-forward neural networks? [duplicate]

Why do we need recurrent neural networks instead of feed-forward neural networks? What are the advantages of RNNs compared with FFNNs?
0answers
5 views

### Identifying and Labeling multiple letters in image

While I attempt to learn AI/ML I have taken on the task to create a Boggle solver. The idea is that a system could take an image of a Boggle arrangement of letters and identify the letters (and the ...
0answers
7 views

### LSTM for imbalanced panel data

The available tutorials are most focused on time series prediction. I am wondering how shall we prepare the input data when it is an imbalanced data? Here is how data looks like. ...
1answer
22 views

### What is the main contribution of the paper Disentangling by Factorising?

Considering the paper Disentangling by Factorising, in addition to introducing a new model for Disentangled Representation Learning, FactorVAE (see figure), what is the main theoretical contribution ...
1answer
37 views

### What is the intuition behind the Xavier initialization for deep neural networks?

The aim of weight initialization is to prevent layer activation outputs from exploding or vanishing during the course of a forward pass through a deep neural network I am really having trouble ...
0answers
23 views

### Why is this GAN not converging?

This GAN being trained with CelebA dataset doesn't seem to mode collapse, discriminator is not really over confident, and yet the quality is stuck on these rough Picasso-like generator images. Using ...
0answers
18 views

### Deep learning techniques with time-fixed, time-dependent and imaging data

I have a question about the use of deep learning techniques with time-fixed features and images (setting 1) and time-dependent features (setting 2). (I am pretty new to the deep learning world so ...
0answers
17 views

### Incorporating domain knowledge into recurrent network

I am currently trying to solve a classification task with a recurrent artificial neural network (RNN). Situation There are up to 350 inputs (X) mapped on one categorical output (y)(13 differnt ...
0answers
8 views

### Flatten in simple feedforward networks [migrated]

I am working on the CIFAR10 dataset and came across this example in Keras, using data augmentation: https://keras.io/examples/cifar10_cnn/ The example uses CNN. I want to implement just a simple ...
0answers
9 views

### Training Conditional DCGAN with GAN-CLS loss

I am trying to implement conditional GAN using GAN-CLS loss as described in paper: https://arxiv.org/abs/1605.05396 So, while training discriminator, I should I have three batches of data: [...
1answer
45 views

### How and when should we update the Q-target in deep Q-learning?

I have recently watched David silver's course, and started implementing the deep Q-learning algorithm. I thought I should make a switch between the Q-target and Q-current directly (meaning, every ...
0answers
18 views

### Can you find another reason for sample inefficiency of model-free on-policy Deep Reinforcement Learning?

The following mindmap gives an overview of multiple reasons for sample inefficiency. The list is definitely not complete. Can you see another reason not mentioned so far? Some related links: ...
0answers
12 views

### Is the number of layers in the simple RNN fixed or is it random? [closed]

When I search for RNN, I find LSTM most of the times. Before I go on o for reading more about LSTM, I want to explore the vanilla RNN. I want to know an explanation of working with mathematical ...
1answer
49 views

### How would researchers determine the best deep learning model if every run of the code yields different results?

There are many factors that cause the results of ML models to be different for every run of the same piece of code. One factor could be different initialization of weights in the neural network. ...
1answer
38 views

### Can we increase the speed of training a reinforcement learning algorithm?

I am new in reinforcement learning. I started reading the PyTorch's documentation about the cart pole control. Whenever an agent fails, they restart the environment. When I run the code, the time in ...
1answer
90 views

### Why do we need convolutional neural networks instead of feed-forward neural networks?

Why do we need convolutional neural networks instead of feed-forward neural networks? What is the significance of a CNN? Even a feed-forward neural network will able to solve the image classification ...
1answer
34 views

### Learning policy where action involves discrete and continuous parameters

Typically it seems like reinforcement learning involves learning over either a discrete or a continuous action space. An example might be choosing from a set of pre-defined game actions in Gym Retro ...
1answer
52 views

### What is the intuition behind the attention mechanism?

Attention idea is one of the most influential ideas in deep learning. The main idea behind attention technique is that it allows the decoder to "look back” at the complete input and extracts ...
0answers
81 views

### Can deep learning be used to help mathematical research?

I am currently learning about deep learning and artificial intelligence and exploring his possibilities, and, as a mathematician at heart, I am inquisitive about how it can be used to solve problems ...
0answers
10 views

### How to set only the modified weights for each convolutional layers? [migrated]

I am currently doing some experiments on modifying the weights and not of the bias for each convolutional layers of a model. For each of the layers of the model, I used ...
1answer
23 views

### What is the advantage of using Google's Coral over Nvidia's Xavier?

I was reading about the possibility of using Google's Coral for deep learning-based object detection and image classification. I heard it has a good speed in terms of frames/sec. I also read that ...
1answer
64 views

### How does publishing in the deep learning world work, with respect to journals and arXiv?

Let's say I implemented a new deep learning model that pushed some SOTA a little bit further, and I wrote a new paper about for publication. How does it work now? I pictured three options: Submit it ...
1answer
28 views

### Can I find a mapping that minimizes the maximum distance ratio of certain vectors?

Let's say we have several vector points. My goal is to distinguish the vectors, so I want to make them far from each other. Some of them are already far from each other, but some of them can be ...
1answer
45 views

### How does the repetition of features across states at different time steps affect learning?

Let's say you are training a neural network in an RL setting, where the state (i.e. features/input data) can be the same for multiple successive steps (~typically around 8 steps) of an episode. For ...
0answers
17 views

### How does sampling works in case of imbalanced image datasets?

I am solving a problem of image classification of the image dataset for 3 classes. Dataset is highly imbalanced. How will sampling (either over- or under-sampling) work in that case? Should I remove (...
0answers
22 views

### Where does reinforcement learning actually show up in Deepmind's game engines?

From the brief research I've done on the topic, it appears that the way Deepmind's Alphazero or Muzero makes decisions is through Monte Carlo tree searches, where in the randomized simulations allows ...
1answer
84 views

### What is the difference between deep learning and shallow learning?

What is the difference between deep learning and shallow learning? What I am interested in knowing is not the definition of deep learning & shallow learning, but understanding the actual ...
0answers
18 views

### How is visual attention mechanism different from a two branch convolutional neural network?

I am doing some research on the visual attention mechanism in remote sensing domain (where the features learnt from one layer are highlighted using the attention mask derived from another layer). From ...
0answers
11 views

CycleGAN can map between two different distributions $X$ and $Y$ with cycle consistency. This is done with generator functions $F: X \mapsto Y$ and $G: Y \mapsto X$, such that $||G(F(x)) - x||_1 \... 1answer 95 views ### How do I calculate the partial derivative with respect to$x\$?

I am trying to implement CNN using python Numpy. I searched so much, but all I found was for one filter with one channel for Convolution. Suppose we have an X as Image with this shape: ...
0answers
19 views

### Should we start with a small batch-size and increase during training to improve sample efficiency?

Just made an interesting observation playing around with the stable-baseline's implementation of PPO and the BipedalWalker environment from OpenAI's Gym. But I believe this should be a general ...
1answer
45 views

### How to evaluate a Deep Q-Network

Good day, it's a pleasure having joined this Stack. In my master thesis I have to expand a Deep Reinforcement Learning Network, to be precise a Deep Q-Network, which is used to control machines in an ...
3answers
4k views

### Why do most deep learning papers not include an implementation?

I'm a novice researcher, and as I started to read papers in the area of deep learning I noticed that the implementation is normally not added and is needed to be searched elsewhere, and my question is ...
1answer
54 views

### How is the Jacobian a generalisation of the gradient?

I came across these slides Natural Language Processing with Deep Learning CS224N/Ling284, in the context of natural language processing, which talk about the Jacobian as a generalization of the ...
2answers
33 views

### Calculating accuracy for cross validation

I'm struggling with calculating accuracy when I do cross-validation for a deep learning model. I have two candidates for doing this. 1. Train a model with 10 different folds and get the best accuracy ...
1answer
13 views

### What are non-held-out data or non-held-out classes?

I'm Spanish and I don't understand the meaning of "non-held-out". I have tried Google Translator and online dictionaries like Longman but I can't find a suitable translation for this term. You can ...
0answers
13 views

### How's the action represented in MuZero for Atari?

MuZero seems to use two different methods to encode actions into planes for Atari games: For the input action to the representation function, MuZero encodes historical actions as simple bias planes, ...
0answers
11 views

### Number of parameters in Resnet-50 [migrated]

I'm using Keras, and I am struggling to know how many parameters Resnet-50 has. Keras documentation says around 25M, while if I ...
0answers
16 views

### Is the high dimensionality of input vectors a problem for a radial basis function neural network?

I have a dataset A of videos. I've extracted the feature vector of each video (with a convolutional neural network, via transfer learning) creating a dataset B. Now, every vector of the dataset B has ...
1answer
41 views

### Simplification of expected reward under the limit in continuous tasks

I was reading the average reward setting for continuous tasks from rich sutton's book (page 202, 2nd edition). There he perform a simplification over the expected reward under the limit approaching to ...
0answers
6 views

### How to feed decimal_y_values for training where last layer of model has 10 neurons with softmax activation and loss is Earth Mover Loss [migrated]

What should be the format/data types of y-labels for training if the actual y-labels cab be any decimal number between 0-9 (4.1,8.5 etc) and the last output layer is defined as: ...
2answers
56 views

### How to implement AI strategy for Mastermind

I'm looking to implement a AI for the turn-based game Mastermind in Node.JS, using Google's Tensorflow library. Basically the AI needs to predict the 4D input for the optimal 2D output ...
0answers
7 views

### What is the advantages of using FPGA for deep learning- computer vision task? [migrated]

I noticed some developers prefer to use FPGA like XILINX for their deep learning applications, why they prefer to use FPGA instead of GPU?! Is there any reason for that? I suppose GPUs are very ...
0answers
17 views

### An Encoder-Decoder based CNN to predict a tensor of points

So I have with me a data of rendered 2D images of a 3D object and along with that, I have the image projection coordinates (X, Y) of all the voxels that are in the ...
0answers
19 views

### Can SqueezeNet be used for regression?

I want a model that outputs the pixel coordinates of the tip of my forefinger, and whether it's touching something or not. Those would be 3 output neurons: 2 for the X-Y coordinates and 1, with a ...
2answers
50 views

### Combine two feature vectors for a correct input of a neural network

Let's consider this scenario. I have two conceptually different video datasets, for example a dataset A composed of videos about cats and a dataset B composed of videos about houses. Now, I'm able to ...
0answers
34 views

### Can a typical supervised learning problem be solved with reinforcement learning methods?

Let's say I want to teach a neural to classify images, and, for some reason, I insist on using reinforcement learning rather than supervised learning. I have a dataset of images and their matching ...
0answers
20 views

### Backward pass of CNN like Resnet: how to manually compute flops during backprop?

I've been trying to figure out how to compute the number of Flops in backward pass of ResNet. For forward pass, it seems straightforward: apply the conv filters to the input for each layer. But how ...
1answer
36 views

### Accuracy dropped when I ran the program the second time

I was following a tutorial about Feed Forward Networks and wrote this code for a simple FFN : ...
3answers
101 views

### What are some resources with exercises related to neural networks?

I am asking for a book (or any other online resource) where we can solve exercises related to neural networks, similar to the books or online resources dedicated to mathematics where we can solve ...