Questions tagged [deep-neural-networks]
For questions related to deep neural networks, which are artificial neural networks with "many" layers, where "many" can vary depending on the context.
177
questions
0
votes
0
answers
35
views
Problem about Xavier Initialization [closed]
I'm learning about Xavier initialization and reading the paper "Understanding the difficulty of training deep feedforward neural networks": https://www.researchgate.net/publication/...
- 1
0
votes
1
answer
52
views
How vision models based on CNNs learn the relative positions of each pixel in the image?
A CNN model is based on a series of filters applied to an image. However, these filters can only "see" a small portion of the image and they have no information of the relative position of ...
- 63
0
votes
1
answer
40
views
What should I do, reinforcement learning agent gives different result on every train?
I'm using PPO+LSTM to create a trading bot. The agent is trained on 3 years of data and tested on 1 year. Every time I train the agent with same set of hyper-parameters, I get very different results ...
- 1
0
votes
0
answers
18
views
How to improve classification accuracy in TF deep neural network model?
I need help in increasing the accuracy of a classification model using Neural Networks on Tensorflow.
I am trying to train a model on sequential data ...
- 437
1
vote
0
answers
53
views
What does Deep Q-Learning (DQL) do?
Hello :) I'm required to write a document where I describe what DQL does in short.
This is what I wrote:
DQL: instead of a Q-table, a DNN is used to approximate the Q-values for each action based on a ...
- 166
1
vote
1
answer
89
views
Formally, what are the layers in an Artificial Neural Network?
You may not believe it, but I am an ANN expert. Perhaps, for that reason, I am unable to grasp completely what the layers are in a Deep Forward Artificial Neural Network (DFANN).
According to the Deep ...
- 11
0
votes
0
answers
17
views
Is it possible to create a distance estimation model from something like the KITTI dataset?
I am unsure about how to word this question correctly, edits appreciated.
I am trying to create a neural network model that can predict distance from camera feed. And I am doing it by feeding actual ...
0
votes
1
answer
61
views
How to handle the size difference of highway network or residual network in cnn?
For highway network, it looks like this:
For residual network, it looks like this:
Pictures are from What is the name of this neural network architecture with layers that are also connected to non-...
- 53
0
votes
0
answers
27
views
What are the inputs of a neural network when learning a difference equation?
The time series y[n] is the solution of the difference equation
...
0
votes
0
answers
38
views
Does the number of epochs measure a correlation?
i have built a two-layers neural network (1000 => 1000) to predict a dynamical system driven by two real-world parameters.
When using the first parameter as input to the first layer, training the ...
0
votes
0
answers
33
views
Does it make sense to build a two-layer neural network with a triangular weight matrix?
I need to implement a rule and have defined a lower triangular boolean mask for the weights that I want to keep static for a zero value. In which condition triangular weight matrix will be used?
0
votes
0
answers
14
views
What are the best ways to preprocess landmarks to train a Neural Network?
I'd like to know, generally speaking, which are the most useful ways to preprocess landmarks to use as training set for a Deep Neural Network. Since they're put in a 3D space, would that be enough ...
- 1
2
votes
2
answers
256
views
Why should one expect the backward pass to take twice as long as the forward pass?
I have seen it stated that, as a rule of thumb, a backward pass in a neural network should take about twice as long as the forward pass. Examples:
From DeepSpeed's Flops Profiler docs, the profiler:
...
- 121
0
votes
1
answer
95
views
Is it redunant to add more layers to a neural network with same number of neurons as the previous layer?
Lets say I have a neural network with three layers and the last layer has 3 outputs.
If I added additional layer of 3 neurons to the end of the network, would that be a more powerful neural network?
...
- 211
0
votes
0
answers
34
views
Camera specific dataset for training CLRnet model
I am using CLRnet to generate a model to do Lane Detection. I will be having proprietry camera(fish eye) on which the model will work.
The idea is to train a model for lane detectection using CLRNet, ...
1
vote
1
answer
216
views
Why MSE and MAE yield poor results when used with gradient-based optimization for classification?
Deep learning book chapter 6: In 6.2.1.2 last paragraph:
Unfortunately, mean squared error and mean absolute error often lead to poor results when used with gradient-based optimization. Some output ...
- 13
0
votes
3
answers
113
views
why validation accuracy be greater than training accuracy for deep learning models? [closed]
I hope you are well.
I had a problem and didn't understand the answers given on questions similar to my question.
If possible, please answer this problem in a simpler way.
Val_acc : %99.4 _
Train_acc :...
1
vote
1
answer
50
views
MobileNetV2 - Some particularities
So I was studying MobileNetV2 architecture and came across this table from the original paper that represents its architecture:
Table Description: "Table 2: MobileNetV2 : Each line describes a ...
- 17
0
votes
1
answer
60
views
Training a neural network in full batch training
It is a trend in deep learning to train models using multi-batches, i.e., to show the model a subset of the entire dataset for each weight update. In some cases, as in continual learning, we see that ...
- 165
1
vote
4
answers
139
views
Why, in deep learning, do we get computational power by going deeper?
I know by the expressiveness of a neural networks that it can be seen as a chain of function compositions, i.e. $g(f(.. z(x)..))$ and also that, if we go deep, we can approximate complex functions $f: ...
- 11
0
votes
0
answers
35
views
How feasible is it to partition a DNN model into functional pieces?
Just read Auto-Split: A General Framework of Collaborative Edge-Cloud AI by a group of Huawei researchers (https://arxiv.org/pdf/2108.13041.pdf). How feasible is it to break up the models and serve ...
0
votes
0
answers
18
views
Difference between training algorithms
I'm using GPS Data for my Total electron Content (TEC) Prediction, for which I'm using Non-linear Autoregressive with External (Exogenous) Input (NARX) Model.
My question is what's the difference ...
- 101
1
vote
0
answers
34
views
Why "Good Model" that performs great on holdout validation data fails on production data
I have this binary regression model that has ~500 futures with an unbalanced dataset with the following results.
...
- 23
1
vote
1
answer
96
views
How to handle out-of-bound values in Production data?
So I have this model but the data may vary. And it is virtually impossible to always have the values in bounds. If I do I`d have to use larger period leading to concept shift which is worse.
The ...
- 23
1
vote
0
answers
51
views
How can I get an integer as output for continuous action space PPO reinforcement learning?
I have a huge discrete action space, the learning stability is not good. I'd like to move to continuous action space but the only output for my task can be a positive integer (let's say in the range 0 ...
- 111
0
votes
0
answers
35
views
ML algorithm to estimate height and breadth of building like structure from 2D image
Following is my image in the question.
What ML algorithm will help me to estimate the height and breadth of building like towers in the image. The blue painted buildings that look like shadow is the ...
- 71
1
vote
1
answer
22
views
How to use information on a function to design a neural network learning that function?
I have a function $g$ that takes a vector $x$ of size $n$ and an integer $k$ in $1, \ldots, n$. I know this function is of the form
$$g(x,k) = G\left(\sum_{i=1}^k f(x_{i})\right),$$ where $f$ and $G$ ...
- 111
1
vote
1
answer
34
views
How to identify important features in data?
I have a couple opportunities to write a paper, or papers over some of the neural networks I have made.
I was wondering if there are anyways to figure out why the neural network classifies the data I ...
- 11
1
vote
0
answers
23
views
How Can We Create Neural Networks with Different Depths and Widths But Same Number of Parameters?
Right now I am doing a research project investigating how the depth of a Neural Network affects its capacity to learn. In order to do this, I wanted to test different Networks with the same number of ...
- 11
0
votes
1
answer
153
views
How to compare memory requirements for tabular Q-learning vs deep neural network?
I want to compare the space complexity/memory requirement of tabular Q-learning v.s. deep neural Q-network (DQN). I think DQN would be faster and Q-table has a disadvantage at large table sizes but ...
1
vote
1
answer
41
views
How to preserve Markov Property in Deep Reinforcement Learning when using "mixup" or "mixreg"?
I've read through these two papers:
(original about "mixup") https://arxiv.org/pdf/1710.09412.pdf
(variant for RL, "mixreg") https://arxiv.org/pdf/2010.10814.pdf
They are about a ...
- 312
0
votes
1
answer
245
views
Why gradients are used in Layer-wise Relevance Propagation (LRP)?
To give you an overview, Layer-wise Relevance Propagation is a technique by which we can get relevance values at each node of the neural network. These calculated relevance values (per node) are ...
- 111
2
votes
1
answer
449
views
Should I use an unsupervised approach or train a classifier with many classes to build a deep image feature extractor?
I'd like to build a deep feature extractor of images (using a Bi-linear CNN).
What would lead to the best results:
an unsupervised approach (such as https://iopscience.iop.org/article/10.1088/1742-...
0
votes
0
answers
416
views
What is the time complexity of Deep VGG-16 Net?
I would like to compute the time complexity (in Big O notation) of Deep Vgg16 Net by feeding forward 1000 images with size 224x224x3 and retrieving features from the second fully connected layer 'fc7' ...
2
votes
0
answers
769
views
How many layers do GPT-3, AlphaFold 2, and DALL-E 2 have?
Unsuccessfully, I tried to find out the "depth" (definition below) in large neural networks such as GPT-3, AlphaFold 2, and DALL-E 2.
Formally, my question is about their computational graph:...
- 29
1
vote
0
answers
160
views
How to define a custom layer in Pytorch [closed]
I am new to PyTorch and seeking your help regarding a problem I have. I need to add a costume layer to a NN in training phase. Please see the figure which shows a simple DNN with the custom layer. NN ...
- 11
2
votes
1
answer
31
views
Why does backprop algorithm store the inputs to the non-linearity of the hidden layers?
I have been reading the Deep Learning book by Ian Goodfellow and it mentions in Section 6.5.7 that
The main memory cost of the algorithm is that we need to store the input to the nonlinearity of the ...
- 143
-1
votes
1
answer
224
views
What is exactly sparse annotation?
What is exactly sparse annotation? Is it different from labeling images?
I've been reading a paper about vessel segmentation and have some issues understanding this part.
1
vote
0
answers
31
views
References for Nvidia's DLSS
Nvidia's deep learning super-sampling is presented as an application of deep learning techniques to video-rendering in videogames.
Question: I'm asking for a technical reference that explains what is ...
- 133
1
vote
1
answer
781
views
Using "softmax" (non-linear) vs "linear" activation function in Deep Reinforcement Learning
I am following the tutorial in this video: https://youtu.be/cO5g5qLrLSo which implements deep reinforcement learning (DQN) to balance cart pole in OpenAI default environment.
The DQN model looks like ...
- 111
0
votes
2
answers
457
views
How to count overlapping objects with neural networks
Consider the following task to be solved by a neural network: Given a $N\times N$ pixel grid with up to $M$ objects drawn on it, either squares (9 pixels) or diamonds (5 pixels):
square
diamond
The ...
1
vote
0
answers
46
views
Output representation for a neural network to learn grid-based game with multiple units
I have a round based game played on a grid map with multiple units that I would like to control in some fashion using neural network (NN). All of the units are moved at once. Each unit can move in any ...
- 159
0
votes
0
answers
53
views
If the model always underfits, do I really need a larger model?
I train my neural network on random points generated for a data set that theoretically consists of approximately $1.8 * 10^{39}$ elements. I sample (generate) tens of thousands of random points on ...
- 53
3
votes
0
answers
199
views
Are there neural networks with (hard) constraints on the weights?
I don't know too much about Deep Learning, so my question might be silly. However, I was wondering whether there are NN architectures with some hard constraints on the weights of some layers. For ...
- 173
1
vote
1
answer
271
views
Multi label classification on non binary labels with pytorch
I am working on a project consisting of medical images and a huge dataset of multi-label and non-binary labels/outcomes ( sex, blood pressure, age and 40 more ).
Would be the best approach to hard ...
- 11
1
vote
1
answer
136
views
Are there any animation tools available to visualise and simulate deep neural networks? [closed]
Deep learning researchers have to work with a lot of models. The models may include different types of Layers: They include convolutional neural network layers, recurrent neural network layers, batch ...
- 3,491
2
votes
0
answers
338
views
How does back propagation adjust the hidden layers' weights and biases?
I'm new to neural networks and trying to figure out its fundamentals but I cannot fully understand the back propagation algorithm.
In back propagation, I understand we want to go backwards from the ...
- 21
0
votes
1
answer
298
views
Dissection of a depth map
I am curious about how depth maps work. While searching I came across this website which contains some images and their depth maps. I took this depth map and tried to study it using a python pillow.
<...
- 1,006
0
votes
0
answers
347
views
Computational complexity of a CNN network
In the following network, the convolution operations of convolutional blocks are performed by three 1-D kernels with the sizes 8, 5, and 3 respectively along with stride equal to 1. The final network ...
- 1
1
vote
1
answer
78
views
Would this count as a Transfer Learning approach?
I have two datasets, Dataset 1(D1) and Dataset 2(D2). D1 has around 22000 samples, and D2 has around 8000 samples. What I am doing is that I train a Deep Neural Network model with around three layers ...
- 133