Questions tagged [deep-neural-networks]

For questions related to deep neural networks, which are artificial neural networks with "many" layers, where "many" can vary depending on the context.

Filter by
Sorted by
Tagged with
0 votes
0 answers
35 views

Problem about Xavier Initialization [closed]

I'm learning about Xavier initialization and reading the paper "Understanding the difficulty of training deep feedforward neural networks": https://www.researchgate.net/publication/...
0 votes
1 answer
52 views

How vision models based on CNNs learn the relative positions of each pixel in the image?

A CNN model is based on a series of filters applied to an image. However, these filters can only "see" a small portion of the image and they have no information of the relative position of ...
0 votes
1 answer
40 views

What should I do, reinforcement learning agent gives different result on every train?

I'm using PPO+LSTM to create a trading bot. The agent is trained on 3 years of data and tested on 1 year. Every time I train the agent with same set of hyper-parameters, I get very different results ...
0 votes
0 answers
18 views

How to improve classification accuracy in TF deep neural network model?

I need help in increasing the accuracy of a classification model using Neural Networks on Tensorflow. I am trying to train a model on sequential data ...
1 vote
0 answers
53 views

What does Deep Q-Learning (DQL) do?

Hello :) I'm required to write a document where I describe what DQL does in short. This is what I wrote: DQL: instead of a Q-table, a DNN is used to approximate the Q-values for each action based on a ...
  • 166
1 vote
1 answer
89 views

Formally, what are the layers in an Artificial Neural Network?

You may not believe it, but I am an ANN expert. Perhaps, for that reason, I am unable to grasp completely what the layers are in a Deep Forward Artificial Neural Network (DFANN). According to the Deep ...
  • 11
0 votes
0 answers
17 views

Is it possible to create a distance estimation model from something like the KITTI dataset?

I am unsure about how to word this question correctly, edits appreciated. I am trying to create a neural network model that can predict distance from camera feed. And I am doing it by feeding actual ...
0 votes
1 answer
61 views

How to handle the size difference of highway network or residual network in cnn?

For highway network, it looks like this: For residual network, it looks like this: Pictures are from What is the name of this neural network architecture with layers that are also connected to non-...
0 votes
0 answers
27 views

What are the inputs of a neural network when learning a difference equation?

The time series y[n] is the solution of the difference equation ...
0 votes
0 answers
38 views

Does the number of epochs measure a correlation?

i have built a two-layers neural network (1000 => 1000) to predict a dynamical system driven by two real-world parameters. When using the first parameter as input to the first layer, training the ...
0 votes
0 answers
33 views

Does it make sense to build a two-layer neural network with a triangular weight matrix?

I need to implement a rule and have defined a lower triangular boolean mask for the weights that I want to keep static for a zero value. In which condition triangular weight matrix will be used?
0 votes
0 answers
14 views

What are the best ways to preprocess landmarks to train a Neural Network?

I'd like to know, generally speaking, which are the most useful ways to preprocess landmarks to use as training set for a Deep Neural Network. Since they're put in a 3D space, would that be enough ...
  • 1
2 votes
2 answers
256 views

Why should one expect the backward pass to take twice as long as the forward pass?

I have seen it stated that, as a rule of thumb, a backward pass in a neural network should take about twice as long as the forward pass. Examples: From DeepSpeed's Flops Profiler docs, the profiler: ...
  • 121
0 votes
1 answer
95 views

Is it redunant to add more layers to a neural network with same number of neurons as the previous layer?

Lets say I have a neural network with three layers and the last layer has 3 outputs. If I added additional layer of 3 neurons to the end of the network, would that be a more powerful neural network? ...
0 votes
0 answers
34 views

Camera specific dataset for training CLRnet model

I am using CLRnet to generate a model to do Lane Detection. I will be having proprietry camera(fish eye) on which the model will work. The idea is to train a model for lane detectection using CLRNet, ...
1 vote
1 answer
216 views

Why MSE and MAE yield poor results when used with gradient-based optimization for classification?

Deep learning book chapter 6: In 6.2.1.2 last paragraph: Unfortunately, mean squared error and mean absolute error often lead to poor results when used with gradient-based optimization. Some output ...
0 votes
3 answers
113 views

why validation accuracy be greater than training accuracy for deep learning models? [closed]

I hope you are well. I had a problem and didn't understand the answers given on questions similar to my question. If possible, please answer this problem in a simpler way. Val_acc : %99.4 _ Train_acc :...
1 vote
1 answer
50 views

MobileNetV2 - Some particularities

So I was studying MobileNetV2 architecture and came across this table from the original paper that represents its architecture: Table Description: "Table 2: MobileNetV2 : Each line describes a ...
0 votes
1 answer
60 views

Training a neural network in full batch training

It is a trend in deep learning to train models using multi-batches, i.e., to show the model a subset of the entire dataset for each weight update. In some cases, as in continual learning, we see that ...
  • 165
1 vote
4 answers
139 views

Why, in deep learning, do we get computational power by going deeper?

I know by the expressiveness of a neural networks that it can be seen as a chain of function compositions, i.e. $g(f(.. z(x)..))$ and also that, if we go deep, we can approximate complex functions $f: ...
  • 11
0 votes
0 answers
35 views

How feasible is it to partition a DNN model into functional pieces?

Just read Auto-Split: A General Framework of Collaborative Edge-Cloud AI by a group of Huawei researchers (https://arxiv.org/pdf/2108.13041.pdf). How feasible is it to break up the models and serve ...
0 votes
0 answers
18 views

Difference between training algorithms

I'm using GPS Data for my Total electron Content (TEC) Prediction, for which I'm using Non-linear Autoregressive with External (Exogenous) Input (NARX) Model. My question is what's the difference ...
1 vote
0 answers
34 views

Why "Good Model" that performs great on holdout validation data fails on production data

I have this binary regression model that has ~500 futures with an unbalanced dataset with the following results. ...
  • 23
1 vote
1 answer
96 views

How to handle out-of-bound values in Production data?

So I have this model but the data may vary. And it is virtually impossible to always have the values in bounds. If I do I`d have to use larger period leading to concept shift which is worse. The ...
  • 23
1 vote
0 answers
51 views

How can I get an integer as output for continuous action space PPO reinforcement learning?

I have a huge discrete action space, the learning stability is not good. I'd like to move to continuous action space but the only output for my task can be a positive integer (let's say in the range 0 ...
  • 111
0 votes
0 answers
35 views

ML algorithm to estimate height and breadth of building like structure from 2D image

Following is my image in the question. What ML algorithm will help me to estimate the height and breadth of building like towers in the image. The blue painted buildings that look like shadow is the ...
1 vote
1 answer
22 views

How to use information on a function to design a neural network learning that function?

I have a function $g$ that takes a vector $x$ of size $n$ and an integer $k$ in $1, \ldots, n$. I know this function is of the form $$g(x,k) = G\left(\sum_{i=1}^k f(x_{i})\right),$$ where $f$ and $G$ ...
  • 111
1 vote
1 answer
34 views

How to identify important features in data?

I have a couple opportunities to write a paper, or papers over some of the neural networks I have made. I was wondering if there are anyways to figure out why the neural network classifies the data I ...
1 vote
0 answers
23 views

How Can We Create Neural Networks with Different Depths and Widths But Same Number of Parameters?

Right now I am doing a research project investigating how the depth of a Neural Network affects its capacity to learn. In order to do this, I wanted to test different Networks with the same number of ...
0 votes
1 answer
153 views

How to compare memory requirements for tabular Q-learning vs deep neural network?

I want to compare the space complexity/memory requirement of tabular Q-learning v.s. deep neural Q-network (DQN). I think DQN would be faster and Q-table has a disadvantage at large table sizes but ...
1 vote
1 answer
41 views

How to preserve Markov Property in Deep Reinforcement Learning when using "mixup" or "mixreg"?

I've read through these two papers: (original about "mixup") https://arxiv.org/pdf/1710.09412.pdf (variant for RL, "mixreg") https://arxiv.org/pdf/2010.10814.pdf They are about a ...
0 votes
1 answer
245 views

Why gradients are used in Layer-wise Relevance Propagation (LRP)?

To give you an overview, Layer-wise Relevance Propagation is a technique by which we can get relevance values at each node of the neural network. These calculated relevance values (per node) are ...
  • 111
2 votes
1 answer
449 views

Should I use an unsupervised approach or train a classifier with many classes to build a deep image feature extractor?

I'd like to build a deep feature extractor of images (using a Bi-linear CNN). What would lead to the best results: an unsupervised approach (such as https://iopscience.iop.org/article/10.1088/1742-...
0 votes
0 answers
416 views

What is the time complexity of Deep VGG-16 Net?

I would like to compute the time complexity (in Big O notation) of Deep Vgg16 Net by feeding forward 1000 images with size 224x224x3 and retrieving features from the second fully connected layer 'fc7' ...
2 votes
0 answers
769 views

How many layers do GPT-3, AlphaFold 2, and DALL-E 2 have?

Unsuccessfully, I tried to find out the "depth" (definition below) in large neural networks such as GPT-3, AlphaFold 2, and DALL-E 2. Formally, my question is about their computational graph:...
1 vote
0 answers
160 views

How to define a custom layer in Pytorch [closed]

I am new to PyTorch and seeking your help regarding a problem I have. I need to add a costume layer to a NN in training phase. Please see the figure which shows a simple DNN with the custom layer. NN ...
  • 11
2 votes
1 answer
31 views

Why does backprop algorithm store the inputs to the non-linearity of the hidden layers?

I have been reading the Deep Learning book by Ian Goodfellow and it mentions in Section 6.5.7 that The main memory cost of the algorithm is that we need to store the input to the nonlinearity of the ...
-1 votes
1 answer
224 views

What is exactly sparse annotation?

What is exactly sparse annotation? Is it different from labeling images? I've been reading a paper about vessel segmentation and have some issues understanding this part.
1 vote
0 answers
31 views

References for Nvidia's DLSS

Nvidia's deep learning super-sampling is presented as an application of deep learning techniques to video-rendering in videogames. Question: I'm asking for a technical reference that explains what is ...
1 vote
1 answer
781 views

Using "softmax" (non-linear) vs "linear" activation function in Deep Reinforcement Learning

I am following the tutorial in this video: https://youtu.be/cO5g5qLrLSo which implements deep reinforcement learning (DQN) to balance cart pole in OpenAI default environment. The DQN model looks like ...
0 votes
2 answers
457 views

How to count overlapping objects with neural networks

Consider the following task to be solved by a neural network: Given a $N\times N$ pixel grid with up to $M$ objects drawn on it, either squares (9 pixels) or diamonds (5 pixels): square    diamond The ...
1 vote
0 answers
46 views

Output representation for a neural network to learn grid-based game with multiple units

I have a round based game played on a grid map with multiple units that I would like to control in some fashion using neural network (NN). All of the units are moved at once. Each unit can move in any ...
  • 159
0 votes
0 answers
53 views

If the model always underfits, do I really need a larger model?

I train my neural network on random points generated for a data set that theoretically consists of approximately $1.8 * 10^{39}$ elements. I sample (generate) tens of thousands of random points on ...
3 votes
0 answers
199 views

Are there neural networks with (hard) constraints on the weights?

I don't know too much about Deep Learning, so my question might be silly. However, I was wondering whether there are NN architectures with some hard constraints on the weights of some layers. For ...
  • 173
1 vote
1 answer
271 views

Multi label classification on non binary labels with pytorch

I am working on a project consisting of medical images and a huge dataset of multi-label and non-binary labels/outcomes ( sex, blood pressure, age and 40 more ). Would be the best approach to hard ...
1 vote
1 answer
136 views

Are there any animation tools available to visualise and simulate deep neural networks? [closed]

Deep learning researchers have to work with a lot of models. The models may include different types of Layers: They include convolutional neural network layers, recurrent neural network layers, batch ...
  • 3,491
2 votes
0 answers
338 views

How does back propagation adjust the hidden layers' weights and biases?

I'm new to neural networks and trying to figure out its fundamentals but I cannot fully understand the back propagation algorithm. In back propagation, I understand we want to go backwards from the ...
0 votes
1 answer
298 views

Dissection of a depth map

I am curious about how depth maps work. While searching I came across this website which contains some images and their depth maps. I took this depth map and tried to study it using a python pillow. <...
  • 1,006
0 votes
0 answers
347 views

Computational complexity of a CNN network

In the following network, the convolution operations of convolutional blocks are performed by three 1-D kernels with the sizes 8, 5, and 3 respectively along with stride equal to 1. The final network ...
1 vote
1 answer
78 views

Would this count as a Transfer Learning approach?

I have two datasets, Dataset 1(D1) and Dataset 2(D2). D1 has around 22000 samples, and D2 has around 8000 samples. What I am doing is that I train a Deep Neural Network model with around three layers ...