Last call to make your voice heard! Our 2022 Developer Survey closes in less than a week. Take survey.

Questions tagged [deep-neural-networks]

For questions related to deep neural networks, which are artificial neural networks with "many" layers, where "many" can vary depending on the context.

Filter by
Sorted by
Tagged with
0 votes
0 answers
38 views

How to choose window size in convolutional layers? [closed]

I have a built a deep learning network for a specific biomedical application. I have used 5x5 convolution in my network. I read that successive 3x3 convolution layers instead of one layer having ...
user avatar
  • 105
0 votes
1 answer
18 views

Why gradients are used in Layer-wise Relevance Propagation (LRP)?

To give you an overview, Layer-wise Relevance Propagation is a technique by which we can get relevance values at each node of the neural network. These calculated relevance values (per node) are ...
user avatar
  • 111
2 votes
1 answer
39 views

Deep Features of Images - Better to use an unsupervised approach, or train a classifier with many classes?

I'd like to build a deep feature extractor of images (using a Bi-linear CNN). I was wondering what would lead to the best results: An unsupervised approach (such as https://iopscience.iop.org/article/...
user avatar
0 votes
0 answers
35 views

What is the time complexity of Deep VGG-16 Net?

I would like to compute the time complexity (in Big O notation) of Deep Vgg16 Net by feeding forward 1000 images with size 224x224x3 and retrieving features from the second fully connected layer 'fc7' ...
user avatar
1 vote
0 answers
40 views

How many layers do GPT-3, AlphaFold 2, and DALL-E 2 have?

Unsuccessfully, I tried to find out the "depth" (definition below) in large neural networks such as GPT-3, AlphaFold 2, and DALL-E 2. Formally, my question is about their computational graph:...
user avatar
1 vote
0 answers
16 views

How to define a custom layer in Pytorch [closed]

I am new to PyTorch and seeking your help regarding a problem I have. I need to add a costume layer to a NN in training phase. Please see the figure which shows a simple DNN with the custom layer. NN ...
user avatar
  • 11
2 votes
1 answer
27 views

Why does backprop algorithm store the inputs to the non-linearity of the hidden layers?

I have been reading the Deep Learning book by Ian Goodfellow and it mentions in Section 6.5.7 that The main memory cost of the algorithm is that we need to store the input to the nonlinearity of the ...
user avatar
-1 votes
1 answer
38 views

What is exactly sparse annotation?

What is exactly sparse annotation? Is it different from labeling images? I've been reading a paper about vessel segmentation and have some issues understanding this part.
user avatar
0 votes
0 answers
30 views

DNN architecture for regression with large output layers

I'm facing a supervised regression problem where I need to predict the outcome of the numerical simulation of a physical process. Each simulation-sample outputs a 256x256x6 tensor of real numbers ...
user avatar
1 vote
0 answers
20 views

References for Nvidia's DLSS

Nvidia's deep learning super-sampling is presented as an application of deep learning techniques to video-rendering in videogames. Question: I'm asking for a technical reference that explains what is ...
user avatar
1 vote
1 answer
122 views

Using "softmax" (non-linear) vs "linear" activation function in Deep Reinforcement Learning

I am following the tutorial in this video: https://youtu.be/cO5g5qLrLSo which implements deep reinforcement learning (DQN) to balance cart pole in OpenAI default environment. The DQN model looks like ...
user avatar
0 votes
2 answers
127 views

How to count overlapping objects with neural networks

Consider the following task to be solved by a neural network: Given a $N\times N$ pixel grid with up to $M$ objects drawn on it, either squares (9 pixels) or diamonds (5 pixels): square    diamond The ...
user avatar
0 votes
0 answers
32 views

Differential Privacy measures of any trained ML model

I wonder if there is any practical way to assess/measure the level of privacy that a trained ML model has using Pytorch? I know that there are different techniques which helps to guarantee certain ...
user avatar
1 vote
0 answers
16 views

Output representation for a neural network to learn grid-based game with multiple units

I have a round based game played on a grid map with multiple units that I would like to control in some fashion using neural network (NN). All of the units are moved at once. Each unit can move in any ...
user avatar
  • 159
0 votes
0 answers
16 views

what are the factors that make different Kernels in CNN? [duplicate]

what are the factors that affect the number of kernels and filters on CNN ? like one is 256 sometimes or 128 .. What does this change depend on? what is the difference between kernel and filter in the ...
user avatar
0 votes
0 answers
49 views

If the model always underfits, do I really need a larger model?

I train my neural network on random points generated for a data set that theoretically consists of approximately $1.8 * 10^{39}$ elements. I sample (generate) tens of thousands of random points on ...
user avatar
0 votes
0 answers
14 views

What to infer If a parameter value is the same as I initialized while training?

I am running a neural network model. There are several parameters. I initialized a new parameter to a value (say k). When I observe its change during the training, it is not changing like other ...
user avatar
  • 3,059
3 votes
0 answers
65 views

Are there neural networks with (hard) constraints on the weights?

I don't know too much about Deep Learning, so my question might be silly. However, I was wondering whether there are NN architectures with some hard constraints on the weights of some layers. For ...
user avatar
  • 163
1 vote
1 answer
117 views

Multi label classification on non binary labels with pytorch

I am working on a project consisting of medical images and a huge dataset of multi-label and non-binary labels/outcomes ( sex, blood pressure, age and 40 more ). Would be the best approach to hard ...
user avatar
1 vote
1 answer
56 views

Are there any animation tools available to visualise and simulate deep neural networks? [closed]

Deep learning researchers have to work with a lot of models. The models may include different types of Layers: They include convolutional neural network layers, recurrent neural network layers, batch ...
user avatar
  • 3,059
0 votes
0 answers
27 views

What all are the known reasons for the decline in the performance of a neural network if we keep on increasing the depth of it?

Progress in many application tasks in artificial intelligence is achieved by increasing the depth of the neural networks. But if we keep on increasing the number of layers in the neural network, the ...
user avatar
  • 3,059
0 votes
0 answers
35 views

Can I perform 3D point cloud per-point labeling from binary classification alone?

All, It seems that the process of individually labeling points in 3D point clouds is no small task. I believe that's why tools like these exist: Sagemaker Pointly But ... what if there are only two ...
user avatar
  • 21
2 votes
0 answers
63 views

How does back propagation adjust the hidden layers' weights and biases?

I'm new to neural networks and trying to figure out its fundamentals but I cannot fully understand the back propagation algorithm. In back propagation, I understand we want to go backwards from the ...
user avatar
0 votes
0 answers
48 views

How to calculate computational efficiency of Deep Learning Models?

I am trying to make a comparison between two simple 5 layer neural network models. One of the models has 3 frozen layers as I've implemented transfer learning in that architecture. The other is ...
user avatar
1 vote
0 answers
28 views

How Tesla and other companies use outputs from neural networks to drive the car?

Here is the short description of Tesla Autopilot AI: https://www.tesla.com/autopilotAI And here are some videos about how Tesla uses neural networks: Andrej Karpathy - AI for Full-Self Driving at ...
user avatar
  • 173
0 votes
1 answer
94 views

Dissection of a depth map

I am curious about how depth maps work. While searching I came across this website which contains some images and their depth maps. I took this depth map and tried to study it using a python pillow. <...
user avatar
  • 936
0 votes
0 answers
217 views

Computational complexity of a CNN network

In the following network, the convolution operations of convolutional blocks are performed by three 1-D kernels with the sizes 8, 5, and 3 respectively along with stride equal to 1. The final network ...
user avatar
0 votes
0 answers
40 views

How can a CNN be used in machine translation?

How can a convolutional neural net (CNN) be used in machine translation? Convolution is a mathematical operation, so how are natural languages translated into matrices? e.g., DeepL_Translator#...
user avatar
  • 111
0 votes
0 answers
22 views

In CNN, how the conversion of convolution layer to fully connected layer decides the no. of kernel

I am trying to understand the shape of the activation map after every operation. Here is the model summary . All is clear, but from the point labeled 1, how 7x7x512 turns out to be 4096 specifically ...
user avatar
  • 11
1 vote
1 answer
66 views

Would this count as a Transfer Learning approach?

I have two datasets, Dataset 1(D1) and Dataset 2(D2). D1 has around 22000 samples, and D2 has around 8000 samples. What I am doing is that I train a Deep Neural Network model with around three layers ...
user avatar
0 votes
0 answers
36 views

Bridging the gap between simulation and real-world scenarios!

I've got a DRL model that was trained on a simulation at a frame rate of 100fps, after testing it with 100fps it gives good results however when testing it with another frame rate say 50fps it gives a ...
user avatar
  • 36
0 votes
0 answers
21 views

What does it mean when predicted results are constant values?

I'm practicing with some data with a LSTM neural nets to come up with predicted data, comparing with actual data. I generated an image to show what I came up with. The blue line is actual data, and ...
user avatar
  • 189
2 votes
1 answer
87 views

What is meant by "stable training" of a deep learning model?

I have read it said that the "stable training" of a deep learning model is important. What is meant by "stable training" of a deep learning model?
user avatar
0 votes
1 answer
76 views

Number of classes vs number of parameters/layers?

How to estimate the number of parameters in CNN for object detection? I know that there are some well-known architectures that was trained on a lot of data (AlexNet, ResNet, VGG, GoogleLeNet). But ...
user avatar
  • 173
0 votes
1 answer
31 views

A neural network to learn the connection between two totally different type of images

I have a dataset of two different type of images. Say, I have images of a person and his all 10 fingerprints. I want to create a relation between them to predict one from another. How I can do that ...
user avatar
  • 1
2 votes
1 answer
97 views

Do we ever need more then 1 hidden layer in a binary classification problem with ANNs? If yes why?

I have read about the universal approximation theorem. So, why do we need more than 1 layer? Is it somehow computationally efficient to add layers instead of more neurons in the hidden layer?
user avatar
8 votes
1 answer
610 views

When should you not use the bias in a layer?

I'm not really that experienced with deep learning, and I've been looking at research code (mostly PyTorch) for deep neural networks, specifically GANs, and, in many cases, I see the authors setting <...
user avatar
2 votes
1 answer
43 views

What is the difference between multi-head and normal output?

Let's say that I have a neural network with 2 heads. The first consists of X neurons. The second consists of Y neurons. I have these 2 heads because I want to predict 2 different variables. And I can ...
user avatar
  • 173
0 votes
1 answer
39 views

What algorithm to use to classify data by spatial relations?

Let's assume I have dataset of image-like 2D samples where values can be divided into few discrete levels (for example 1, 2, 3 and 4) like in the image below, where each color maps different value, ...
user avatar
  • 1,027
1 vote
1 answer
86 views

Unable to 'learn' a rotational angle by parametrising the angle as a neural network layer

I'm trying to implement a neural network that can capture the drift in a measured angle as a way of dynamic calibration. i.e, I have a reference system that may change throughout the course of the ...
user avatar
1 vote
0 answers
26 views

The MLP output of a neural network can be written as $\|x\|\|w_l\|\cos(\theta_l)$: why is the norm easier to maximize?

The MLP output of a neural network is a dot product between the weights and the input and therefore can be written as $\|x\|\|w_l\|\cos(\theta_l)$ (see this for more details), where $x$ is the input, $...
user avatar
1 vote
0 answers
40 views

SAGAN - is there a mistake in the original paper?

in the original paper the following scheme of the self-attention appears: https://arxiv.org/pdf/1805.08318.pdf In a later overview: https://arxiv.org/pdf/1906.01529.pdf this scheme appears: ...
user avatar
  • 111
3 votes
0 answers
49 views

What's up with Neural Stochastic Differential Equations from a practical standpoint?

I've spent a few days reading some of the new papers about Neural SDEs. For example, here is one from Tzen and Raginsky and here is one that came out simultaneously by Peluchetti and Favaro. There are ...
user avatar
3 votes
2 answers
87 views

How do we know that the neurons of an artificial neural network start by learning small features?

I'd like to ask you how do we know that neural networks start by learning small, basic features or "parts" of the data and then use them to build up more complex features as we go through ...
user avatar
  • 489
2 votes
0 answers
51 views

Estimating dimensions to reduce input image size to in CNNs

Considering input images to a CNN that have a large dimension (e.g. 256X256), what are some possible methods to estimate the exact dimensions (e.g. 16X16 or 32X32) to which it can be condensed in the ...
user avatar
0 votes
0 answers
25 views

How to deal with dynamically changing input tensor in neural networks without padding?

I have a dataset about the monitored health/growth of a community of people. The dataset has tensor shaped (batch_size, features, person, window), where: person==...
user avatar
2 votes
0 answers
84 views

Are there any new weight initialization techniques for DNN published after 2015?

Considering weights initialization in my personal projects, I always used some standard techniques such as: Glorot (also known as Xavier) initialization (2010). Mertens initialization (2010). He ...
user avatar
1 vote
1 answer
74 views

Regression For Elliptical Curve Public Key Generation Possible?

As part of a learning more about deep learning, I have been experimenting with writing ResNets with Dense layers to do different types of regression. I was interested in trying a harder problem and ...
user avatar
0 votes
0 answers
48 views

How should I use deep learning to find the rotation of an object from its 2D image?

I have 6600 images and I am supposed to know the rotation of the object in each image. So, given an image, I want to regress to a single value. My attempt: I use Resnet-18 to extract a feature vector ...
user avatar
1 vote
1 answer
390 views

How to build a DQN agent with state and action being arrays?

I have a Reinforcement-Learning environment where the state is an array of 0s and 1s with length equals to the number of users the agent must satisfy (11 users). The agent must choose one of 12 ...
user avatar
  • 147