Questions tagged [deep-neural-networks]

For questions related to deep neural networks, which are artificial neural networks with "many" layers, where "many" can vary depending on the context.

Filter by
Sorted by
Tagged with
0
votes
1answer
15 views

Dissection of a depth map

I am curious about how depth maps work. While searching I came across this website which contains some images and their depth maps. I took this depth map and tried to study it using python pillow. <...
0
votes
0answers
16 views

Computational complexity of a CNN network

In the following network, the convolution operations of convolutional blocks are performed by three 1-D kernels with the sizes 8, 5, and 3 respectively along with stride equal to 1. The final network ...
1
vote
1answer
52 views

How much money is spent training neural networks each year by companies such as Google and Facebook?

I am wondering what order of magnitude estimates for the following are for companies Google and Facebook, as well as total globally. What is the rough amount of money spent to train neural networks? ...
0
votes
0answers
26 views

How can a CNN be used in machine translation?

How can a convolutional neural net (CNN) be used in machine translation? Convolution is a mathematical operation, so how are natural languages translated into matrices? e.g., DeepL_Translator#...
0
votes
0answers
12 views

In CNN, how the conversion of convolution layer to fully connected layer decides the no. of kernel

I am trying to understand the shape of the activation map after every operation. Here is the model summary . All is clear, but from the point labeled 1, how 7x7x512 turns out to be 4096 specifically ...
1
vote
1answer
44 views

Would this count as a Transfer Learning approach?

I have two datasets, Dataset 1(D1) and Dataset 2(D2). D1 has around 22000 samples, and D2 has around 8000 samples. What I am doing is that I train a Deep Neural Network model with around three layers ...
0
votes
0answers
32 views

Bridging the gap between simulation and real-world scenarios!

I've got a DRL model that was trained on a simulation at a frame rate of 100fps, after testing it with 100fps it gives good results however when testing it with another frame rate say 50fps it gives a ...
0
votes
0answers
45 views

Is it possible to mimic someone's handwriting given their sample notes?

I have a pet project, but I'm not very well versed with AI. I posted this question on datascience.stackexchange with no luck. I think this forum is more apt, so can anyone here help me start this in ...
4
votes
1answer
43 views

In the field of Deep Learning research, what considerations do researchers take into account when inventing new neural network models?

I am not a researcher, but I am curious to know what considerations are relevant to take into account during research for the invention of a new neural network model, and what relevant knowledge ...
0
votes
0answers
11 views

What does it mean when predicted results are constant values?

I'm practicing with some data with a LSTM neural nets to come up with predicted data, comparing with actual data. I generated an image to show what I came up with. The blue line is actual data, and ...
1
vote
1answer
26 views

What is meant by “stable training” of a deep learning model?

I have read it said that the "stable training" of a deep learning model is important. What is meant by "stable training" of a deep learning model?
0
votes
1answer
32 views

Number of classes vs number of parameters/layers?

How to estimate the number of parameters in CNN for object detection? I know that there are some well-known architectures that was trained on a lot of data (AlexNet, ResNet, VGG, GoogleLeNet). But ...
0
votes
0answers
117 views

If I have a feature matrix of shape $1200 \times 2000$, should I have $2000$ input neurons?

I have an input feature matrix (row-normalized) of size $1200 \times 2000$. When I am writing my neural network, is the number of input neurons equal to the number of columns, that is $2000$? Is it ...
0
votes
1answer
18 views

A neural network to learn the connection between two totally different type of images

I have a dataset of two different type of images. Say, I have images of a person and his all 10 fingerprints. I want to create a relation between them to predict one from another. How I can do that ...
2
votes
1answer
70 views

Do we ever need more then 1 hidden layer in a binary classification problem with ANNs? If yes why?

I have read about the universal approximation theorem. So, why do we need more than 1 layer? Is it somehow computationally efficient to add layers instead of more neurons in the hidden layer?
7
votes
1answer
100 views

When should you not use the bias in a layer?

I'm not really that experienced with deep learning, and I've been looking at research code (mostly PyTorch) for deep neural networks, specifically GANs, and, in many cases, I see the authors setting <...
2
votes
1answer
27 views

What is the difference between multi-head and normal output?

Let's say that I have a neural network with 2 heads. The first consists of X neurons. The second consists of Y neurons. I have these 2 heads because I want to predict 2 different variables. And I can ...
3
votes
1answer
93 views

Which ANN structure to use?

Let $\mathcal{S}$ be the training input data set where each input $u^i \in \mathcal{S}$ has $d$ features. I want to design a ANN so that the cost function below is minimized (the sum of square of ...
0
votes
1answer
19 views

What algorithm to use to classify data by spatial relations?

Let's assume I have dataset of image-like 2D samples where values can be divided into few discrete levels (for example 1, 2, 3 and 4) like in the image below, where each color maps different value, ...
1
vote
1answer
52 views

Unable to 'learn' a rotational angle by parametrising the angle as a neural network layer

I'm trying to implement a neural network that can capture the drift in a measured angle as a way of dynamic calibration. i.e, I have a reference system that may change throughout the course of the ...
1
vote
0answers
25 views

The MLP output of a neural network can be written as $\|x\|\|w_l\|\cos(\theta_l)$: why is the norm easier to maximize?

The MLP output of a neural network is a dot product between the weights and the input and therefore can be written as $\|x\|\|w_l\|\cos(\theta_l)$ (see this for more details), where $x$ is the input, $...
1
vote
0answers
37 views

SAGAN - is there a mistake in the original paper?

in the original paper the following scheme of the self-attention appears: https://arxiv.org/pdf/1805.08318.pdf In a later overview: https://arxiv.org/pdf/1906.01529.pdf this scheme appears: ...
0
votes
0answers
16 views

How to have a DNN output a classification for each user at once?

I have a Reinforcement Learning environment with an agent that allocates power values to different users. To do so, I have thought of implementing a deep neural network like the one shown in the ...
3
votes
0answers
28 views

What's up with Neural Stochastic Differential Equations from a practical standpoint?

I've spent a few days reading some of the new papers about Neural SDEs. For example, here is one from Tzen and Raginsky and here is one that came out simultaneously by Peluchetti and Favaro. There are ...
3
votes
2answers
78 views

How do we know that the neurons of an artificial neural network start by learning small features?

I'd like to ask you how do we know that neural networks start by learning small, basic features or "parts" of the data and then use them to build up more complex features as we go through ...
0
votes
0answers
18 views

How to augment 2.5D keypoints?

I am currently working on 3D hand pose estimation. The idea is to first detect the 2.5D pose representation and then obtain 3D pose with the help of camera parameters. For some reason, I was trying to ...
0
votes
0answers
26 views

Are all activation functions in a layer same? [duplicate]

I understand that for you can have multiple activation functions in different layers. CNN's usually have Relu followed by softmax for the classification. But what stops us in having multiple ...
2
votes
0answers
46 views

Estimating dimensions to reduce input image size to in CNNs

Considering input images to a CNN that have a large dimension (e.g. 256X256), what are some possible methods to estimate the exact dimensions (e.g. 16X16 or 32X32) to which it can be condensed in the ...
0
votes
0answers
18 views

How to deal with dynamically changing input tensor in neural networks without padding?

I have a dataset about the monitored health/growth of a community of people. The dataset has tensor shaped (batch_size, features, person, window), where: person==...
2
votes
0answers
49 views

Are there any new weight initialization techniques for DNN published after 2015?

Considering weights initialization in my personal projects, I always used some standard techniques such as: Glorot (also known as Xavier) initialization (2010). Mertens initialization (2010). He ...
1
vote
1answer
70 views

Regression For Elliptical Curve Public Key Generation Possible?

As part of a learning more about deep learning, I have been experimenting with writing ResNets with Dense layers to do different types of regression. I was interested in trying a harder problem and ...
0
votes
0answers
28 views

How should I use deep learning to find the rotation of an object from its 2D image?

I have 6600 images and I am supposed to know the rotation of the object in each image. So, given an image, I want to regress to a single value. My attempt: I use Resnet-18 to extract a feature vector ...
0
votes
0answers
44 views

CNN for a DQN agent with a 2D matrix state and action as a 2D matrix

I have a custom environment, where the state is a 2D matrix of 11 rows (equals to number of users to satisfy) and 3 columns. Each column can take the value of either 0 or 1, and in each row, there can ...
1
vote
1answer
147 views

How to build a DQN agent with state and action being arrays?

I have a Reinforcement-Learning environment where the state is an array of 0s and 1s with length equals to the number of users the agent must satisfy (11 users). The agent must choose one of 12 ...
1
vote
0answers
88 views

DQN Agent with a 2D matrix as input in Keras

I have a Reinforcement Learning environment where the state is a 2D matrix with 0s and 1s (only one column with the value of 1 in each row). Example: ...
1
vote
1answer
45 views

What are the conceptual differences between regularisation and optimisation in deep neural nets?

I'm trying to wrap my mind around the concepts of regularisation and optimisation in neural nets, especially around their differences. In my current understanding, regularisation is intended to tackle ...
3
votes
1answer
177 views

In GradCAM, why is activation strength considered an indicator of relevant regions?

In the GradCAM paper section 3 they implicitly propose that two things are needed to understand which areas of an input image contribute most to the output class (in a multi-label classification ...
0
votes
0answers
123 views

Am I using transfer learning when I use SSD ResNet 50 model architecture?

Using Label-img, I have successfully labeled my images (dimensions 1100 x 1100 pixels), and am currently training the SSD ResNet50 model (from the TensorFlow 2 Detection Model Zoo). I downloaded the ...
1
vote
1answer
98 views

DQN layers when state space and action space are multi dimensional

I have built my own RL environment, where a state is composed of two elements: the agent's position and a matrix of 0s and 1s (1 if a user has requested a service from the agent, 0 otherwise); an ...
0
votes
1answer
37 views

How is the performance of a model affected by adding a ReLU to fully connected layers?

How significant is adding a ReLU to fully connected (FC) layers? Is it necessary, or how is the performance of a model affected by adding ReLU to FC layers?
0
votes
0answers
10 views

Is there a framework or method that would help visualise inner workings of a feedforward neural network?

I wonder if there is some framework or method to help visualising inner workings of a feedforward deep neural network? What I mean by this is something similar to what is being done with CNNs where we ...
1
vote
1answer
35 views

Should binary feature be in one or two columns in deep neural networks?

Let's assume I have a simple feedforward neural network whose input contains binary 0/1 features and output is also binary two classes. Is it better, worse, or maybe totally indifferent, for every ...
0
votes
0answers
45 views

How to use Deep Q-Network with two-dimensional input? Hands-on Machine Learning 2

I'm studying with the book Hands-on Machine Learning with Scikit-Learn, Keras and TensorFlow, and I'm trying to implement the Deep Q-Network example that can be found on Github but that the input ...
3
votes
1answer
44 views

Are there deep neural networks that have inputs connected with deeper hidden layers?

Are there any architectures of deep neural networks that connect input neurons not only with the first hidden layer but also with deeper ones (red lines on the picture)? If so could you give some ...
1
vote
1answer
68 views

Has “deep vs. wide” been resolved?

All else being equal, including total neuron count, I give the following definitions: wide is a parallel ensemble, where good chunks of the neurons have the same inputs because the inputs are shared ...
0
votes
0answers
45 views

Deep Continuous Clustering algorithm - just one output cluster

I use the DCC algorithm to cluster some data. The whole algorithm is available here, but shortly it is: construct mkNN graph of the data points (the connected components of it are the clusters). ...
3
votes
1answer
108 views

What exactly is an interpretable machine learning model?

From this page in Interpretable-ml book and this article on Analytics Vidhya, it means to know what has happened inside an ML model to arrive at the result/prediction/conclusion. In linear regression, ...
0
votes
1answer
46 views

What are the rules behind vector product in gradient?

Let's suppose we have calculated the gradient and it came out to be $f(WX)(1-f(W X))X$, where $f()$ is the sigmoid function, $W$ of order $2\times2$ is the weight matrix, and $X$ is an input vector of ...
0
votes
1answer
78 views

What are the mathematical prerequisites needed to understand research papers on neural networks? [closed]

I know we have developed some mathematical tools to understand deep neural networks, gradient descent for optimization, and basic calculus. Recently, I encountered arxiv paper that describes higher ...
2
votes
1answer
52 views

Why does using a higher representation space lead to performance increase on the training data but not on the test data?

I read the following from a book: You can intuitively understand the dimensionality of your representation space as “how much freedom you’re allowing the model to have when learning internal ...