Questions tagged [deep-neural-networks]

For questions related to deep neural networks, which are artificial neural networks with "many" layers, where "many" can vary depending on the context.

Filter by
Sorted by
Tagged with
0
votes
1answer
14 views

Multi label classification on non binary labels with pytorch

I am working on a project consisting of medical images and a huge dataset of multi-label and non-binary labels/outcomes ( sex, blood pressure, age and 40 more ). Would be the best approach to hard ...
1
vote
1answer
37 views

Are there any animation tools available to visualise and simulate deep neural networks? [closed]

Deep learning researchers have to work with a lot of models. The models may include different types of Layers: They include convolutional neural network layers, recurrent neural network layers, batch ...
0
votes
0answers
26 views

What all are the known reasons for the decline in the performance of a neural network if we keep on increasing the depth of it?

Progress in many application tasks in artificial intelligence is achieved by increasing the depth of the neural networks. But if we keep on increasing the number of layers in the neural network, the ...
0
votes
0answers
8 views

Can I perform 3D point cloud per-point labeling from binary classification alone?

All, It seems that the process of individually labeling points in 3D point clouds is no small task. I believe that's why tools like these exist: Sagemaker Pointly But ... what if there are only two ...
2
votes
0answers
44 views

How does back propagation adjust the hidden layers' weights and biases?

I'm new to neural networks and trying to figure out its fundamentals but I cannot fully understand the back propagation algorithm. In back propagation, I understand we want to go backwards from the ...
0
votes
0answers
25 views

Rebirth Architecture for Deep Learning

Intro: Lots of Machine Learning methods are inspired Biology, Nature, Physics, Neurology... I just thought of a Deep Learning approach inspired on religion: Rebirth Network Some eastern religions ...
0
votes
0answers
14 views

How to calculate computational efficiency of Deep Learning Models?

I am trying to make a comparison between two simple 5 layer neural network models. One of the models has 3 frozen layers as I've implemented transfer learning in that architecture. The other is ...
0
votes
0answers
20 views

How Tesla and other companies use outputs from neural networks to drive the car?

Here is the short description of Tesla Autopilot AI: https://www.tesla.com/autopilotAI And here are some videos about how Tesla uses neural networks: Andrej Karpathy - AI for Full-Self Driving at ...
0
votes
1answer
33 views

Dissection of a depth map

I am curious about how depth maps work. While searching I came across this website which contains some images and their depth maps. I took this depth map and tried to study it using a python pillow. <...
0
votes
0answers
82 views

Computational complexity of a CNN network

In the following network, the convolution operations of convolutional blocks are performed by three 1-D kernels with the sizes 8, 5, and 3 respectively along with stride equal to 1. The final network ...
1
vote
1answer
60 views

How much money is spent training neural networks each year by companies such as Google and Facebook?

I am wondering what order of magnitude estimates for the following are for companies Google and Facebook, as well as total globally. What is the rough amount of money spent to train neural networks? ...
0
votes
0answers
28 views

How can a CNN be used in machine translation?

How can a convolutional neural net (CNN) be used in machine translation? Convolution is a mathematical operation, so how are natural languages translated into matrices? e.g., DeepL_Translator#...
0
votes
0answers
14 views

In CNN, how the conversion of convolution layer to fully connected layer decides the no. of kernel

I am trying to understand the shape of the activation map after every operation. Here is the model summary . All is clear, but from the point labeled 1, how 7x7x512 turns out to be 4096 specifically ...
1
vote
1answer
57 views

Would this count as a Transfer Learning approach?

I have two datasets, Dataset 1(D1) and Dataset 2(D2). D1 has around 22000 samples, and D2 has around 8000 samples. What I am doing is that I train a Deep Neural Network model with around three layers ...
0
votes
0answers
33 views

Bridging the gap between simulation and real-world scenarios!

I've got a DRL model that was trained on a simulation at a frame rate of 100fps, after testing it with 100fps it gives good results however when testing it with another frame rate say 50fps it gives a ...
0
votes
1answer
81 views

Is it possible to mimic someone's handwriting given their sample notes?

I have a pet project, but I'm not very well versed with AI. I posted this question on datascience.stackexchange with no luck. I think this forum is more apt, so can anyone here help me start this in ...
0
votes
0answers
11 views

What does it mean when predicted results are constant values?

I'm practicing with some data with a LSTM neural nets to come up with predicted data, comparing with actual data. I generated an image to show what I came up with. The blue line is actual data, and ...
2
votes
1answer
45 views

What is meant by "stable training" of a deep learning model?

I have read it said that the "stable training" of a deep learning model is important. What is meant by "stable training" of a deep learning model?
0
votes
1answer
36 views

Number of classes vs number of parameters/layers?

How to estimate the number of parameters in CNN for object detection? I know that there are some well-known architectures that was trained on a lot of data (AlexNet, ResNet, VGG, GoogleLeNet). But ...
0
votes
0answers
230 views

If I have a feature matrix of shape $1200 \times 2000$, should I have $2000$ input neurons?

I have an input feature matrix (row-normalized) of size $1200 \times 2000$. When I am writing my neural network, is the number of input neurons equal to the number of columns, that is $2000$? Is it ...
0
votes
1answer
23 views

A neural network to learn the connection between two totally different type of images

I have a dataset of two different type of images. Say, I have images of a person and his all 10 fingerprints. I want to create a relation between them to predict one from another. How I can do that ...
2
votes
1answer
75 views

Do we ever need more then 1 hidden layer in a binary classification problem with ANNs? If yes why?

I have read about the universal approximation theorem. So, why do we need more than 1 layer? Is it somehow computationally efficient to add layers instead of more neurons in the hidden layer?
7
votes
1answer
163 views

When should you not use the bias in a layer?

I'm not really that experienced with deep learning, and I've been looking at research code (mostly PyTorch) for deep neural networks, specifically GANs, and, in many cases, I see the authors setting <...
2
votes
1answer
28 views

What is the difference between multi-head and normal output?

Let's say that I have a neural network with 2 heads. The first consists of X neurons. The second consists of Y neurons. I have these 2 heads because I want to predict 2 different variables. And I can ...
3
votes
1answer
104 views

Which ANN structure to use?

Let $\mathcal{S}$ be the training input data set where each input $u^i \in \mathcal{S}$ has $d$ features. I want to design a ANN so that the cost function below is minimized (the sum of square of ...
0
votes
1answer
23 views

What algorithm to use to classify data by spatial relations?

Let's assume I have dataset of image-like 2D samples where values can be divided into few discrete levels (for example 1, 2, 3 and 4) like in the image below, where each color maps different value, ...
1
vote
1answer
57 views

Unable to 'learn' a rotational angle by parametrising the angle as a neural network layer

I'm trying to implement a neural network that can capture the drift in a measured angle as a way of dynamic calibration. i.e, I have a reference system that may change throughout the course of the ...
1
vote
0answers
25 views

The MLP output of a neural network can be written as $\|x\|\|w_l\|\cos(\theta_l)$: why is the norm easier to maximize?

The MLP output of a neural network is a dot product between the weights and the input and therefore can be written as $\|x\|\|w_l\|\cos(\theta_l)$ (see this for more details), where $x$ is the input, $...
1
vote
0answers
38 views

SAGAN - is there a mistake in the original paper?

in the original paper the following scheme of the self-attention appears: https://arxiv.org/pdf/1805.08318.pdf In a later overview: https://arxiv.org/pdf/1906.01529.pdf this scheme appears: ...
0
votes
0answers
17 views

How to have a DNN output a classification for each user at once?

I have a Reinforcement Learning environment with an agent that allocates power values to different users. To do so, I have thought of implementing a deep neural network like the one shown in the ...
3
votes
0answers
35 views

What's up with Neural Stochastic Differential Equations from a practical standpoint?

I've spent a few days reading some of the new papers about Neural SDEs. For example, here is one from Tzen and Raginsky and here is one that came out simultaneously by Peluchetti and Favaro. There are ...
3
votes
2answers
81 views

How do we know that the neurons of an artificial neural network start by learning small features?

I'd like to ask you how do we know that neural networks start by learning small, basic features or "parts" of the data and then use them to build up more complex features as we go through ...
0
votes
0answers
19 views

How to augment 2.5D keypoints?

I am currently working on 3D hand pose estimation. The idea is to first detect the 2.5D pose representation and then obtain 3D pose with the help of camera parameters. For some reason, I was trying to ...
0
votes
0answers
26 views

Are all activation functions in a layer same? [duplicate]

I understand that for you can have multiple activation functions in different layers. CNN's usually have Relu followed by softmax for the classification. But what stops us in having multiple ...
2
votes
0answers
46 views

Estimating dimensions to reduce input image size to in CNNs

Considering input images to a CNN that have a large dimension (e.g. 256X256), what are some possible methods to estimate the exact dimensions (e.g. 16X16 or 32X32) to which it can be condensed in the ...
0
votes
0answers
18 views

How to deal with dynamically changing input tensor in neural networks without padding?

I have a dataset about the monitored health/growth of a community of people. The dataset has tensor shaped (batch_size, features, person, window), where: person==...
2
votes
0answers
59 views

Are there any new weight initialization techniques for DNN published after 2015?

Considering weights initialization in my personal projects, I always used some standard techniques such as: Glorot (also known as Xavier) initialization (2010). Mertens initialization (2010). He ...
1
vote
1answer
71 views

Regression For Elliptical Curve Public Key Generation Possible?

As part of a learning more about deep learning, I have been experimenting with writing ResNets with Dense layers to do different types of regression. I was interested in trying a harder problem and ...
0
votes
0answers
33 views

How should I use deep learning to find the rotation of an object from its 2D image?

I have 6600 images and I am supposed to know the rotation of the object in each image. So, given an image, I want to regress to a single value. My attempt: I use Resnet-18 to extract a feature vector ...
0
votes
0answers
55 views

CNN for a DQN agent with a 2D matrix state and action as a 2D matrix

I have a custom environment, where the state is a 2D matrix of 11 rows (equals to number of users to satisfy) and 3 columns. Each column can take the value of either 0 or 1, and in each row, there can ...
1
vote
1answer
202 views

How to build a DQN agent with state and action being arrays?

I have a Reinforcement-Learning environment where the state is an array of 0s and 1s with length equals to the number of users the agent must satisfy (11 users). The agent must choose one of 12 ...
1
vote
0answers
126 views

DQN Agent with a 2D matrix as input in Keras

I have a Reinforcement Learning environment where the state is a 2D matrix with 0s and 1s (only one column with the value of 1 in each row). Example: ...
1
vote
1answer
47 views

What are the conceptual differences between regularisation and optimisation in deep neural nets?

I'm trying to wrap my mind around the concepts of regularisation and optimisation in neural nets, especially around their differences. In my current understanding, regularisation is intended to tackle ...
3
votes
1answer
196 views

In GradCAM, why is activation strength considered an indicator of relevant regions?

In the GradCAM paper section 3 they implicitly propose that two things are needed to understand which areas of an input image contribute most to the output class (in a multi-label classification ...
0
votes
0answers
132 views

Am I using transfer learning when I use SSD ResNet 50 model architecture?

Using Label-img, I have successfully labeled my images (dimensions 1100 x 1100 pixels), and am currently training the SSD ResNet50 model (from the TensorFlow 2 Detection Model Zoo). I downloaded the ...
1
vote
1answer
141 views

DQN layers when state space and action space are multi dimensional

I have built my own RL environment, where a state is composed of two elements: the agent's position and a matrix of 0s and 1s (1 if a user has requested a service from the agent, 0 otherwise); an ...
0
votes
1answer
39 views

How is the performance of a model affected by adding a ReLU to fully connected layers?

How significant is adding a ReLU to fully connected (FC) layers? Is it necessary, or how is the performance of a model affected by adding ReLU to FC layers?
0
votes
0answers
10 views

Is there a framework or method that would help visualise inner workings of a feedforward neural network?

I wonder if there is some framework or method to help visualising inner workings of a feedforward deep neural network? What I mean by this is something similar to what is being done with CNNs where we ...
1
vote
1answer
37 views

Should binary feature be in one or two columns in deep neural networks?

Let's assume I have a simple feedforward neural network whose input contains binary 0/1 features and output is also binary two classes. Is it better, worse, or maybe totally indifferent, for every ...
0
votes
0answers
53 views

How to use Deep Q-Network with two-dimensional input? Hands-on Machine Learning 2

I'm studying with the book Hands-on Machine Learning with Scikit-Learn, Keras and TensorFlow, and I'm trying to implement the Deep Q-Network example that can be found on Github but that the input ...