Questions tagged [deep-neural-networks]

For questions related to deep neural networks, which are artificial neural networks with "many" layers, where "many" can vary depending on the context.

Filter by
Sorted by
Tagged with
3
votes
0answers
55 views
+50

In GradCAM, why is activation strength considered an indicator of relevant regions?

In the GradCAM paper section 3 they implicitly propose that two things are needed to understand which areas of an input image contribute most to the output class (in a multi-label classification ...
1
vote
3answers
712 views

How to create Partially Connected NNs with prespecified connections using Tensorflow?

I'd like to implement a partially connected neural network with ~3 to 4 hidden layers (a sparse deep neural network?) where I can specify which node connects to which node from the previous/next layer....
1
vote
1answer
28 views

DQN layers when state space and action space are multi dimensional

I have built my own RL environment, where a state is composed of two elements: the agent's position and a matrix of 0s and 1s (1 if a user has requested a service from the agent, 0 otherwise); an ...
3
votes
2answers
2k views

Why is no activation function used at the final layer of super-resolution models?

I'm trying to implement some image super-resolution models on medical images. After reading a set of papers, I found that none of the existing models use any activation function for the last layer. ...
0
votes
0answers
18 views

Am I using transfer learning when I use SSD ResNet 50 model architecture?

Using Label-img, I have successfully labeled my images (dimensions 1100 x 1100 pixels), and am currently training the SSD ResNet50 model (from the TensorFlow 2 Detection Model Zoo). I downloaded the ...
1
vote
1answer
85 views
3
votes
3answers
1k views

How can the generalization error be estimated?

How would you estimate the generalization error? What are the methods of achieving this?
0
votes
1answer
29 views

How is the performance of a model affected by adding a ReLU to fully connected layers?

How significant is adding a ReLU to fully connected (FC) layers? Is it necessary, or how is the performance of a model affected by adding ReLU to FC layers?
0
votes
0answers
8 views

Is there a framework or method that would help visualise inner workings of a feedforward neural network?

I wonder if there is some framework or method to help visualising inner workings of a feedforward deep neural network? What I mean by this is something similar to what is being done with CNNs where we ...
0
votes
0answers
24 views

How to use Deep Q-Network with two-dimensional input? Hands-on Machine Learning 2

I'm studying with the book Hands-on Machine Learning with Scikit-Learn, Keras and TensorFlow, and I'm trying to implement the Deep Q-Network example that can be found on Github but that the input ...
2
votes
1answer
28 views

Are there deep neural networks that have inputs connected with deeper hidden layers?

Are there any architectures of deep neural networks that connect input neurons not only with first hidden layer but also with deeper ones (red lines on the picture)? If so could you give some names ...
1
vote
3answers
107 views

Is there a way of pre-determining whether a CNN model will perform better than another?

I developed a CNN for image analysis. I've around 100K labeled images. I'm getting a accuracy around 85% and a validation accuracy around 82%, so it looks like the model generalize better than ...
1
vote
1answer
48 views

Has “deep vs. wide” been resolved?

All else being equal, including total neuron count, I give the following definitions: wide is a parallel ensemble, where good chunks of the neurons have the same inputs because the inputs are shared ...
0
votes
0answers
23 views

Deep Continuous Clustering algorithm - just one output cluster

I use the DCC algorithm to cluster some data. The whole algorithm is available here, but shortly it is: construct mkNN graph of the data points (the connected components of it are the clusters). ...
3
votes
1answer
82 views

What exactly is an interpretable machine learning model?

From this page in Interpretable-ml book and this article on Analytics Vidhya, it means to know what has happened inside an ML model to arrive at the result/prediction/conclusion. In linear regression, ...
11
votes
4answers
2k views

What is the “dropout” technique?

What purpose does the "dropout" method serve and how does it improve the overall performance of the neural network?
4
votes
3answers
558 views

What is the most time-consuming part of training deep networks?

Deep networks notoriously take a long time to train. What is the most time-consuming aspect of training them? Is it the matrix multiplications? Is it the forward pass? Is it some component of the ...
4
votes
0answers
60 views

Why Pixel RNN (Row LSTM) can capture triangular contexts?

I'm reading the paper Pixel Recurrent Neural Network. I have a question about Row LSTM. Why Row LSTM can capture triangular contexts? In this paper, the kernel of the one-dimensional convolution ...
0
votes
1answer
42 views

What are the rules behind vector product in gradient?

Let's suppose we have calculated the gradient and it came out to be $f(WX)(1-f(W X))X$, where $f()$ is the sigmoid function, $W$ of order $2\times2$ is the weight matrix, and $X$ is an input vector of ...
0
votes
1answer
61 views

What are the mathematical prerequisites needed to understand research papers on neural networks? [closed]

I know we have developed some mathematical tools to understand deep neural networks, gradient descent for optimization, and basic calculus. Recently, I encountered arxiv paper that describes higher ...
2
votes
1answer
49 views

Why does using a higher representation space lead to performance increase on the training data but not on the test data?

I read the following from a book: You can intuitively understand the dimensionality of your representation space as “how much freedom you’re allowing the model to have when learning internal ...
1
vote
0answers
28 views

How to understand this NN architecture?

I was reading a paper Multi-Agent Reinforcement Learning for Adaptive User Association in Dynamic mmWave Networks and I was stuck understanding the deep neural network architecture that was used. The ...
0
votes
1answer
79 views
0
votes
1answer
42 views

What type of model should I fit to increase accuracy?

Currently, I'm working on 6-axis IMU(Inertial Measurment Unit) dataset. This dataset contain 6 axis IMU data of 7 different drivers. The Imu sensor attached on vehicle. The drivers drives same path. ...
4
votes
2answers
42 views

What kind of output should be used for predicting angles in DNNs?

I am building a model which predicts angles as output. What are the different kinds of outputs that can be used to predict angles? For example, output the angle in radians cyclic nature of the ...
28
votes
2answers
1k views

How is a deep neural network different from other neural networks?

How is a neural network having the "deep" adjective actually distinguished from other similar networks?
1
vote
1answer
23 views

How is weighted average computed in Deep Q networks

I was going through the Sutton book and they said the update formula for Q learning comes from the weighted average of the returns I.e New estimate= old estimate +alpha*[returns- old estimate] So by ...
0
votes
0answers
14 views

How can I decrease the time to compute the mask in the Mask-RCNN for human body detection?

I am using Mask-RCNN to detect human bodies in photos, to get a rough approximation of the ratio of their heights to the length of their chests. I want to decrease the time for making the mask of the ...
0
votes
0answers
9 views

Generative Adversarial Network with two images in input

I am doing an internship project regarding deep learning, and it is a totally new topic for me as I have never studied machine learning in the bachelor's degree courses. I have to implement a GAN that ...
4
votes
2answers
117 views

My Deep Q-Learning Network does not learn for OpenAI gym's cartpole problem

I am implementing OpenAI gym's cartpole problem using Deep Q-Learning (DQN). I followed tutorials (video and otherwise) and learned all about it. I implemented a code for myself and I thought it ...
2
votes
1answer
64 views

Spikes in of Train and Test error

I learn a DNN for image recognition. During each epoch, I calculate mean loss in the training set. After each epoch, I calculate loss and number of errors over both training and test set. The problem ...
0
votes
0answers
42 views

What is the state-of-the-art algorithm for neural style transfer?

I've read the paper A Neural Algorithm of Artistic Style by Gatys et. al. and I find the application of neural style transfer very fun. I also read that Exploring the structure of a real-time, ...
0
votes
0answers
32 views

Why do we use the Target Network for action evaluation in Double deep Q networks

Is there any specific reason as to why The target Network is used for evaluation and The online network Is used for selection, what would be the difference if both roles were switched, our online ...
0
votes
0answers
44 views

Is it known how neural networks work? [duplicate]

There was a chess neural network AlphaZero that taught itself how to play chess better than any chess program that has been created before in just 4 hours. I'm not asking to have it explained to me ...
3
votes
1answer
78 views

Is batch normalization not suitable for non-gaussian input?

I generate some non-Gaussian data, and use two kinds of DNN models, one with BN and the other without BN. I find that the model DNN with BN can't predict well. The codes is shown as follow: <...
0
votes
0answers
45 views

Possible reasons that validation recall is fluctuating across different epochs but the precision is stable?

I'm training a deep learning model. After each epoch I measure the performance of the model on validation set. Here is how the performance looks like while training: It's a binary classification task ...
1
vote
0answers
37 views

How can one be sure that a particular neural network architecture would work?

Traditionally, when working with tabular data, one can be sure(or at least know) that a model works because the included features could explain a target variable, say "Price of a ticket" ...
1
vote
0answers
46 views

Advantages of training Neural Networks based on analytic success criteria

What is the reason to train a Neural Network to estimate a task's success (i.e. robotic grasp planning) using a simulator that is based on analytic grasp quality metrics? Isn't a perfectly trained NN ...
1
vote
1answer
81 views

Why do we need target network in deep Q learning? [duplicate]

I already know deep RL, but to learn it deeply I want to know why do we need 2 networks in deep RL. What does the target network do? I now there is huge mathematics into this, but I want to know deep ...
6
votes
1answer
2k views

Suitable reward function for trading buy and sell orders

I am working to build an deep reinforcement learning agent which can place orders (i.e. limit buy and limit sell orders). The actions are ...
0
votes
0answers
31 views

DQN Tic-Tac-Toe does not quite become optimal

I am trying to implement a DQN agent for playing standard 3x3 Tic-Tac-Toe (it is a double DQN with experience replay, and using a target network). I got the hyperparameters to the point where the ...
0
votes
0answers
22 views

Computational difference between ANN and Pattern Matching

Conceptually, if you had an internal 3d model of all objects in CV you could do a scan matching algorithm. This algorithm would be ridiculously computationally intensive, but it would have a high ...
0
votes
0answers
41 views

Why are YOLO Darknet weights so heavy?

I have been trying to understand how YOLO Darknet works, and for the most part reading the documentation and checking the code helps me understand. But when it comes to the weight file, I can't find ...
0
votes
0answers
17 views

Separated LSTMs or a global one for cluster of related features

I have an $n$-dimensional time-series to apply LSTM to, $n$ is the number of features for each time point. These features can be clustered according to their concept, for example $n_1, ..., n_4$ are ...
3
votes
1answer
49 views

What's the right way of building a deep Q-network?

I'm new to RL and to deep q-learning and I have a simple question about the architecture of the neural network to use in an environment with a continous state space a discrete action space. I tought ...
1
vote
0answers
82 views

Understanding the role of the target network in this DQN algorithm

I've found online this interesting algorithm: From what I understand reading this algorithm, I can't figure out why I should "perform the opposite action" and consequently storing that second ...
44
votes
6answers
2k views

What is fuzzy logic?

I'm new to A.I. and I'd like to know in simple words, what is the fuzzy logic concept? How does it help, and when is it used?
1
vote
0answers
26 views

How can I find the similar non-zero connections between different levels of sparsity of the same network?

I am pruning a neural network (CNN and Dense) and for different sparsity levels, I have different sub-networks. Say for sparsity levels of 20%, 40%, 60% and 80%, I have 4 different sub-networks. Now, ...
1
vote
1answer
86 views

Are there some guidelines for designing the architecture of neural networks?

I started to study neural networks recently. I understand how I should define the input and output layers. But I can't find any guidelines on how to build hidden layers. More concretely, for each ...
1
vote
1answer
47 views

Random value generator using a single neuron or DNN

AI is supposed to do anything human or traditional computer can do, that is what we expect AI to be. So 'generating random value' is also a task included in the scope that AI should be able to do I'...