Questions tagged [deep-neural-networks]

For questions related to deep neural networks, which are artificial neural networks with "many" layers, where "many" can vary depending on the context.

Filter by
Sorted by
Tagged with
75
votes
10answers
6k views

How is it possible that deep neural networks are so easily fooled?

The following page/study demonstrates that the deep neural networks are easily fooled by giving high confidence predictions for unrecognisable images, e.g. How this is possible? Can you please ...
44
votes
6answers
2k views

What is fuzzy logic?

I'm new to A.I. and I'd like to know in simple words, what is the fuzzy logic concept? How does it help, and when is it used?
30
votes
4answers
979 views

How to find the optimal number of neurons per layer?

When you're writing your algorithm, how do you know how many neurons you need per single layer? Are there any methods for finding the optimal number of them, or is it a rule of thumb?
28
votes
2answers
1k views

How is a deep neural network different from other neural networks?

How is a neural network having the "deep" adjective actually distinguished from other similar networks?
21
votes
4answers
348 views

Is the pattern recognition capability of CNNs limited to image processing?

Can a Convolutional Neural Network be used for pattern recognition in a problem domain where there are no pre-existing images, say by representing abstract data graphically? Would that always be less ...
14
votes
2answers
672 views

Should deep residual networks be viewed as an ensemble of networks?

The question is about the architecture of Deep Residual Networks (ResNets). The model that won the 1-st places at "Large Scale Visual Recognition Challenge 2015" (ILSVRC2015) in all five main tracks: ...
12
votes
1answer
407 views

Can layers of deep neural networks be seen as Hopfield networks?

Hopfield networks are able to store a vector and retrieve it starting from a noisy version of it. They do so setting weights in order to minimize the energy function when all neurons are set equal to ...
11
votes
5answers
2k views

Why are deep neural networks and deep learning insufficient to achieve general intelligence?

Everything related to Deep Learning (DL) and deep(er) networks seems "successful", at least progressing very fast, and cultivating the belief that AGI is at reach. This is popular imagination. DL is a ...
11
votes
4answers
2k views

What is the “dropout” technique?

What purpose does the "dropout" method serve and how does it improve the overall performance of the neural network?
11
votes
1answer
207 views

What kind of problems require more than 2 hidden layers?

I've read that the most of the problems can be solved with 1-2 hidden layers. How do you know you need more than 2? For what kind of problems you would need them (give me an example)?
10
votes
3answers
236 views

What is a deep neural network? [duplicate]

What is the definition of a deep neural network? Why are they so popular or important?
9
votes
1answer
351 views

How much of a problem is white noise for the real-world usage of a DNN?

I read that deep neural networks can be relatively easily fooled (link) to give high confidence in recognition of synthetic/artificial images that are completely (or at least mostly) out of the ...
8
votes
2answers
1k views

What's the difference between hyperbolic tangent and sigmoid neurons?

Two common activation functions used in deep learning are the hyperbolic tangent function and the sigmoid activation function. I understand that the hyperbolic tangent is just a rescaling and ...
6
votes
4answers
1k views

Which machine learning algorithm is used in self-driving cars?

Which deep neural network is used in Google's driverless cars to analyze the surroundings? Is this information open to the public?
6
votes
1answer
2k views

Suitable reward function for trading buy and sell orders

I am working to build an deep reinforcement learning agent which can place orders (i.e. limit buy and limit sell orders). The actions are ...
6
votes
3answers
275 views

Has anybody tried unsupervised deep learning from youtube videos?

YouTube has a huge amount of videos, many of which also containing various spoken languages. This would presumably provide something like the data that a "challenged" baby would experience - "...
6
votes
1answer
155 views

Has any research been done on DNN Music?

DNNs are typically used to classify things (of course) but can we let them go wild with sounds and then tell them if we think it sounds good or not? I'd like to think after a training class has been ...
6
votes
2answers
857 views

Neural networks efficiently solve traveling salesmen problems?

I occasionally read papers that show neural networks solving traveling salesmen problems and multi traveling salesmen problems efficiently? 1) Is there any analysis of the meaning of efficiency of ...
5
votes
1answer
646 views

Using crowdsourcing for deep learning

Most companies dealing with deep learning (automotive - Comma.ai, Mobileye, various automakers etc.) do collect large amounts of data to learn from and then use lots of computational power to train a ...
5
votes
2answers
250 views

Is it possible to construct an ANN that is more efficient than the human brain?

Intelligence ... changes based on the environment and situation Human are now inventing machines exhibiting some features of their own Intelligence. There appears to be a possibility that, in the ...
5
votes
2answers
304 views

What do neural connection weights represent 'conceptually'?

I understand how Neural Networks work and have studied its theory well. My question is at the intricacies of Deep Neural networks and perhaps is a bit beyond common understanding (as I have been told (...
5
votes
1answer
707 views

Precise localization and characterization of rudimentary shapes with neural networks

I understand that there are flavors of (convolutional) neural networks that are useful for object localization and detection tasks of reasonable difficulty. In all of the examples I have seen so far, ...
5
votes
1answer
95 views

How can neural networks that extract many features be fooled by adversarial images?

I have been reading a bit about networks where deep layers able to deal with a bunch of features (be it edges, colours, whatever). I am wondering: how can possibly a network based on this '...
4
votes
2answers
330 views

Iteratively and adaptively increasing the network size during training

For an experiment that I'm working on, I want to train a deep network in a special way. I want to initialize and train a small network first, then, in a specific way, I want to increase network depth ...
4
votes
1answer
70 views

How can I use one neural network for both players in Alpha Zero (Connect 4)?

First of all, it is great to have found this community! I am currently implementing my own Alpha Zero clone on Connect4. However, I have a mental barrier I cannot overcome. How can I use one neural ...
4
votes
1answer
168 views

Regression with more than one output, neural network

Currently in my country, there is a system in which certain groups of researchers upload information on products of scientific interest, such as research articles, books, patents, software, among ...
4
votes
2answers
117 views

My Deep Q-Learning Network does not learn for OpenAI gym's cartpole problem

I am implementing OpenAI gym's cartpole problem using Deep Q-Learning (DQN). I followed tutorials (video and otherwise) and learned all about it. I implemented a code for myself and I thought it ...
4
votes
1answer
189 views

What can be done to correct for sampling bias introduced from (noisy) training data while training a DNN?

The obvious solution is to ensure that the training data is balanced - but in my particular case that is impossible. What corrections can one perform in such a scenario? I know that my training data ...
4
votes
1answer
3k views

Does it make sense to use batch normalization in deep (stacked) or sparse auto-encoders?

Does it make sense to use batch normalization in deep (stacked) or sparse auto-encoders? I cannot find any resources for that. Is it safe to assume that, since it works for other DNNs, it will also ...
4
votes
3answers
558 views

What is the most time-consuming part of training deep networks?

Deep networks notoriously take a long time to train. What is the most time-consuming aspect of training them? Is it the matrix multiplications? Is it the forward pass? Is it some component of the ...
4
votes
1answer
947 views

Dataset containing images of varying dimensions and orientations

I am new to deep learning. I have a dataset of images of varying dimensions of a certain object. A few images of the object are also in varying orientations. The objective is to learn the features ...
4
votes
0answers
60 views

Why Pixel RNN (Row LSTM) can capture triangular contexts?

I'm reading the paper Pixel Recurrent Neural Network. I have a question about Row LSTM. Why Row LSTM can capture triangular contexts? In this paper, the kernel of the one-dimensional convolution ...
4
votes
2answers
42 views

What kind of output should be used for predicting angles in DNNs?

I am building a model which predicts angles as output. What are the different kinds of outputs that can be used to predict angles? For example, output the angle in radians cyclic nature of the ...
4
votes
0answers
520 views

Sparsity constraint in a deep autoencoder

Is there any way and any reason why one would introduce a sparsity constraint on a deep autoencoder? In particular, in deep autoencoders the first layer often has more units than the dimensionality ...
3
votes
2answers
224 views

Are there any learning algorithms as powerful as “deep” architectures?

This article suggests that deep learning is not designed to produce the universal algorithm and cannot be used to create such a complex systems. First of all it requires huge amounts of computing ...
3
votes
3answers
1k views

How can the generalization error be estimated?

How would you estimate the generalization error? What are the methods of achieving this?
3
votes
4answers
330 views

What could an oscillating training loss curve represent?

I tried to create a simple model that receives an $80 \times 130$ pixel image. I only had 35 images and 10 test images. I trained this model for a binary classification task. The architecture of the ...
3
votes
1answer
65 views

What are the methods of optimizing overfitted models?

I'm worrying that my network has become too complex. I don't want to end up with half of the network doing nothing but just take up space and resources. So, what are the techniques for detecting and ...
3
votes
2answers
2k views

Why is no activation function used at the final layer of super-resolution models?

I'm trying to implement some image super-resolution models on medical images. After reading a set of papers, I found that none of the existing models use any activation function for the last layer. ...
3
votes
1answer
124 views

Alpha zero before move 8

The Alpha zero paper says that the The first set of features are repeated for each position in a T = 8-step history. So what happens before the first 8 moves? Do they just repeat the starting position?...
3
votes
2answers
245 views

Should the actor or actor-target model be used to make predictions after training is complete (DDPG)?

The situation I am referring to the paper T. P. Lillicrap et al, "Continuous control with deep reinforcement learning" where they discuss deep learning in the context of continuous action spaces ("...
3
votes
1answer
82 views

What exactly is an interpretable machine learning model?

From this page in Interpretable-ml book and this article on Analytics Vidhya, it means to know what has happened inside an ML model to arrive at the result/prediction/conclusion. In linear regression, ...
3
votes
1answer
49 views

What's the right way of building a deep Q-network?

I'm new to RL and to deep q-learning and I have a simple question about the architecture of the neural network to use in an environment with a continous state space a discrete action space. I tought ...
3
votes
1answer
295 views

How to detect vanishing gradients?

Edit: I've reworked my question to generalize better and be more on-topic, and be mostly software implementation agnostic. Can vanishing gradients be detected by the change in distribution (or lack ...
3
votes
1answer
71 views

Can Google's patented ML algorithms be used commercially?

I just find that Google patents some of the widely used machine learning algorithms. For example: System and method for addressing overfitting in a neural network (Dropout?) Processing images using ...
3
votes
1answer
197 views

Can you learn parameters in nonlinear function?

In the paper Nonlinear Interference Mitigation via Deep Neural Networks, the the following network is illustrated. The network structure is The network parameters are $\theta = \{W_1^{1},...,W_1^{l-...
3
votes
1answer
197 views

Significance of depth of a deep neural network

How is a feed-forward neural network with few hidden layers and lots of nodes in those hidden layers different from a network with a lot of hidden layers but relatively lesser nodes in those hidden ...
3
votes
2answers
211 views

How to build my own dataset and model for an LSTM neural network

I have a sort of mathematical problem and I'm not sure which model I should choose to make an LSTM neural network. Currently in my country, there is a system in which certain groups of researchers ...
3
votes
1answer
582 views

what's the definition of singularity in the context of neural networks?

The following paper explains the use of skip connections to break the singularity in deep networks. But, I have not fully understood what singularity is. https://arxiv.org/pdf/1701.09175v8.pdf Any ...
3
votes
0answers
57 views
+50

In GradCAM, why is activation strength considered an indicator of relevant regions?

In the GradCAM paper section 3 they implicitly propose that two things are needed to understand which areas of an input image contribute most to the output class (in a multi-label classification ...