Questions tagged [deep-network]

For questions about deep neural networks (DNNs), neural networks with multiple hidden layers between the input and output layer.

Filter by
Sorted by
Tagged with
1
vote
3answers
509 views

How to create Partially Connected NNs with prespecified connections using Tensorflow?

I'd like to implement a partially connected neural network with ~3 to 4 hidden layers (a sparse deep neural network?) where I can specify which node connects to which node from the previous/next layer....
0
votes
3answers
95 views

Is there a way of pre-determining whether a CNN model will perform better than another?

I developed a CNN for image analysis. I've around 100K labeled images. I'm getting a accuracy around 85% and a validation accuracy around 82%, so it looks like the model generalize better than ...
11
votes
1answer
343 views

Can layers of deep neural networks be seen as Hopfield networks?

Hopfield networks are able to store a vector and retrieve it starting from a noisy version of it. They do so setting weights in order to minimize the energy function when all neurons are set equal to ...
0
votes
1answer
387 views

Target values of 0.1 for 0 and 0.9 for 1 for sigmoid

I recently read an article about neural networks saying that, when using sigmoid as activation function, it's advised to use 0.1 as target value instead of 0, and 0.9 instead of 1. This was to avoid "...
2
votes
0answers
24 views

Using two generative adversarial nets to classify articles - what is a good approach?

I'm trying to create a deep learning network to classify news article based on the text and associated image. The idea comes from a novel use of GANs to classify based on generated data. My approach ...
1
vote
0answers
78 views

Confidence interval around a DNN prediction

I am facing a problem and do not know whether it is even solvable: I want to predict the behaviour of a system using a DNN, say a CNN, in the sense that I want to predict the time and intensity of a ...
4
votes
1answer
3k views

Does it make sense to use batch normalization in deep (stacked) or sparse auto-encoders?

Does it make sense to use batch normalization in deep (stacked) or sparse auto-encoders? I cannot find any resources for that. Is it safe to assume that, since it works for other DNNs, it will also ...
4
votes
0answers
476 views

Sparsity constraint in a deep autoencoder

Is there any way and any reason why one would introduce a sparsity constraint on a deep autoencoder? In particular, in deep autoencoders the first layer often has more units than the dimensionality ...
2
votes
0answers
150 views

Deep learning model (LSTM) with temporal and non temporal attributes

I'm working on a project to predict the usage of all the files in a filesystem in near future based on the metadata of the file system for past 6 months. I've got the following attributes about the ...
3
votes
2answers
198 views

How to build my own dataset and model for an LSTM neural network

I have a sort of mathematical problem and I'm not sure which model I should choose to make an LSTM neural network. Currently in my country, there is a system in which certain groups of researchers ...
5
votes
2answers
253 views

What do neural connection weights represent 'conceptually'?

I understand how Neural Networks work and have studied its theory well. My question is at the intricacies of Deep Neural networks and perhaps is a bit beyond common understanding (as I have been told (...
2
votes
1answer
181 views

Elon musk's comment on “non-benign AI scenarios”

I watched a youtube clip of Elon Musk talking about his view on the future of AI. He gave two examples. One of the examples was a benign scenario and the other example was a non benign scenario where ...
4
votes
1answer
159 views

Regression with more than one output, neural network

Currently in my country, there is a system in which certain groups of researchers upload information on products of scientific interest, such as research articles, books, patents, software, among ...
3
votes
1answer
523 views

what's the definition of singularity in the context of neural networks?

The following paper explains the use of skip connections to break the singularity in deep networks. But, I have not fully understood what singularity is. https://arxiv.org/pdf/1701.09175v8.pdf Any ...
2
votes
1answer
34 views

Do Le et al. (2012) train all three autoencoder layers at a time, or just one?

Le et al. 2012 use a network of 1 billion parameters to learn neurons that respond to faces, cats, pedestrians, etc. without labels (unsupervised). Their network is built with three autoregressive ...
5
votes
1answer
90 views

How can neural networks that extract many features be fooled by adversarial images?

I have been reading a bit about networks where deep layers able to deal with a bunch of features (be it edges, colours, whatever). I am wondering: how can possibly a network based on this '...
6
votes
3answers
265 views

Has anybody tried unsupervised deep learning from youtube videos?

YouTube has a huge amount of videos, many of which also containing various spoken languages. This would presumably provide something like the data that a "challenged" baby would experience - "...
4
votes
2answers
352 views

What is the most time-consuming part of training deep networks?

Deep networks notoriously take a long time to train. What is the most time-consuming aspect of training them? Is it the matrix multiplications? Is it the forward pass? Is it some component of the ...
3
votes
2answers
203 views

Should the actor or actor-target model be used to make predictions after training is complete (DDPG)?

The situation I am referring to the paper T. P. Lillicrap et al, "Continuous control with deep reinforcement learning" where they discuss deep learning in the context of continuous action spaces ("...
3
votes
3answers
1k views

What activation function is not used at the final layer of super resolution neural models?

I'm trying to implement some Image super-resolution models on medical images. After reading a set of papers, I found that none of the existing models use any activation layer for the last layer. What'...
0
votes
1answer
301 views

The connection between number of layer of DNN and computational complexity of it

number of layer of DNN and computational complexity of it are correlated after optimization, but how to estimate it before designing DNN?
1
vote
0answers
26 views

Are there a finite set of computable functions constructing deep neural network which can form or implement any c.e. function or computable function?

Are there a finite set of computable functions constructing deep neural network which can form or implement any c.e. function or computable function? Or does there exist a finite set of computable ...
2
votes
0answers
75 views

How would I implement this New Type of NN

CIO NN CIO NN stands for Controller Input Output Nerual Network note due to a typo the "nearon" means "neron" For this we have to redefine the Nearon 2 Inputs 2 Outputs 4 Weights (each input and ...
1
vote
1answer
82 views

Are there some guidelines for designing the architecture of neural networks?

I started to study neural networks recently. I understand how I should define the input and output layers. But I can't find any guidelines on how to build hidden layers. More concretely, for each ...
1
vote
1answer
119 views

How can I design the input layer of a feed-forward neural network to be trained with a medical dataset with three features?

I am building a feed-forward neural network with two hidden layers, which I will train with a medical dataset, which consists of both data, such as age and sex, and images of x-ray scans ($1024 \times ...
14
votes
2answers
2k views

What's the main concept behind Capsule Networks? [duplicate]

As you might know, Capsule Networks have been recently introduced by Hinton. There also have been several heads up within his talks. As expected, the paper elaborates on the idea way theoretically! ...
1
vote
1answer
456 views

Dense-Sparse-Dense CNN training

I want to implement DSD: Dense-Sparse-Dense training for deep neural networks by Han et al. In short, the paper suggest the following training scheme to improve the network accuracy: Train as usual ...
5
votes
2answers
248 views

Is it possible to construct an ANN that is more efficient than the human brain?

Intelligence ... changes based on the environment and situation Human are now inventing machines exhibiting some features of their own Intelligence. There appears to be a possibility that, in the ...
0
votes
1answer
81 views

What are the pros and cons of using a spatial transformation network to predict the next video frame?

I've read through a few papers on next frame prediction from a sequence of frames and several of them use spatial transformations (STNs). See this as an example. I want to know what are the pros and ...
1
vote
2answers
554 views

How many GPUs can these deep learning algorithms be parallelized across (batch parallelization)?

The deep learning algorithms I would to know the limits of are: CNTK Caffe TensorFlow Torch7 Theano For example: I've heard TensorFlow is near impossible to parallelize on 8 GPUs and above. So, in ...
5
votes
1answer
700 views

Precise localization and characterization of rudimentary shapes with neural networks

I understand that there are flavors of (convolutional) neural networks that are useful for object localization and detection tasks of reasonable difficulty. In all of the examples I have seen so far, ...
6
votes
2answers
425 views

Neural networks efficiently solve traveling salesmen problems?

I occasionally read papers that show neural networks solving traveling salesmen problems and multi traveling salesmen problems efficiently? 1) Is there any analysis of the meaning of efficiency of ...
9
votes
1answer
343 views

How much of a problem is white noise for the real-world usage of a DNN?

I read that deep neural networks can be relatively easily fooled (link) to give high confidence in recognition of synthetic/artificial images that are completely (or at least mostly) out of the ...
0
votes
3answers
3k views

SSD or YOLO on arm

Is it possible to run SSD or YOLO object detection on raspberry pi 3 for live object detection (2/4frames x second)? I've tried this SSD implementation but it takes 14 s per frame. Is there anything I ...
11
votes
5answers
2k views

Why are deep neural networks and deep learning insufficient to achieve general intelligence?

Everything related to Deep Learning (DL) and deep(er) networks seems "successful", at least progressing very fast, and cultivating the belief that AGI is at reach. This is popular imagination. DL is a ...
4
votes
1answer
637 views

Using crowdsourcing for deep learning

Most companies dealing with deep learning (automotive - Comma.ai, Mobileye, various automakers etc.) do collect large amounts of data to learn from and then use lots of computational power to train a ...
3
votes
1answer
765 views

Dataset containing images of varying dimensions and orientations

I am new to deep learning. I have a dataset of images of varying dimensions of a certain object. A few images of the object are also in varying orientations. The objective is to learn the features ...
2
votes
1answer
147 views

Network representation for Q-Learning in carrom

I am trying to build an agent to play carrom. The problem statement is roughly to estimate three parameters (normalized) : force angle of striker position of strike Since the state and action ...
12
votes
2answers
644 views

Should deep residual networks be viewed as an ensemble of networks?

The question is about the architecture of Deep Residual Networks (ResNets). The model that won the 1-st places at "Large Scale Visual Recognition Challenge 2015" (ILSVRC2015) in all five main tracks: ...
2
votes
1answer
307 views

How does deepmind's Atari game AI work?

I know that deepmind used deep Q learning (DQN) for its Atari game AI. It used a conv neural network (CNN) to approximate Q(s,a) from pixels instead of from a Q-...
4
votes
1answer
183 views

What can be done to correct for sampling bias introduced from (noisy) training data while training a DNN?

The obvious solution is to ensure that the training data is balanced - but in my particular case that is impossible. What corrections can one perform in such a scenario? I know that my training data ...
-2
votes
1answer
230 views

How many neurons would a network have after a training of 100k small images?

Is there any way to estimate how big the neural network would be after training session of 100,000 unlabeled images for unsupervised learning (like in STL-10 dataset: 96x96 pixels and color)? Not the ...
3
votes
2answers
218 views

Are there any learning algorithms as powerful as “deep” architectures?

This article suggests that deep learning is not designed to produce the universal algorithm and cannot be used to create such a complex systems. First of all it requires huge amounts of computing ...
6
votes
1answer
150 views

Has any research been done on DNN Music?

DNNs are typically used to classify things (of course) but can we let them go wild with sounds and then tell them if we think it sounds good or not? I'd like to think after a training class has been ...
1
vote
1answer
53 views

Do you need single or multiple networks to detect multiple faces?

Given pictures with multiple features such as faces, can single AI algorithm detect all of them, or for better reliability is it preferred to use separate instances? In other words I'm talking about ...
8
votes
2answers
886 views

What's the difference between hyperbolic tangent and sigmoid neurons?

Two common activation functions used in deep learning are the hyperbolic tangent function and the sigmoid activation function. I understand that the hyperbolic tangent is just a rescaling and ...
6
votes
4answers
1k views

Which machine learning algorithm is used in self-driving cars?

Which deep neural network is used in Google's driverless cars to analyze the surroundings? Is this information open to the public?
10
votes
3answers
209 views

What is a deep neural network?

What is the definition of a deep neural network? Why are they so popular or important?
72
votes
10answers
5k views

How is it possible that deep neural networks are so easily fooled?

The following page/study demonstrates that the deep neural networks are easily fooled by giving high confidence predictions for unrecognisable images, e.g. How this is possible? Can you please ...
26
votes
2answers
977 views

How is a deep neural network different from other neural networks?

How is a neural network having the "deep" adjective actually distinguished from other similar networks?