Questions tagged [deep-network]

For questions about deep neural networks (DNNs), neural networks with multiple hidden layers between the input and output layer.

Filter by
Sorted by
Tagged with
2
votes
1answer
20 views

Spikes in of Train and Test error

I learn a DNN for image recognition. During each epoch, I calculate mean loss in the training set. After each epoch, I calculate loss and number of errors over both training and test set. The problem ...
1
vote
1answer
21 views

Further Normalization of Standardized data - ANN

I want to develop a regression model using the artificial neural network. For developing such a model I use standardised ( z-score normalised ) data. given below is the sample data set. Here MAX is ...
2
votes
0answers
55 views

How to define cost function for custom nonlinear functions?

For logistic regression, the Cost function is defined as: \begin{equation} Cost(h_{\theta}(x)-y) = -ylog(h_{\theta}(x))-(1-y)log(1-h_{\theta}(x)) \end{equation} I now have a nonlinear function \begin{...
1
vote
0answers
26 views

One end to end Neural network or many task-specific ones?

Is it better to train one neural network for a dispersed labeled data with large number of classes or first classify data by unsupervised learning then train each part by a separate NN? I mean by ...
1
vote
2answers
58 views

How long it takes to train face recognition deep neural network? (rough estimation)

If I use a desktop PC with a GPU, how long it might take to train face recognition deep neural network on let's say dataset of 2.6 million images and 2600 identities? I guess it should depend on ...
3
votes
1answer
124 views

Can you learn parameters in nonlinear function?

In the paper Nonlinear Interference Mitigation via Deep Neural Networks, the the following network is illustrated. The network structure is The network parameters are $\theta = \{W_1^{1},...,W_1^{l-...
1
vote
0answers
37 views

What is the feasible neural network structure that can learn to identify types of trajectory of moving dots?

I have multiple image sequences, each of which contains an animation of two moving dots. The trajectory of the dots in a sequence is always cyclic (not necessarily circular). There are two types of ...
0
votes
0answers
34 views

How to use 0 padding with mask layer to handle variable lenght of my inputs in case of Multi-Layers Perceptrons?

We wanna build a DNN model to predict unrolling factor though our features represent variable length of inputs. Knowing that we have to give our features at once "0 padding" look like the only ...
1
vote
0answers
31 views

How to handle varying length of inputs that represent dependencies and recursivity in deep neural networks in case of regression?

I wanna solve a problem of regression to predict a factor. I decide to go with Deep Neural Networks as solution for my problem. The features in this problem represent loop characteristic such us loop ...
-1
votes
2answers
51 views

Contractive auto-encoders

I am trying to implement Contractive auto-encoders in PyTorch but I don't know what I'm doing is right or not. The architecture of the auto-encoder is given below: ...
2
votes
1answer
375 views

Suitable reward function for trading buy and sell orders

I am working to build an deep reinforcement learning agent which can place orders (i.e. limit buy and limit sell orders). The actions are ...
3
votes
1answer
81 views

Alpha zero before move 8

The Alpha zero paper says that the The first set of features are repeated for each position in a T = 8-step history. So what happens before the first 8 moves? Do they just repeat the starting position?...
1
vote
1answer
230 views

Deep Q-Learning poor convergence on Stochastic Environment

I'm trying to implement a Deep Q-network in Keras/TF that learns to play Minesweeper (our stochastic environment). I have noticed that the agent learns to play the game pretty well with both small and ...
2
votes
1answer
118 views

Significance of depth of a deep neural network

How is a feed-forward neural network with few hidden layers and lots of nodes in those hidden layers different from a network with a lot of hidden layers but relatively lesser nodes in those hidden ...
1
vote
1answer
115 views

Chess policy network

I am interested in making a simple chess engine using neural networks. I already have a fairly good value network but I can't figure out how to train a policy network. I know that Leela chess zero ...
1
vote
3answers
254 views

How to create Partially Connected NNs with prespecified connections using Tensorflow?

I'd like to implement a partially connected neural network with ~3 to 4 hidden layers (a sparse deep neural network?) where I can specify which node connects to which node from the previous/next layer....
0
votes
1answer
70 views

Is there a way of pre-determining whether a CNN model will perform better than another?

I developed a CNN for image analysis. I've around 100K labeled images. I'm getting a accuracy around 85% and a validation accuracy around 82%, so it looks like the model generalize better than ...
0
votes
1answer
154 views

Target values of 0.1 for 0 and 0.9 for 1 for sigmoid

I recently read an article about neural networks saying that, when using sigmoid as activation function, it's advised to use 0.1 as target value instead of 0, and 0.9 instead of 1. This was to avoid "...
2
votes
0answers
21 views

Using two generative adversarial nets to classify articles - what is a good approach?

I'm trying to create a deep learning network to classify news article based on the text and associated image. The idea comes from a novel use of GANs to classify based on generated data. My approach ...
1
vote
0answers
59 views

Confidence interval around a DNN prediction

I am facing a problem and do not know whether it is even solvable: I want to predict the behaviour of a system using a DNN, say a CNN, in the sense that I want to predict the time and intensity of a ...
4
votes
1answer
2k views

Batch Normalization in Deep Autoencoders?

Does it make sense to use Batch Normalization in Deep (stacked) or/and Sparse Autoencoders? I cannot find any resources for that, so is it safe to assume that since it works for other DNNs it will ...
4
votes
0answers
335 views

Sparsity constraint in a deep autoencoder

Is there any way and any reason why one would introduce a sparsity constraint on a deep autoencoder? In particular, in deep autoencoders the first layer often has more units than the dimensionality ...
1
vote
0answers
124 views

Deep learning model (LSTM) with temporal and non temporal attributes

I'm working on a project to predict the usage of all the files in a filesystem in near future based on the metadata of the file system for past 6 months. I've got the following attributes about the ...
3
votes
2answers
133 views

How to build my own dataset and model for an LSTM neural network

I have a sort of mathematical problem and I'm not sure which model I should choose to make an LSTM neural network. Currently in my country, there is a system in which certain groups of researchers ...
5
votes
2answers
185 views

What do neural connection weights represent 'conceptually'?

I understand how Neural Networks work and have studied its theory well. My question is at the intricacies of Deep Neural networks and perhaps is a bit beyond common understanding (as I have been told (...
2
votes
1answer
150 views

Elon musk's comment on “non-benign AI scenarios”

I watched a youtube clip of Elon Musk talking about his view on the future of AI. He gave two examples. One of the examples was a benign scenario and the other example was a non benign scenario where ...
4
votes
1answer
117 views

Regression with more than one output, neural network

Currently in my country, there is a system in which certain groups of researchers upload information on products of scientific interest, such as research articles, books, patents, software, among ...
2
votes
1answer
370 views

what's the definition of singularity in the context of neural networks?

The following paper explains the use of skip connections to break the singularity in deep networks. But, I have not fully understood what singularity is. https://arxiv.org/pdf/1701.09175v8.pdf Any ...
2
votes
1answer
34 views

Do Le et al. (2012) train all three autoencoder layers at a time, or just one?

Le et al. 2012 use a network of 1 billion parameters to learn neurons that respond to faces, cats, pedestrians, etc. without labels (unsupervised). Their network is built with three autoregressive ...
5
votes
1answer
84 views

How can neural networks that extract many features be fooled by adversarial images?

I have been reading a bit about networks where deep layers able to deal with a bunch of features (be it edges, colours, whatever). I am wondering: how can possibly a network based on this '...
6
votes
3answers
252 views

Has anybody tried unsupervised deep learning from youtube videos?

YouTube has a huge amount of videos, many of which also containing various spoken languages. This would presumably provide something like the data that a "challenged" baby would experience - "...
3
votes
3answers
154 views

What is the most time consuming part of training deep networks?

Deep networks notoriously take a long time to train. What is the most time consuming aspect of training them? Is it the matrix multiplications? Is it the forward pass? Is it some component of the ...
2
votes
2answers
107 views

Should the actor or actor-target model be used to make predictions after training is complete (DDPG)?

The situation I am referring to the paper T. P. Lillicrap et al, "Continuous control with deep reinforcement learning" where they discuss deep learning in the context of continuous action spaces ("...
2
votes
3answers
1k views

What activation function is not used at the final layer of super resolution neural models?

I'm trying to implement some Image super-resolution models on medical images. After reading a set of papers, I found that none of the existing models use any activation layer for the last layer. What'...
0
votes
1answer
227 views

The connection between number of layer of DNN and computational complexity of it

number of layer of DNN and computational complexity of it are correlated after optimization, but how to estimate it before designing DNN?
1
vote
0answers
25 views

Are there a finite set of computable functions constructing deep neural network which can form or implement any c.e. function or computable function?

Are there a finite set of computable functions constructing deep neural network which can form or implement any c.e. function or computable function? Or does there exist a finite set of computable ...
2
votes
0answers
70 views

How would I implement this New Type of NN

CIO NN CIO NN stands for Controller Input Output Nerual Network note due to a typo the "nearon" means "neron" For this we have to redefine the Nearon 2 Inputs 2 Outputs 4 Weights (each input and ...
0
votes
2answers
73 views

Is there any common principle/ build algorithm for deep NN structure?

I started to study NN recently. So I understand principles with which I should define input and output layers. But I can't find any guide/directions how to build hidden layers: how many layers do I ...
-1
votes
1answer
76 views

input layer in deep learning

I am building model with medical dataset using deep learning methods. Medical dataset consists of both numerical data such as age, sex and images of xray scans(1024 x 1024) . Labels consists of ...
4
votes
2answers
524 views

How do I create an ai for a two players board game?

Brief idea I want to create an artificial intelligence to compete against other players in a board game. Game explanation I have a board game similar to 'snakes and ladders'. You have to get to a ...
14
votes
2answers
1k views

What's the main concept behind Capsule Networks? [duplicate]

As you might know, Capsule Networks have been recently introduced by Hinton. There also have been several heads up within his talks. As expected, the paper elaborates on the idea way theoretically! ...
1
vote
1answer
335 views

Dense-Sparse-Dense CNN training

I want to implement DSD: Dense-Sparse-Dense training for deep neural networks by Han et al. In short, the paper suggest the following training scheme to improve the network accuracy: Train as usual ...
5
votes
2answers
237 views

Is it possible to construct an ANN that is more efficient than the human brain?

Intelligence ... changes based on the environment and situation Human are now inventing machines exhibiting some features of their own Intelligence. There appears to be a possibility that, in the ...
0
votes
1answer
77 views

What are the pros and cons of using a spatial transformation network to predict the next video frame?

I've read through a few papers on next frame prediction from a sequence of frames and several of them use spatial transformations (STNs). See this as an example. I want to know what are the pros and ...
1
vote
2answers
507 views

How many GPUs can these deep learning algorithms be parallelized across (batch parallelization)?

The deep learning algorithms I would to know the limits of are: CNTK Caffe TensorFlow Torch7 Theano For example: I've heard TensorFlow is near impossible to parallelize on 8 GPUs and above. So, in ...
5
votes
1answer
589 views

Precise localization and characterization of rudimentary shapes with neural networks

I understand that there are flavors of (convolutional) neural networks that are useful for object localization and detection tasks of reasonable difficulty. In all of the examples I have seen so far, ...
3
votes
2answers
731 views

Is there any proof based literature out there on neural networks?

Is there any mathematical proof (like in proof of a theorem) based literature out there on neural networks ? Everything is empirically based but no math proof for instance on why certain parameters ...
5
votes
2answers
167 views

Neural networks efficiently solve traveling salesmen problems?

I occasionally read papers that show neural networks solving traveling salesmen problems and multi traveling salesmen problems efficiently? 1) Is there any analysis of the meaning of efficiency of ...
9
votes
1answer
302 views

Is deep neural network fooling a problem in real world?

I read that deep neural networks can be relatively easily fooled (link) to give high confidence in recognition of synthetic/artificial images that are completely (or at least mostly) out of the ...
0
votes
3answers
3k views

SSD or YOLO on arm

Is it possible to run SSD or YOLO object detection on raspberry pi 3 for live object detection (2/4frames x second)? I've tried this SSD implementation but it takes 14 s per frame. Is there anything I ...