Questions tagged [hidden-layers]

For questions about functioning, applications, structure and improvement of performance of hidden layers in a Neural Network.

Filter by
Sorted by
Tagged with
15
votes
2answers
270 views

How do I decide the optimal number of layers for a neural network?

How do I decide the optimal number of layers for a neural network (feedforward or recurrent)?
11
votes
1answer
203 views

What kind of problems require more than 2 hidden layers?

I've read that the most of the problems can be solved with 1-2 hidden layers. How do you know you need more than 2? For what kind of problems you would need them (give me an example)?
8
votes
2answers
2k views

Why number of hidden units in a layer are suggested to be in powers of 2?

It is suggested that the number of hidden units in a layer should be in powers of 2 because it helps converge faster. Is it a fact and if it is, how this helps the NN learn faster. Does it have to do ...
8
votes
2answers
1k views

What's the difference between hyperbolic tangent and sigmoid neurons?

Two common activation functions used in deep learning are the hyperbolic tangent function and the sigmoid activation function. I understand that the hyperbolic tangent is just a rescaling and ...
7
votes
1answer
536 views

Why aren't there neural networks that connect the output of each layer to all next layers?

Why aren't there neural networks that connect the output of each layer to all next layers? For example, the output of layer 1 would be fed to the input of layers 2, 3, 4, etc. Beyond computational ...
6
votes
2answers
3k views

What is the purpose of the hidden layers?

Why would anybody want to use "hidden layers"? How do they enhance the learning ability of the network in comparison to the network which doesn't have them (linear models)?
6
votes
3answers
144 views

To what does the number of hidden layers in a neural network correspond?

In a neural network, the number of neurons in the hidden layer corresponds to the complexity of the model generated to map the inputs to output(s). More neurons creates a more complex function (and ...
6
votes
1answer
327 views

How many nodes/hidden layers are required to solve a classification problem where the boundary is a sinusoidal function?

A single neuron is capable of forming a decision boundary between linearly seperable data. Is there any intuition as to how many, and in what configuration, would be necessary to correctly approximate ...
5
votes
1answer
1k views

Maximum nodes in hidden layer

I have an idea to find the optimal number of hidden neurons required in a neural network but I'm not sure how accurate it is. Assuming that it has only one hidden layer, it is a classification ...
5
votes
1answer
123 views

How does a single hidden layer affect output? [duplicate]

I'm learning about multilayer perceptrons, and I have a quick theory question in regards to hidden layer neurons. I know we can use two hidden layers to solve a non-linearly separable problem by ...
4
votes
3answers
766 views

Convolutional Neural Network: does each filter in each convolution layer create a new image?

Say I have a CNN with this structure: input = 1 image (say, 30x30 RGB pixels) first convolution layer = 10 5x5 convolution filters second convolution layer = 5 3x3 convolution filters one dense layer ...
4
votes
1answer
1k views

How can I understand this statement about RNNs and hidden layers?

In the lecture, there was a statement: Recurrent neural networks with multiple hidden layers are just a special case that has some of the hidden to hidden connections missing. I understand ...
4
votes
1answer
74 views

Is it expected that adding an additional hidden layer to my 3-layer ANN reduces accuracy significantly?

I've been using several resources to implement my own artificial neural network package in C++. Among some of the resources I've been using are https://www.anotsorandomwalk.com/backpropagation-...
3
votes
2answers
1k views

What type of neural network would be most feasible for playing a realtime game?

For implementing a neural network algorithm that can play air hockey, I had two ideas for input, and I'm trying to figure out which design would be most viable. The output must be two analog values ...
3
votes
1answer
610 views

Do all neurons in a layer have the same activation function?

I'm new to machine learning (so excuse my nomenclature), and not being a python developer, I decided to jump in at the deep (no pun intended) end writing my own framework in C++. In my current design ...
3
votes
2answers
78 views

What could be the problem when a neural network with four hidden layers with the sigmoid activation function is not learning?

I have a large set of data points describing mappings of binary vectors to real-valued outputs. I am using TensorFlow, and would like to train a model to predict these relationships. I used four ...
3
votes
3answers
100 views

Are these statements about the performance of neural networks as a function of the number of hidden layers contradictory?

In this note Justin Domke says that In practice, neural networks seem to usually find a reasonable solution when the number of layers is not too large, but find poor solutions when using more than, ...
2
votes
2answers
332 views

Why is there a sigmoid function in the hidden layer of a neural network? [duplicate]

I got this slide from CMU's lecture notes. The $x_i$s on the right are inputs and the $w_i$s are weights that get multiplied together then summed up at each hidden layer node. So I'm assuming this is ...
2
votes
1answer
191 views

Can the hidden layer prior to the ouput layer have less hidden units than the output layer?

I attended an introductory class about neural network and I had a question regarding how to choose the number of hidden units per hidden layer. I remember that the Professor saying that there is no ...
2
votes
2answers
970 views

What is hidden state exactly in LSTM and RNN?

I'm working on research rn using LSTM as an encoder decoder in hopes to make inferences. The reason we are using encoder decoder for this is because there is hopes that the hidden state given by the ...
2
votes
1answer
377 views

How do I choose the size of the hidden state of a GRU?

I'm trying to understand how the size of the hidden state affects the GRU. For example, suppose I want to make a GRU count. I'm gonna feed it with three numbers, and I expect it to predict the ...
2
votes
1answer
72 views

Does the encoding of a restricted Boltzmann machine improve with more layers?

I'm using a restricted Boltzmann machine (RBM) as an autoencoder. For now, I use a simple architecture of two layers, the input (~100 nodes) and the output (3 nodes) layers. I'm thinking to add more ...
2
votes
1answer
50 views

Representation of hidden layer in neural network diagrams

Why is the representation of hidden layers made in terms of circles? What is the difference between a circle and a box in diagrams of neural networks?
2
votes
0answers
33 views

Multi-field text input for LSTM

I'm using LSTM to categorize medium-sized pieces of text. Each item to be categorized has several free-form text fields, in addition to several categorical fields. What is the best approach to using ...
2
votes
0answers
124 views

Data prepared to linear regression. Can I use it with backpropagation?

I'm studying a Master's Degree in Artificial Intelligence and I need to learn how to use the Java Neural Network Simulator, JavaNNS, program. In one practice I have to build a neural network to use ...
1
vote
2answers
79 views

How are non-linear surfaces formed in the training of a neural network?

Desperate trying to understand something for couple of weeks. All those questions are actually one big question.Please help me. Time-codes and screens in my question refer to this great(IMHO) 3d ...
1
vote
2answers
85 views

Why is embedding important in NLP, and how does autoencoder work?

People say embedding is necessary in NLP because if using just the word indices, the efficiency is not high as similar words are supposed to be related to each other. However, I still don't truly get ...
1
vote
1answer
1k views

How to chose dense layer size?

I am fine-tuning a VGG16 model on 20 classes with 500k images I was wondering how do you chose the size of the dense layer (the one before the prediction layer which has a size 20). I would prefer not ...
1
vote
1answer
33 views

How to work on different models for a given problem?

I am working on the MNIST data on my own. The idea is to use different values for the number of hidden layers, number of nodes in a given layer, etc. How do you organize these things while you are ...
1
vote
1answer
85 views

Are there some guidelines for designing the architecture of neural networks?

I started to study neural networks recently. I understand how I should define the input and output layers. But I can't find any guidelines on how to build hidden layers. More concretely, for each ...
1
vote
1answer
72 views

How many layers and neurons on the layers does a neural network need for for sales prediction?

The inputs (features) and expected output for my ANN are these: Input 1: Product id (number, cast to double) Input 2: Year in the past (1900..2017, cast to double) Input 3: Month of year (1..12, cast ...
1
vote
1answer
81 views

When are multiple hidden layers necessary?

I know that my question probably seems like being asked many times, but Ill try to be more speciffic: Limitations to my question: I am NOT asking about convolutional neural networks, so please, try ...
1
vote
1answer
48 views

How to determine the number of hidden layers and units of a deep auto-encoder?

I am using a deep autoencoder for my problem. However, the way I choose the number of hidden layers and hidden units in a hidden layer is still based on my feeling. The size of the model that ...
1
vote
1answer
60 views

What is the exact structure within the nodes of a hidden layer? [closed]

I've been reading on neural networks, but for me, seems like the easiest way for me to learn is seeing some code. I am curious about what is the exact structure within a node of a hidden layer and ...
1
vote
0answers
30 views

How does a single neuron in hidden layer affect training accuracy

I'm currently a student learning about AI Networks. I've came across a statement in one of my Professor's books that a FFBP (Feed-Forward Back-Propagation) Neural Network with a single hidden layer ...
0
votes
1answer
55 views

How many hidden layers are needed for this training data set

I'm trying to separate classes in 3D space, the data are as in the sketch below: There are 3 classes: 0,1,2; and with the look into the sketch, it seems that I need 3 planes to separate the classes, ...
0
votes
1answer
80 views

What ANN layer widths support the learning of digit recognition?

I have created an ANN in Python (without libs). On beginning, it had been learned in target of solve linear problems like distinguishing between negative and positive numbers, where the layer widths ...
0
votes
1answer
40 views

Why does the output shape of a Dense layer contain a batch size?

I understand that the batch size is the number of examples you pass into the neural network (NN). If the batch size is 10, it means you feed the NN 10 examples at once. Assuming I have an NN with a ...
0
votes
1answer
39 views

Number of LSTM layers needed to learn a certain number of sequences

Theoretically, number of units for a LSTM layer is the number of hidden states or the max length of sequences as per my practice. For example, in Keras: ...
0
votes
2answers
191 views

How do we choose the activation function for each hidden node? [duplicate]

I am new to neural networks. I would like to use them as a fitting or forecasting method. A simple NN model that does not contain hidden layers, that is, the input nodes are directly connected to the ...