Questions tagged [layers]

For questions related to layers of neural networks, so you can use this tag if your question involves the input, output, hidden, feedforward, recurrent, convolutional, pooling (aka subsampling), upsampling, etc., layers.

Filter by
Sorted by
Tagged with
2
votes
1answer
51 views

What are the major layers in a Vision Transformer?

Currently, I am studying deepfake detection using deep learning methods. Convolution neural networks, recurrent neural networks, long-short term memory networks, and vision transformers are famous ...
2
votes
0answers
35 views

Has the idea of using different learning rates for different layers been explored in the literature?

I wonder whether there are heuristic rules for the optimal selection of learning rates for different layers. I expect that there is no general recipe, but probably there are some choices that may be ...
4
votes
2answers
122 views

In a neural network, by how much does the number of neurons typically vary from layer to layer?

In a neural network, by how much does the number of neurons typically vary from layer to layer? Note that I am NOT asking how to find the optimal number of neurons per layer. As a hardware design ...
32
votes
4answers
1k views

How to find the optimal number of neurons per layer?

When you're writing your algorithm, how do you know how many neurons you need per single layer? Are there any methods for finding the optimal number of them, or is it a rule of thumb?