Skip to main content
Share Your Experience: Take the 2024 Developer Survey

Questions tagged [layers]

For questions related to layers of neural networks, so you can use this tag if your question involves the input, output, hidden, feedforward, recurrent, convolutional, pooling (aka subsampling), upsampling, etc., layers.

Filter by
Sorted by
Tagged with
5 votes
3 answers
2k views

Can you confirm that the transformer works strictly deterministically and there is no randomness inside or between the attention layers?

On a high-level temperature and randomness affect the output of a generative language model: Lower temperature: Produces more focused, conservative, and consistent responses. Moderate temperature: ...
Hans-Peter Stricker's user avatar
1 vote
1 answer
130 views

Formally, what are the layers in an Artificial Neural Network?

You may not believe it, but I am an ANN expert. Perhaps, for that reason, I am unable to grasp completely what the layers are in a Deep Forward Artificial Neural Network (DFANN). According to the Deep ...
neoglez's user avatar
  • 11
2 votes
1 answer
635 views

What are the major layers in a Vision Transformer?

Currently, I am studying deepfake detection using deep learning methods. Convolution neural networks, recurrent neural networks, long-short term memory networks, and vision transformers are famous ...
Pawara Siriwardhane's user avatar
2 votes
0 answers
101 views

Has the idea of using different learning rates for different layers been explored in the literature?

I wonder whether there are heuristic rules for the optimal selection of learning rates for different layers. I expect that there is no general recipe, but probably there are some choices that may be ...
spiridon_the_sun_rotator's user avatar
5 votes
2 answers
260 views

In a neural network, by how much does the number of neurons typically vary from layer to layer?

In a neural network, by how much does the number of neurons typically vary from layer to layer? Note that I am NOT asking how to find the optimal number of neurons per layer. As a hardware design ...
Angela Johnson's user avatar
33 votes
4 answers
2k views

How to find the optimal number of neurons per layer?

When you're writing your algorithm, how do you know how many neurons you need per single layer? Are there any methods for finding the optimal number of them, or is it a rule of thumb?
kenorb's user avatar
  • 10.5k