Questions tagged [weight-normalization]
The weight-normalization tag has no usage guidance.
4
questions
3
votes
0
answers
64
views
Are there neural networks with (hard) constraints on the weights?
I don't know too much about Deep Learning, so my question might be silly. However, I was wondering whether there are NN architectures with some hard constraints on the weights of some layers. For ...
- 163
3
votes
1
answer
184
views
Can you help me understand how weight normalization works?
I am trying to dissect the paper Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks.
Unfortunately, because my math is a little bit rusty, I got a little ...
- 33
7
votes
1
answer
185
views
How to solve the problem of too big activations when using genetic algorithms to train neural networks?
I am trying to create a fixed-topology MLP from scratch (with C#), which can solve some simple problems, such as the XOR problem and MNIST classification. The network will be trained purely with ...
- 73
8
votes
1
answer
4k
views
How does weight normalization work?
I was reading the paper Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks about improving the learning of an ANN using weight normalization.
They ...
- 135