Questions tagged [weight-normalization]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
3 votes
0 answers
64 views

Are there neural networks with (hard) constraints on the weights?

I don't know too much about Deep Learning, so my question might be silly. However, I was wondering whether there are NN architectures with some hard constraints on the weights of some layers. For ...
user avatar
  • 163
3 votes
1 answer
184 views

Can you help me understand how weight normalization works?

I am trying to dissect the paper Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks. Unfortunately, because my math is a little bit rusty, I got a little ...
user avatar
7 votes
1 answer
185 views

How to solve the problem of too big activations when using genetic algorithms to train neural networks?

I am trying to create a fixed-topology MLP from scratch (with C#), which can solve some simple problems, such as the XOR problem and MNIST classification. The network will be trained purely with ...
user avatar
8 votes
1 answer
4k views

How does weight normalization work?

I was reading the paper Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks about improving the learning of an ANN using weight normalization. They ...
user avatar
  • 135