Skip to main content
OverflowAI is here! AI power for your Stack Overflow for Teams knowledge community. Learn more

Questions tagged [weight-normalization]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0 votes
0 answers
27 views

FastGAN implementation has redundant SpectralNorm followed by BatchNorm?

I am implementing a version of FastGAN, and it seems like (see the official repo) they use a Spectral norm directly followed by a Batch norm. Doesn't the spectral norm get fully cancelled by the ...
Ronald's user avatar
  • 21
0 votes
1 answer
177 views

Why and when do we need to normalize weights in Reinforcement Learning?

I recently came across this SO question, wherein the poster was asked to normalize their weights while using a function approximator with SARSA. I don't remember having to normalize any weights while ...
desert_ranger's user avatar
3 votes
0 answers
283 views

Are there neural networks with (hard) constraints on the weights?

I don't know too much about Deep Learning, so my question might be silly. However, I was wondering whether there are NN architectures with some hard constraints on the weights of some layers. For ...
Onil90's user avatar
  • 183
3 votes
1 answer
275 views

Can you help me understand how weight normalization works?

I am trying to dissect the paper Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks. Unfortunately, because my math is a little bit rusty, I got a little ...
Grumpy C's user avatar
7 votes
1 answer
379 views

How to solve the problem of too big activations when using genetic algorithms to train neural networks?

I am trying to create a fixed-topology MLP from scratch (with C#), which can solve some simple problems, such as the XOR problem and MNIST classification. The network will be trained purely with ...
Joshua Jang's user avatar
9 votes
1 answer
6k views

How does weight normalization work?

I was reading the paper Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks about improving the learning of an ANN using weight normalization. They ...
Mike AI's user avatar
  • 145