# Are there any downsides of using a fixed seed for a neural network's weight initialization?

For example, if we set the random seed to be 0, will we run into any problems? For example, maybe for seed 0, we can only reach a certain training error, but other seeds will converge to a much lower error

I'm specifically concerned about supervised learning on point cloud data, but curious about whether it matters in general whenever you use a neural network.

• Do you know what the seed is used for? (I did not downvote, by the way).
– nbro
Nov 14 '20 at 14:04

## 1 Answer

When you use a particular seed, it actually ceases to become a random initialization and is instead fixed. I believe the only reason to actually do this would be for reliable reproduction in research and not as a method of training production models.

• But why doesn't it work? If the "random" initialization is important, then we would expect different convergence results for different initializations - for supervised learning this is rarely the case. Nov 16 '20 at 7:13
• different random initializations will give different convergence results (slightly different). If you read research papers, they will often cite a $\pm$ after the results which are the different values on different random initializations. They use the seed so that they can reproduce the EXACT same results again. Different random initializations without a seed will give slightly different results. If you use a fixed seed and flip a coin 100 times, you will get the exact same pattern of H and T every time with that seed. While a different random seed would give a random pattern
– Joff
Nov 16 '20 at 8:53
• Why would we ever care about having slightly different convergence results? Nov 23 '20 at 11:38
• You just gave me a good reasoning for why we can set and forget seed=0 in production as well. +- .001% is not going to make a difference (plus, you have to maximize over the seed to get the best result, not just randomly pick one) Nov 23 '20 at 11:39