This might sound silly to someone who has plenty of experience with neural networks but it bothers me...
Random initial weights might give you better results that would be somewhat closer to what a trained neural network should look like, but it might as well be the exact opposite of what it should be, while 0.5, or some other average for the range of reasonable weights' values, would sound like a good default setting.
Why are the initial weights of neural networks randomly initialized rather than being all set to, for example, 0.5?