I, unfortunately, cannot provide you with a scientifically based answer, so I'll try to logic my way to an answer.
I know that there are things called 'nonnegativity-constrained autoencoders'. I do not know the details of their initialization but you could look into those for possible guidelines. There are possibly also other networks which have papers published talking about a 'nonnegativity-constrained' variant which I am unaware of.
Furthermore, I would have some concerns/thoughts about nonnegative neural nets:
- I would assume that having only positive network weights could lead to an exploding gradient. If anything, I would scale the input in a similar range as your weight initialization and have the upper bound of your weight initialization not exceed 1.
- Logically, but also building further on point 1, I would not use (variants of) ReLu activations. With non-negative constrained weights and non-negative inputs, ReLu becomes a simple linear activation. I would advise against any activation function without an upper limit, as it might again result in exploding weights/gradients.
- I would start testing the model with sigmoid activations in all layers (except possibly the last, of course).
- Xavier/glorot initialization is usually best for layers that also use negative weights and a tanh activation function. I do not know how a nonnegative weight constraint changes its effectiveness. However, if you centre your input around (i.e.) 0.5, taking a uniform distribution around 0.5 and vary its range depending on the layer size might work.
Again these are just my insights, not necessarily based on scientific research.