# What are some concrete steps to deal with the vanishing gradient problem?

I am training an ANN for classification between 3 classes. The ANN has an input layer, one hidden layer and a 3 node output layer.

The problem I am facing is that the output being produced by the 3 output nodes are so close to 1 (for the first few iterations at least, and so I am assuming the problem propagates to future outputs as well) the weights are not being updated (or hardly updated) due to overflow (about $$10^{-11}$$). I can fix the overflow problem (but I don't think it is the culprit). I think such low values of error is the main culprit, and I cannot figure what is causing such low values of error.

What will cause the network to behave more responsively, that is, how will I be actually able to grasp the weight updates and not something in the order of $$10^{-11}$$?

The data set contain values in the order of $$10$$'s, and the weights randomly initialized are in the order of $$0 < w < 1$$. I have tried feature normalization but it is not that effective.