I recently read an article about neural networks saying that, when using sigmoid as activation function, it's advised to use 0.1 as target value instead of 0, and 0.9 instead of 1. This was to avoid "saturation effects". I only understood is halfway, and was hoping someone could clarify a few things for me:
Is this only the case when the output is boolean (0 or 1), or will it also be the case for continual values in the range between 0 and 1. If so, should all values be scaled to the interval [0.1, 0.9]?
What exactly is the problem of output 0 or 1? Does it have something to do with the derivative of sigmoid being 0 when it's value is 0 or 1? As I understood it weights could end up approaching infinity, but I didn't understand why.
Is this the case only when sigmoid is used in the output layer (which it rarely is, I believe), or is it also the case when sigmoid is used in hidden layers only?