I am using a fully connected neural network that uses sigmoid activation function. If we feed a big enough weights the sigmoid function will finally become 1 or 0, is there any solution to avoid this?
Will this lead to classical sigmoid problems vanishing gradient or exploding gradient?