# What is it about sigmoid activations in particular that allows for the keeping and forgetting of past information from different time scales?

My understanding is that normal recurrent neural networks (RNNs) are not good at keeping past information from different time scales. Furthermore, my understanding is that Gated RNNs, such as Long Short-Term Memory, model the keeping and forgetting mechanisms explicitly with sigmoid activations, namely gates. What is it about sigmoid activations in particular that allows for the keeping and forgetting of past information from different time scales?

Observe the top horizontal line in the image below. The line represents the "internal state" $$C$$. Such state is not used as output for the outside world; it is only passed from the cell to itself at different time steps. This is where the old information is "stored".
The sigmoid is a convenient means to write information in $$C$$, as the output of a sigmoid ranges between $$[0,1]$$. For example: think of computing point-wise multiplication between $$C$$ and some other generic vector $$v$$. If all values of $$v$$ are in range $$[0,1]$$, we are essentially masking $$C$$, i.e. selectively erasing information from it.
LSTMs try to learn the values of such vector $$v$$. To make sure that is it in the desired range, sigmoid activations are used.