# Predicting sine using LSTM: Small output range and delayed output?

I have coded a very basic LSTM with forget gates (no libraries used). I'm trying to predict $$0.5*sin(t + N)$$ given $$0.5*sin(t)$$ as an exercise.

I have tweaked the model, changing the output layer activation function, weight initialization, number of memory blocks/cells, etc. However, I still couldn't manage to correct the output.

The problem is that the output range is much smaller than desired, $$[-0.2, 0.2]$$ instead of $$[-0.5, 0.5]$$. The output also is slightly delayed, meaning it is predicting $$sin(t + N - 1)$$ for example.

Is there something that I'm missing?

As an example, for output layer activation function as a centered logistic from $$(-1, 1)$$, the validation output looks like

Training output looks like

• Weights: generated with normal distribution, from $$[-1, 1]$$
• Output layer activation function used: logistic $$[0, 1]$$, centered logistic, tanh, ReLU, leaky ReLU, $$f(x) = x$$ (identity)