I am working on an autoregression problem where I use sequential LSTM. My target is well defined, but I think I am facing a problem with the features. As the features were non-stationary, then I decided to apply the log-returns to each of them. In other words, if $F_t$ is a feature at a certain time $t$, then I apply $$\log(\frac{F_t}{F_{t-1}}).$$

$\Rightarrow$ Why non-stationary data is hard to analyse?

The property makes it easier to analyse, but produce a lot of zeros when $F_t = F_{t-1}$.

As there is lots of zeros, then the predictions tend to be closer to 0. How can I reduce that effect mathematically?


That the features are not stationary is a reason why LSTM is a good approach. Calculating the ratio of adjacent values in the sequence is unnecessary. Since the logarithm would remove a skew in the input of numbers close to zero, the acquisition is in question. Why are the values not often varying? Are their transient events followed by inactivity?

Aggregating unchanged items in the sequence and adding duration as a feature may add entropy to the input and improve LSTM performance. This is called decimation, and can be useful in adding entropy (removing redundancy) in data that has regions of redundant oversampling.

If the values without the division are still distributed exponentially, then keep the logarithm but maybe make it a natural log.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.