# Reduce the effect of excessive zeros

I am working on an autoregression problem where I use sequential LSTM. My target is well defined, but I think I am facing a problem with the features. As the features were non-stationary, then I decided to apply the log-returns to each of them. In other words, if $$F_t$$ is a feature at a certain time $$t$$, then I apply $$\log(\frac{F_t}{F_{t-1}}).$$

$$\Rightarrow$$ Why non-stationary data is hard to analyse?

The property makes it easier to analyse, but produce a lot of zeros when $$F_t = F_{t-1}$$.

As there is lots of zeros, then the predictions tend to be closer to 0. How can I reduce that effect mathematically?

## 1 Answer

That the features are not stationary is a reason why LSTM is a good approach. Calculating the ratio of adjacent values in the sequence is unnecessary. Since the logarithm would remove a skew in the input of numbers close to zero, the acquisition is in question. Why are the values not often varying? Are their transient events followed by inactivity?

Aggregating unchanged items in the sequence and adding duration as a feature may add entropy to the input and improve LSTM performance. This is called decimation, and can be useful in adding entropy (removing redundancy) in data that has regions of redundant oversampling.

If the values without the division are still distributed exponentially, then keep the logarithm but maybe make it a natural log.