2
$\begingroup$

I'm trying to learn how time series forecasting models work and while reading a tutorial off the TensorFlow website I came across these algorithms. I don't quite understand what the article means by "time signals" and how do sine and cosine functions help accomplish them. Can anyone please explain?

Here's a link to the tutorial

The following code was provided along with the caption

"the time in seconds is not a useful model input. Being weather data, it has clear daily and yearly periodicity. There are many ways you could deal with periodicity.

You can get usable signals by using sine and cosine transforms to clear "Time of day" and "Time of year" signals:"

day = 24*60*60
year = (365.2425)*day

df['Day sin'] = np.sin(timestamp_s * (2 * np.pi / day))
df['Day cos'] = np.cos(timestamp_s * (2 * np.pi / day))
df['Year sin'] = np.sin(timestamp_s * (2 * np.pi / year))
df['Year cos'] = np.cos(timestamp_s * (2 * np.pi / year))
$\endgroup$
0

2 Answers 2

1
$\begingroup$

In simple terms, without this transformation the network doesn't know that 2022-01-01 data likely correlates with the January 1st on previous (and future!) years. And actually it is likely to also correlate with January 3rd, and maybe even December 29th.

It is also possible to encode the day of year as float between 0.0 and 1.0 (first day of year and last day of year), and that would also provide this "correlation" information. However it isn't as good, since 2021-12-31 would have a value of 1.0, very far from the next day's 2022-01-01 value of 0.0.

Yet an other option is to treat the day of year as a categorical value between 1 and 365 (except on leap years, which is annoying). This would provide some degree of information, but it would lose the apriori knowledge that the 10th day of the year correlates more with the 11th than 100th.

Splitting this to sine and cosine signals circumvents these issues. (now I'm thinking whether this got too verbose.)

This same logic applies to the time of day, 23:59:59 should be very correlated with 00:00:00.

$\endgroup$
0
$\begingroup$

Sorry for my weak English. Your are using neural network to forecast times series which often have irregular fluctuations.

Stock values are volatile and have changing frequency. Applying periodic encodings to original data makes it easier to capture frequency information.

Read this paper to understand why it is essential for the neural network to know about frequency of data and how sinusoidal encoding could make it easier for the neural network to predict highly varying patterns inside data.

https://bmild.github.io/fourfeat/

Or you could use periodic activation functions without applying sinusoidal encoding. Networks with non-periodic activation fail to predict time series.

https://arxiv.org/pdf/2006.08195.pdf

Deep networks are biased towards low frequency functions.

$\endgroup$
1
  • $\begingroup$ Thank you for the papers, I'll check them out! $\endgroup$ Commented Dec 31, 2021 at 22:31

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .