Why use sin/cos to give periodicity in time series prediction

Question

In this tutorial https://www.tensorflow.org/tutorials/structured_data/time_series#feature_engineering (scroll down a bit to "Time" heading), they take the sin/cos of the time index, and give this as an input so that the model can see the periodicity.

Why use sin and cos (which map to a circle)? Why not map to a square, or a diamond?

What about if you just mapped time to 1d instead of 2d, so e.g. 23:59 would be 1 and 00:00 (1 minute later) would be 0. Would that "jump" actually cause problems? Any actual research or experiments which look at this issue?

Answer 1

Why use sin and cos (which map to a circle)?

Specifically from the tutorial's point of view, this is the most usual piece of feature engineering applied to periodic input data.

It offers the following advantages - roughly in order of importance (in my opinion):

• Removes discontinuities (e.g. as you suggest the "jump" from 1 to 0 when time of day wraps around). Non-linear learning algorithms - such as neural networks - can learn to approximate discontinuities from inputs or outputs, but why force the learning algorithm to learn about them, when you can make the necessary transform for it? After all, that is a core purpose of feature engineering, to inject knowledge about the problem that assists the learning algorithm.

• Creates a consistent distance measure. The vector distance between two points on the circle is always the same if they are the same distance away linearly on the pre-transformed feature (including the periodicity). This is not the case with other mappings such as squares or diamonds.

• Sin and cos are standard library library functions in most programming languages. Mapping to a square or diamond involves slightly more custom coding.

Why not map to a square, or a diamond?

Squares or diamonds are close enough to circles, that in a lot of cases you would not notice the difference in practice. There may even be cases where some kind of closed polygon is a more natural mapping for a given problem. So feel free to use them and experiment when performing feature engineering.

Any actual research or experiments which look at this issue?

This kind of low-level feature engineering is simple enough that you probably won't find many papers that pull out just solutions to mapping periodic variables. You will find using sin and cos to transform a periodic variable used as a standard transform in many situations though.

If you are interested to compare different options for mapping periodic variables, you will likely need to perform the experiments yourself. For any specific problem where you are not sure, this is good practice if you have the time. There is a chance that you will find a useful generic mapping for periodic data that is not based on circles, but my prediction is that you will find other shapes perform similarly to circles a lot of the time, but are sometimes worse and rarely if ever better.