1
$\begingroup$

In this tutorial https://www.tensorflow.org/tutorials/structured_data/time_series#feature_engineering (scroll down a bit to "Time" heading), they take the sin/cos of the time index, and give this as an input so that the model can see the periodicity.

Why use sin and cos (which map to a circle)? Why not map to a square, or a diamond?

What about if you just mapped time to 1d instead of 2d, so e.g. 23:59 would be 1 and 00:00 (1 minute later) would be 0. Would that "jump" actually cause problems? Any actual research or experiments which look at this issue?

$\endgroup$
1
  • $\begingroup$ I can imagine that one reason is that the author has the properties of the Fourier series and transformations in mind. Every periodic behavior can be decomposed as a sine expansion. $\endgroup$ Aug 30 '21 at 12:55
1
$\begingroup$

Why use sin and cos (which map to a circle)?

Specifically from the tutorial's point of view, this is the most usual piece of feature engineering applied to periodic input data.

It offers the following advantages - roughly in order of importance (in my opinion):

  • Removes discontinuities (e.g. as you suggest the "jump" from 1 to 0 when time of day wraps around). Non-linear learning algorithms - such as neural networks - can learn to approximate discontinuities from inputs or outputs, but why force the learning algorithm to learn about them, when you can make the necessary transform for it? After all, that is a core purpose of feature engineering, to inject knowledge about the problem that assists the learning algorithm.

  • Creates a consistent distance measure. The vector distance between two points on the circle is always the same if they are the same distance away linearly on the pre-transformed feature (including the periodicity). This is not the case with other mappings such as squares or diamonds.

  • Sin and cos are standard library library functions in most programming languages. Mapping to a square or diamond involves slightly more custom coding.

Why not map to a square, or a diamond?

Squares or diamonds are close enough to circles, that in a lot of cases you would not notice the difference in practice. There may even be cases where some kind of closed polygon is a more natural mapping for a given problem. So feel free to use them and experiment when performing feature engineering.

Any actual research or experiments which look at this issue?

This kind of low-level feature engineering is simple enough that you probably won't find many papers that pull out just solutions to mapping periodic variables. You will find using sin and cos to transform a periodic variable used as a standard transform in many situations though.

If you are interested to compare different options for mapping periodic variables, you will likely need to perform the experiments yourself. For any specific problem where you are not sure, this is good practice if you have the time. There is a chance that you will find a useful generic mapping for periodic data that is not based on circles, but my prediction is that you will find other shapes perform similarly to circles a lot of the time, but are sometimes worse and rarely if ever better.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.