I’m trying to use a Transformer Encoder I coded with weather feature vectors which are basically 11 features about the weather in the dimension
I have a data point per day, so this is a time-series but there are no evidences of the time in my vectors and I know I must use positional encoding to tell my network “this vector is from the 21/05/2021, this one is from the 20/04/2019, etc”.
The usual positional encoding with sin & cos used in NLP doesn’t seem to fit my problem as it encodes the position relative to other words in the sentence and my features are independant values (the temperature of the day doesn’t come after the amount of rain for instance). Order of the data matters between two different inputs but not within the features.
I don’t really know how to do encode my feature vectors for my transformer encoder to get something like
positional_encoded = original_vector + date_encoding which I think could be great but I could also just add the day of year as a feature - would it be enough for a transformer?
What would be the best way to do so?