What is the right way to input continuous, temporal (time-series) data into the Transformer? Assume we're using the basic TransformerBlock here.
Since data is continuous with no tokens, Token embedding can be directly skipped. How about positional encoding? I tried this example, removing Token embedding while keeping positional encoding but ended up with shape-related errors. Skipping both token and positional encoding resulted in a network that runs and trains but results were relatively poor compared to the LSTM benchmark with the same data.
I am unsure if the positional encoding is still needed.
Overall, my question is, what is the proper way to process continuous sequence data, such as time-series, using the Transformer architecture?