I want to train a model over variable-length sequential data (e.g. the temperature at different times of day) where the output depends on what the temperature is at a time
Ideally, I want to represent the input using a variable-length compacted format of [temperature, duration]. Alternatively, I can divide a matrix into time slices where each cell contains the current temperature.
I prefer the compacted format as it is more space-efficient and allows me to represent arbitrary-length durations, but I am afraid that a Transformer architecture won't be able to figure out what the temperature is at a time
T using the compact format.
Is it safe to compact sequential inputs?