I want to train a model over a variable-length sequential data (e.g. the temperature at different times of day) where the output depends on what the temperature is at time T.

Ideally I want to represent the input using a variable-length compacted format of [temperature, duration]. Alternatively, I can divide a matrix into time slices where each cell contains the current temperature.

I prefer the compacted format as it is more space-efficient and allows me to represent arbitrary-length durations, but I am afraid that a Transformer architecture won't be able to figure out what the temperature is at time T using the compact format.

Is it safe to compact sequential inputs?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.