0
$\begingroup$

I want to train a model over a variable-length sequential data (e.g. the temperature at different times of day) where the output depends on what the temperature is at time T.

Ideally I want to represent the input using a variable-length compacted format of [temperature, duration]. Alternatively, I can divide a matrix into time slices where each cell contains the current temperature.

I prefer the compacted format as it is more space-efficient and allows me to represent arbitrary-length durations, but I am afraid that a Transformer architecture won't be able to figure out what the temperature is at time T using the compact format.

Is it safe to compact sequential inputs?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.