I wanted to ask what is, in general, the maximum value (the order of magnitude) of the number of time steps I can back-propagate in the past using TBTT (Truncated Backpropagation Through Time) in an LSTM.
I am aware that this could be problem, architecture and optimization dependent. I am curious if there are studies of an order of magnitude among a bunch of test cases (~10? ~100? ~1000?).