1
$\begingroup$

Sequence-to-sequence models with attention are known to be limited by a maximum sequence length. So how can we handle sequences of arbitrarily large size? Do we just set a very large maximum sequence length?

$\endgroup$
2
$\begingroup$

You simply split the sequence into smaller sequences; while there are some long-distance dependencies in language, that is generally not a problem for this.

A sentence would typically be short enough, and very long sentences are composed of shorter clauses which would form independent units (albeit connected with each other).

$\endgroup$
2
  • $\begingroup$ How would this "splitting" be done in practice? How are these clauses found? Would this require another model or is it just a matter of defining a set of segmentation rules? $\endgroup$
    – Blencer
    Aug 23 at 16:10
  • $\begingroup$ I don't know, but I'd guess it would just be by length. I doubt that there would much linguistic knowledge behind it, as it would generally be language independent. Maybe on commas or other punctuation marks. $\endgroup$ Aug 23 at 16:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.