1
$\begingroup$

Sequence-to-sequence models with attention are known to be limited by a maximum sequence length. So how can we handle sequences of arbitrarily large size? Do we just set a very large maximum sequence length?

$\endgroup$

1 Answer 1

2
$\begingroup$

You simply split the sequence into smaller sequences; while there are some long-distance dependencies in language, that is generally not a problem for this.

A sentence would typically be short enough, and very long sentences are composed of shorter clauses which would form independent units (albeit connected with each other).

$\endgroup$
2
  • $\begingroup$ How would this "splitting" be done in practice? How are these clauses found? Would this require another model or is it just a matter of defining a set of segmentation rules? $\endgroup$
    – Blencer
    Aug 23, 2021 at 16:10
  • $\begingroup$ I don't know, but I'd guess it would just be by length. I doubt that there would much linguistic knowledge behind it, as it would generally be language independent. Maybe on commas or other punctuation marks. $\endgroup$ Aug 23, 2021 at 16:14

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .