2
$\begingroup$

According to my experience with Tensorflow and many other frameworks, neural networks have to have a fixed shape for any output, but how does Google translate convert texts of different lengths?

$\endgroup$
4
$\begingroup$

Usually, in natural language processing (NLP), they are using Sequence to Sequence Learning (Seq2Seq) with Neural Networks, such as Recurrent Neural Networks or more recently the Transformer (you can find two very good papers here, and here).

During training, to ensure the same size of the input and output they can just search for the longest sentence they have in the dataset or select a number that is high enough and pad all the other sentences with 0. In addition, they add a stop token where the sentence ends, so that the model is aware of this. When decoding (inference), the decoder is going to predict one word at a time until it predicts the stop token, which signals that the translation is done.

If you're interested to see an actual implementation, I would recommend looking at this tutorial which does a good job at explaining the code and how it works.

$\endgroup$
1
  • 3
    $\begingroup$ Transformers have surpassed RNNs (even LSTMs) in some NLP tasks (including translation), and they are now used by Google Translate (check this), so you probably should mention them in this answer. $\endgroup$
    – nbro
    Nov 10 '20 at 10:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.