What are the main differences between a language model and a machine translation model?


The simple language model will give you the probability of a sequence of tokens(sentence) for that language. So lets say if you have trained a model for English language your model can give you the probability for any random english sentence.

Consider some sentence

$X$ $=$ "the quick brown fox jumps over the lazy dog" $=$ $x_1 \ x_2 \ x_3 \ ... \ x_n$

model will give you $P(X)$

Moreover if the model has been trained properly, in the following scenario, where

$X$ $=$ "the quick brown fox jumps over the lazy dog" $=$ $x_1 \ x_2 \ x_3 \ ... \ x_n$

$Y$ $=$ "dog brown quick fox over the jumps lazy the" $=$ $y_1 \ y_2 \ y_3 \ ... \ y_m$

model will always give $P(X) > P(Y)$, as it has learnt the structure of language.

On the other hand Machine translation model gives you the conditional probability of the next token given your source sentence and partial-target sentence. So if

$X = $ "I am a student" and $Y = $ "je suis" $=y_1, \ y_2$

model will give you $P(y_3 | X,y1_,y_2)$ .

where $X$ is a source sentence and $Y= y_1, y_2$ is a partial target sentence. The probability of word/token "étudiant" would be maximum among all words of vocabulary.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.