0
$\begingroup$

While studying about linear interpolation technique in natural language processing to deal with less frequent $n-$gram. I came across a sophisticated version of linear interpolation.

The simple and sophisticated linear interpolation techniques are as follows:

Simple linear interpolation: The estimate of trigram probability $p(w_n| w_{n-1}w_{n-2})$ is given by

$$\hat{p}(w_n| w_{n-1}w_{n-2}) = \lambda_1 p(w_n| w_{n-1}w_{n-2}) + \lambda_2 p(w_n| w_{n-1}) + \lambda_3 p(w_n) $$

Sophisticated linear interpolation: The estimate of trigram probability $p(w_n| w_{n-1}w_{n-2})$ is given by

$$\hat{p}(w_n| w_{n-1}w_{n-2}) = \lambda_1 (w_{n-2:n-1}) p(w_n| w_{n-1}w_{n-2}) + \lambda_2 (w_{n-2:n-1}) p(w_n| w_{n-1}) + \lambda_3 (w_{n-2:n-1}) p(w_n) $$

I understood the simple linear interpolation technique. but, facing an issue with the RHS expression of the sophisticated linear interpolation technique.

The textbook, in p16, says the following about the sophisticated linear interpolation discussed above

In a slightly more sophisticated version of linear interpolation, each $\lambda$ weight is computed by conditioning on the context. This way, if we have particularly accurate counts for a particular bigram, we assume that the counts of the trigrams based on this bigram will be more trustworthy, so we can make the $\lambda$'s for those trigrams higher and thus give that trigram more weight in the interpolation. Equation given above shows the equation for interpolation with context-conditioned weights.

I cannot understand the contribution of $(w_{n-2:n-1})$ to the estimation equation of sophisticated linear interpolation. Is it a number?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.