# Why is perplexity a good evaluation metric for chatbots?

Some papers say that BLEU is not a appropriate evaluating method for chatbot, instead they use perplexity to estimate chatbot.

First of all, what is perplexity? How to calculate it? And why is perplexity a good evaluation metric for chatbots?

For instance, assuming you have $$M$$ sentences $$s_1, \dots, s_M$$, each with probability $$P(s_i)$$, the perplexity is $$2^{-l},$$ where $$l = \frac{1}{M} \sum P(s_i) \log P(s_i)$$ for $$i \in [1 \dots M]$$.