Skip to main content
Share Your Experience: Take the 2024 Developer Survey

Questions tagged [bleu]

For questions related to BLEU (BiLingual Evaluation Understudy), which is a metric for evaluating the quality of text which has been machine-translated from one natural language to another. The metric was proposed in the paper "BLEU: a Method for Automatic Evaluation of Machine Translation" (2002) by Kishore Papineni et al.

Filter by
Sorted by
Tagged with
2 votes
2 answers
3k views

What are the differences between BLEU and METEOR?

I am trying to understand the concept of evaluating the machine translation evaluation scores. I understand how what BLEU score is trying to achieve. It looks into different n-grams like BLEU-1,BLEU-2,...
Exploring's user avatar
  • 343
1 vote
1 answer
379 views

Does it make sense to use BLEU or ROUGE for any machine translation task?

Many machine translation metrics such as BLEU or ROUGE are used to evaluate sequence to sequence models where, usually, the sequences are pieces of natural language. Is it possible to use these ...
Blencer's user avatar
  • 73
5 votes
2 answers
8k views

What is the difference between a language model and a word embedding?

I am self-studying applications of deep learning on the NLP and machine translation. I am confused about the concepts of "Language Model", "Word Embedding", "BLEU Score". ...
Exploring's user avatar
  • 343
2 votes
1 answer
226 views

What happens when the output length in the brevity penalty is zero?

The brevity penalty is defined as $$bp = e^{(1- r/c)},$$ where $r$ is the reference length and $c$ is the output length. But what happens if the output length gets zero? Is there any standard way ...
ScientiaEtVeritas's user avatar
8 votes
2 answers
2k views

Why is the perplexity a good evaluation metric for chatbots?

A few papers I have come across say that BLEU is not an appropriate evaluation metric for chatbots, so they use the perplexity. First of all, what is perplexity? How to calculate it? And why is ...
RuiZhang1993's user avatar