All tidbits Evaluation Code Course home

TidbitBLEU cannot tell a good translation from a lucky one.

It counts n-gram overlap with a reference. A translation that says the opposite thing in familiar words can score above one that says the right thing in unusual words. Everyone knows this, and the field still reports it, because the alternatives cost money or people.

overlap is not meaning

BLEU cannot tell a good translation from a lucky one.