Chapter 22Evaluating Generated Text

The measurement problem

Chapter 14 evaluated a classifier and the job was straightforward, because there is one right label, the model gives one label, and you compare. Generation breaks that, since there are many correct outputs, they do not resemble each other as strings, and nobody can enumerate them.

Take a translation of “Il pleut”, where It is raining, It’s raining and Rain is falling are all correct, so a metric that credits only the first is measuring the wrong thing. Everything in this chapter is an attempt to work around that, and none of the attempts fully succeeds.

When the answer is a label

Where the output space is closed, use Chapter 14’s machinery unchanged, meaning per-class precision, recall and $F_1$, macro-averaged, with a confusion matrix to say what is failing. Two cautions carry over from there, since accuracy hides which class is wrong and micro-$F_1$ equals accuracy on single-label tasks, so reporting both tells you nothing twice. The rest of the chapter is about the open case.

Overlap metrics

The oldest workable idea is to compare $n$-grams against one or more references and count the matches.

Precision has to be clipped

Start with unigram precision, which asks what fraction of the candidate’s words appear in a reference. It is trivially gameable, so take the references “the cat is on the mat” and “there is a cat on the mat”, along with the candidate:

Every one of those seven words appears in a reference, so unclipped precision is $7/7 = 1.0000$, which is a perfect score for a sentence that says nothing. Modified precision caps each $n$-gram at the most any single reference contains, and reference one has the twice, so the ceiling is two.

Clipping is not a refinement, because without it the metric can be won by repeating one word and would measure nothing at all.

The brevity penalty

Precision still rewards saying less, as it does against the reference “the cat is on the mat”:

The first row is one word, perfectly precise and useless, and precision alone would rank it top. The brevity penalty fixes that:

$$\begin{equation} \text{BP} \;=\; \begin{cases} 1, & c > r, \\ \exp\!\big(1 - r/c\big), & c \le r, \end{cases} \label{eq:bp} \quad\text{(22.1)} \end{equation}$$

where $c$ is the candidate length and $r$ the closest reference length.

It is deliberately asymmetric, so that too short is punished while too long is not, because a long candidate is already punished by precision, as the last row shows.

BLEU, end to end

BLEU (Papineni et al. 2002) combines clipped precisions for $n = 1$ to $4$, using their geometric mean, times the brevity penalty.

Three things are worth reading off that table. The geometric mean means one zero sends the whole score to zero, so the third candidate differs from a reference by a single word and scores $0$. That severity is why BLEU is reported over a corpus rather than a sentence, since on one sentence it is close to useless.

The last candidate is the reference words in scrambled order, where the unigram precision is perfect and the higher $n$-grams collapse, and that is the only way BLEU sees word order at all.

ROUGE, and the other direction

BLEU asks how much of what you said was in the reference, which is precision, and it suits translation.

Summarisation cares about the opposite. Did you leave out the important part? That is recall, and it is what ROUGE (Lin 2004) measures.

One bookkeeping note before the table, because it changes the numbers. The BLEU table above scored against two references, as BLEU classically does, whereas the table below scores against the single reference “the cat is on the mat”. So the BLEU column here is not the BLEU column there, and only the exact match scores $1$ in both.

Row three is padded and ROUGE gives it a perfect score, because every reference $n$-gram is present. Recall alone can be won by saying everything, which is why ROUGE is normally reported with an $F$-measure variant.

The row that matters

Row four is a correct paraphrase of the reference and it shares no content word with it. BLEU gives it $0.0000$, ROUGE-2 gives it $0.0000$, and ROUGE-1 gives it $0.3333$, every point of which comes from the word the. Not one content word was credited, and the candidate is right.

Neither metric has any notion of meaning. They count string overlap and a paraphrase overlaps in the wrong places, so that single row is the argument for everything that came after.

Try it yourself. code/worked_examples/bleu.py produces every table here. --clip builds the degenerate candidate, --brevity sweeps the length term, --bleu scores four candidates in full, and --rouge runs the paraphrase.

Run it in ColabNotebookSource

Beyond surface overlap

There are two families of response, and both trade a known weakness for a new one.

Embedding-based metrics compare meaning rather than strings, so BERTScore (Zhang et al. 2020) matches candidate and reference tokens by the cosine similarity of their contextual embeddings from Chapter 15. The paraphrase row above now scores well, which is the point, and the cost is that the metric is now a model with its own training data and its own blind spots, whose number cannot be recomputed by hand.

Model-as-judge asks a strong language model to rate an output against a rubric (Zheng et al. 2023). It handles open-ended tasks nothing else can score and correlates reasonably with human preference, while inheriting every bias of the judge model, including a documented preference for longer answers and for its own outputs. A judge is a measuring instrument that has opinions, so it is useful and it must be calibrated against human judgement rather than trusted.

Human evaluation

Human judgement remains the standard everything else is validated against, and it has its own methodology. Pairwise comparison is more reliable than absolute rating, because people are better at ranking two things than at scoring one, and multiple annotators with a reported agreement statistic are necessary, because a single annotator’s number is unmeasurable. It is slow and expensive, which is why the field keeps trying to automate it and keeps having to come back.

The metric that becomes the target

One closing caution, and it applies to every number in this chapter.

When a measure becomes a target, it ceases to be a good measure.

That is Goodhart’s law in Strathern’s phrasing (Strathern 1997), and benchmark history is a long demonstration of it. Benchmarks saturate, test sets leak into pretraining corpora, and models are tuned against the leaderboard until the leaderboard stops measuring the thing it was built to measure.

No better metric defends against this. The discipline of Chapter 14 does: state your metric, hold out your test set, report the runs that failed, and look at the actual outputs. A number is a summary of an experiment and not a substitute for having looked.

Further reading.

For the original definitions, read Papineni et al. (2002) on BLEU and Lin (2004) on ROUGE. Zhang et al. (2020) introduce BERTScore. Zheng et al. (2023) study language models as judges and document their biases carefully. Jurafsky and Martin (2025) cover evaluation across tasks with more examples. In practice BLEU comes from sacrebleu, which exists because BLEU scores were for years incomparable between papers, and ROUGE from rouge_score, while nltk.translate.bleu_score is the one closest to the arithmetic above.

Code for this chapter

Each one reproduces the tables above. Open it in Colab, change an input and watch which numbers move.

• bleu.pyBLEU, ROUGE and their blind spots

  Clipped precision, the brevity penalty, and a correct paraphrase that scores zero.

  standard library only Run it in Colab Notebook Source

From Natural Language Processing: From Corpus Statistics to Grounded LLMs, by Ramaseshan Ramachandran.