Natural language processing is the job of getting computers to work with human language, which means reading it, writing it, translating it, answering questions about it, and now holding a conversation in it. Put that way it sounds like any other programming problem, and the fact that it is not is the subject of this book.
A programming language such as Python was designed to be unambiguous, so every well formed program has exactly one meaning, fixed by a grammar of a few pages that a compiler works out mechanically. Human language was designed by nobody. It grew instead, and at every level it is full of the one thing a compiler cannot tolerate, which is ambiguity.
That gives us the first idea to absorb. Understanding language is an act of disambiguation rather than a lookup, and the information you need in order to disambiguate is often nowhere in the sentence at all.
Take the word bank, which is a place that keeps money, the side of a river, what a plane does when it turns, and also a verb meaning to rely on something. You picked the right sense a moment ago without noticing, because the surrounding words supplied it. On its own, though, the word really is ambiguous four ways, and a machine has to settle that question before it can do anything else. This is lexical ambiguity, and almost every common word has some.
Ambiguity gets worse when words combine. Consider “I saw the man with the telescope”, which has two grammatical structures, because either I had the telescope or the man did, and English gives you no rule for choosing between them. Only knowledge of the situation does that.
Newspaper headlines show the effect in its purest form, because editors squeeze the words so hard:
“Juvenile court to try shooting defendant”
“Local high school dropouts cut in half”
“Safety experts say school bus passengers should be belted”
Each headline carries a sensible reading and an absurd one, and both are equally grammatical. What rules out the absurd reading is what we know about the world. These are syntactic and semantic ambiguity, and between them they explain why parsing a sentence cannot be separated from understanding it.
Past the sentence lies pragmatics, which is meaning that depends on context, intent and convention. “Can you pass the salt?” is a question about your abilities only if you read it literally, whereas at a dinner table it is plainly a request. Sarcasm flips meaning outright, so that “Well, that went great” after a disaster means the opposite of what it says, and even people find this hard when tone is the only clue they have.
The same lesson keeps coming back. The words on the page do not pin down the meaning, and closing that gap takes knowledge the text assumes but never states. That is why NLP is hard. Language is an efficient code between minds that already share a world, and the computer starts out sharing none of it.
Faced with a problem that large, the field breaks it into tasks, where a task is a specific, measurable job that carves off one piece of the whole. You will meet many of them in this book, so it is worth naming a few now.
Sentiment analysis labels a text by the attitude it expresses, asking whether a review is positive or negative. Question answering returns the answer to a question, drawing either on a passage you supply or on general knowledge. Given “The cat sat on the mat” and asked “Where did the cat sit?”, it should answer the mat. Textual entailment decides whether one sentence follows from another, and machine translation turns text in one language into another. Named entity recognition finds the people, places and organisations in a text, while summarisation shortens a text and text generation continues it. The list runs on through speech recognition, spelling correction and dialogue.
What holds the list together is the shape of each item, because every task has defined inputs, expected outputs, and a way to score how well a system did. That last part is what makes progress measurable.
The tasks themselves have barely changed in decades, but the way they are solved has changed completely. For most of the field’s history each task had its own specialised machinery, whereas now a single kind of model handles most of them, often just by being asked. That model is a large language model, and its pretraining does one thing, which is to predict the next token. Later stages adjust which continuations it prefers, as Chapter 20 describes, and the objective underneath stays the same. ChatGPT, Claude and Gemini are the visible surface of that change.
This book tells the story of how the field got from counting words to models that converse, and it follows one idea deepening rather than a catalogue of techniques. The idea is that language resists rules but yields to numbers and to learning.
The story has four movements, and they carry the first five parts that follow. First comes language as data, where we gather text into corpora, clean it, and then measure the statistical regularities that refuse to go away, which is the rest of Part I. Second comes meaning as geometry, where we turn words into vectors so that similarity of meaning becomes nearness in space, and that is Part II. Third come language models, where we learn to assign probabilities to word sequences, beginning with plain counting and building through neural and recurrent models across Part III. Fourth comes the transformer era, meaning the self attention architecture, the large language models it made possible, and the craft of working with them, which occupies Parts IV and V.
Two shorter parts close the book. Part VI puts the machinery to work on translation and summarisation, and Part VII asks what these systems cost and whom they can harm.
Each movement keeps what the previous one built and adds one new ability, so by the end you will see what the model of Chapter 18 really is. It is a direct descendant of the word counting in Chapter 10, answering the same question, what comes next?, with machinery that has grown far more powerful.
One word on how to read this. The text explains how each idea works, and the problems at the end of every chapter make you prove it, usually by computing a small case by hand. In this subject a worked example is almost always the fastest route to understanding.
We start, in the next chapter, with the raw material for everything that follows. The corpus.
Jurafsky and Martin’s Speech and Language Processing (Jurafsky and Martin 2025) is the standard textbook and a good companion to this one. Manning and Schütze’s Foundations of Statistical Natural Language Processing (Manning and Schütze 1999) remains the classic treatment of the statistical foundations, and the Handbook of Natural Language Processing (Indurkhya and Damerau 2010) surveys the full breadth of the field’s tasks.
Find the ambiguity. For each sentence, describe the two readings and say what knowledge a reader uses to prefer one: (a) “The chicken is ready to eat.” (b) “Flying planes can be dangerous.” (c) “She told her mother that she had won.” Which of the three is settled by world knowledge rather than by grammar?
Programming versus natural language. List three concrete properties of a programming language that make it mechanically interpretable, and for each one give the property of English that makes it not. Then say why this difference means that parsing a sentence cannot be cleanly separated from understanding it.
Turn a goal into a task. Pick something you would like a computer to do, such as flagging toxic comments or answering questions about a manual, and specify it as a concrete NLP task. What are the inputs, and what are the outputs? Hardest of all, how would you score whether a system did it well, and why is the scoring definition the part that makes progress possible?
Lexical ambiguity census. Take the word run and list as many distinct senses as you can, both noun and verb. For three of them, write a short sentence and underline the context words that settle the meaning. What does the length of your list suggest about representing a word by one fixed meaning? Chapter 15 returns to that difficulty.
Each one reproduces the tables above. Open it in Colab, change an input and watch which numbers move.
bridging_the_gap.pyOne request, three answers
Show me the common words in this file. Nothing in that sentence says what a word is, or what common means. Three faithful readings, three different answers.
standard library only Run it in Colab Notebook SourceFrom Natural Language Processing: From Corpus Statistics to Grounded LLMs, by Ramaseshan Ramachandran.