All labs Animated labs The chapter Course home

Lab 02HAL, one word at a time

Hyperspace Analogue to Language slides a ten-word window along the text. A word right next to the target counts 10, a word ten away counts 1. Watch the two matrices fill and notice they are not the same matrix.

Target 0 of ·
Press play, or step through one target word at a time.

Left pass, filling columns w(a,b): what came before the target

Right pass, filling rows w(b,a): what came after the target

What just happened

Distance is information. A word sitting immediately to the left of woodchuck tells you far more about it than a word nine positions away, so HAL ramps the weight from 10 down to 1 instead of counting every neighbour once.

The two matrices are the reason this lab exists. Filling from the left and filling from the right do not produce mirror images, because "chuck wood" and "wood chuck" are different facts about the language. HAL keeps both, which is why the full HAL vector for a word is its row and its column stuck together, twice as long as the vocabulary.

This animation uses the woodchuck sentence and a ten-word window, so it is a bigger run than the one in the book: hal.py works a seven-word sentence with a window of five, small enough to check every cell by hand.

Reference: Lund and Burgess, Producing high-dimensional semantic spaces from lexical co-occurrence, Behavior Research Methods, Instruments and Computers, 28(2):203, 1996.

Read the chapter · Run the code