6How does it work?

The guess-what's-next game

In chapter 3, AI carried your tune on. How did it do that?

Let's try it with words. Pick what comes next and grow the sentence.

There's a bar beside each word. A longer bar means a more likely word.

Where did the bar come from? It's how many times the machine counted it while reading. If runs followed Poco nine times, that bar is long.

Press Let AI carry on. The machine picks exactly the way you did.

Guessing what comes next like this is called .

Pictures work this way too. Guessing what comes next — that's what AI does.

A little deeper

For grown-ups, and for children who want more.

Those bars are an n-gram language model, the method Shannon used when he founded information theory in the 1940s. It is where language models begin.

Look at one previous word to choose the next and it is a 2-gram; look at two and it is a 3-gram. The bar lengths are not invented — they are counts taken from a real corpus.

Today's large language models do the same job: guess the next piece. The difference is that a neural network learns the relationship as a function instead of a table storing it. With a table, ten words of context already produce more cases than there are atoms in the universe.

Making a machine very good at only that, and finding conversation, summary and translation came along with it, is the biggest discovery of recent years. Nobody expected it.

The moment a child notices this is the same method as chapter 3 is the crux of part 2. There it counted the next note; here the next word. Same method, different material — that is the key to understanding AI.