Interactive infographic
How AI Chatbots Choose the Next Word
Move the temperature slider to see how an AI language model turns word probabilities into an answer, and why the same question can get different replies.
What this infographic shows
An AI chatbot writes by repeatedly guessing the next word. The model behind it, a large language model, has learned from huge amounts of text which words tend to follow which. For any text so far, it gives each possible next word a probability.
The bars show example probabilities for the next word after “The best thing about autumn is the”. The numbers are made up for illustration, but real models work the same way, just with tens of thousands of options.
How to use it
- Click Pick 10 next words to let the “model” choose ten times.
- Move temperature down to 0.1 and pick again. Almost every pick is “leaves”.
- Move it up to 2.0. The bars even out and rare words start to appear.
Why it matters
Temperature is one reason chatbots feel creative, and one reason they sometimes make things up. A model that picks a plausible-sounding but wrong word early in an answer keeps building on it. That is why AI answers on facts, numbers and dates are worth checking against a source.
Running these models takes a lot of computing power. See how much electricity AI data centers use.
Frequently asked questions
How does an AI chatbot decide what to write?
It predicts the next small piece of text, called a token, from everything written so far. It scores every possible token, picks one, adds it, and repeats until the answer is done.
What is temperature in AI?
A setting that controls how random the choice of the next token is. Low values make answers focused and repeatable. High values make them more varied and creative, but also more likely to drift.
Why does ChatGPT give different answers to the same question?
Because each next word is sampled from a set of probabilities rather than always taking the top choice. Different picks early on lead to different answers.
What is a token?
A chunk of text the model works with. It can be a whole word, part of a word or a punctuation mark. Common English words are often a single token.
Sources
Text version of this infographic
How an AI chatbot picks its next word
A language model scores every possible next word, then picks one. Temperature controls how adventurous that pick is.
The best thing about autumn is the …
Temperature
Pick 10 next words