Probability Distribution
ai-engineering llm statistics
A set of scores assigned to every possible next token, representing how likely each one is to come next given everything before it.
↑ ContentsDefinition
When a language model produces the next token, it does not pick from a hat. It assigns a probability to every token in its vocabulary based on the tokens it has already seen. Some tokens get high scores, most get near-zero. The model samples from this distribution to produce the next token. Temperature and top-p settings control how that sampling works: lower temperature makes the model stick to the highest-scoring tokens, higher temperature lets less probable tokens through.
↑ Contents