---
published: true
id: probability-distribution
title: Probability Distribution
description: Knowledge captured in João Lopes' learning graph.
tags:
- ai-engineering
- llm
- statistics
locale: en
resource: https://joaolopes.dev.br/knowledge/concepts/en/probability-distribution/
created: '2026-04-05'
---

# Probability Distribution
A set of scores assigned to every possible next token, representing how likely each one is to come next given everything before it.

## Definition
When a language model produces the next token, it does not pick from a hat. It assigns a probability to every token in its vocabulary based on the tokens it has already seen. Some tokens get high scores, most get near-zero. The model samples from this distribution to produce the next token. Temperature and top-p settings control how that sampling works: lower temperature makes the model stick to the highest-scoring tokens, higher temperature lets less probable tokens through.

## Source
- [Practical Prompt Engineering — Temperature, Top P, Tokens and Context](https://sgoldfarb2.github.io/practical-prompt-engineering/lessons/introduction/temperature-top-p-tokens-and-context.html)

## Related
- [Tokens](/concepts/en/tokens.md)
- [Vocabulary](/concepts/en/vocabulary.md)
- [Model](/concepts/en/model.md)