AI & DataBeginner
Token
PronunciationTOH-ken
Definition
A small piece of text, like part of a word, that an LLM reads and writes. Limits and prices are measured in tokens.
Where you hear it
AI pricing and context limits. Note: "token" also means a login token in security.
Examples
The prompt is 2,000 tokens, so it fits in the context window.
We limit the output to 500 tokens to control cost.
Common mistake
Confusing an AI token with an authentication token. The same word, different meanings.