In One Sentence

A token is the smallest unit of text a language model works with, roughly a word fragment from the model’s point of view. Models read tokens in and write tokens out, and every token costs compute, which is why providers bill by token count.

Why Models Don’t Just Read Characters

Computers work with numbers. To process text, a model first splits it into units and maps each unit to an ID. That step is called tokenization.

There are a few ways to choose the unit size:

  • Characters: a tiny vocabulary, but very long sequences, so the model needs many more steps to get through a sentence.
  • Whole words: short sequences, but an unmanageably large vocabulary, and no way to handle new words or typos.
  • Subwords: the middle ground, and what mainstream LLMs use. Common words stay whole; rare words are split into smaller pieces.

A typical subword algorithm is BPE (byte pair encoding) and its variants. It starts from single characters or bytes, finds the pairs that most often appear together in training text, and merges them into new vocabulary entries, repeating until the vocabulary reaches a target size. Frequent pieces like “the” or “ing” become single tokens, while a rare technical term might be split into three or four.

How Much Text Is One Token?

There’s no fixed answer. It depends on the model’s tokenizer and the text itself. Some general patterns:

  • English: common words are usually one token. Long or rare words get split, and spaces and punctuation take tokens too.
  • Chinese and other non-Latin scripts: tokenizers vary a lot. Some older tokenizers handled these scripts poorly, so a single character could take more than one token. Newer models often have larger vocabularies that cover common words directly.
  • Code, numbers and emoji: often more token-hungry than ordinary prose.

The same passage can produce different token counts on different models. For exact numbers, use the provider’s tokenizer tool or the usage figures returned by the API.

Tokens and the Context Window

A model can only “see” a limited amount at once. That limit is the context window, also measured in tokens. It covers the system prompt, conversation history, uploaded documents, tool results and the answer currently being generated.

Anything outside the window is invisible to the model. That’s why a long chat can seem to “forget” its beginning: the app trims or compresses early content to make room.

Why Billing Is Per Token

Compute cost tracks token count directly:

Input takes compute. The model has to process every token you send and work out how they relate. Longer input means more work.

Output costs more. Generation happens one token at a time, and each new token requires a full pass through the model. That’s why most providers price output tokens higher than input tokens.

Usage-based pricing is fairer. One person sends “hi,” another uploads a whole book to summarize. Per-request pricing can’t cover both sensibly. Per-token pricing ties cost to actual usage.

Many providers also offer prompt caching: when repeated requests share an identical prefix, such as a long system prompt, the repeated part can be billed at a lower rate. Check each provider’s docs for the exact rules.

Estimating and Saving Tokens

  • Trim your prompt: cut repeated instructions and irrelevant background.
  • Manage history: in multi-turn chats, keep only the context you need, or summarize older turns.
  • Cap the output: ask for “three sentences,” or set a maximum output token parameter.
  • Retrieve instead of pasting: pull the relevant passages rather than an entire document.
  • Right-size the model: simple tasks often run fine on a smaller, cheaper model.
  • Read the usage stats: API responses usually report input and output tokens per call. Budget from those.

Common Misconceptions

“One token equals one word or one character.” A token is whatever piece the tokenizer produces: a character, half a word, or a common phrase.

“Every model counts tokens the same way.” Different tokenizers can give noticeably different counts for the same text. Keep that in mind when comparing prices.

“Only what I type gets billed.” System prompts, conversation history, tool results and model output all count. For models that do internal reasoning, reasoning tokens are usually billed as well.

“A bigger context window means I can paste anything.” More room also means higher cost, slower responses, and sometimes weaker focus on what matters.

Further Reading