What Are AI Tokens? A Plain-English Guide to How LLMs Count (and Charge for) Text

I still remember the first time I opened an API bill and saw it split into "input tokens" and "output tokens." My first thought: tokens of what, exactly, and why am I being charged for punctuation?

Turns out the answer is simpler than the pricing page makes it look. A token is the smallest chunk of text a model reads or writes. Not quite a word, not quite a character, somewhere in between. And you're not paying for the text. You're paying for what the model has to do once that text is in front of it.

Let's get into what a token actually is, then why the bill looks the way it does.

What is a token, exactly?

A token is a small unit of text, often a word, part of a word, or a punctuation mark, that the model handles as one piece.

A few quick examples:

• "Hello" → 1 token
• "Hello, world!" → about 3-4 tokens
• "ChatGPT is amazing." → about 5 tokens

For English, the rule of thumb I use:

• 1 token ≈ 0.75 words
• 100 tokens ≈ 75 words
• 1,000 tokens ≈ 750 words

Code doesn't follow the same math. Punctuation, brackets, and keywords usually count as their own tokens, so a single line of code can burn through more tokens than a full sentence of plain English.

Why not just count words?

Because the model isn't reading words the way you are. Everything gets converted into tokens first. It's just a tighter, more consistent way to represent language than raw words or raw characters would be.

How common a word is matters a lot here. "The" is almost always one token. Something rare or long gets chopped up. Unbelievable might land as one or two tokens depending on the tokenizer. Internationalization usually gets split into several pieces.

That's the reason token counts and word counts never quite line up, and why a paragraph full of jargon can cost more tokens than a casual one the same length.

Why do tokens cost money?

Here's the part I got wrong for a while: you're not paying for the token itself. You're paying for what happens to it inside the model.

Every token, whether you typed it or the model generated it, goes through the same expensive pipeline:

First it gets embedded, turned into a long list of numbers the model can actually operate on. Then attention kicks in: the token gets compared against a pile of previous tokens to figure out what's relevant, and the longer the conversation, the more comparisons that takes. From there it runs through dozens or hundreds of transformer layers, each packed with billions of parameters. And at the end, the model has to weigh probabilities across a vocabulary of maybe a few hundred thousand possible next tokens before it picks one.

That whole pipeline runs on GPUs and other accelerators that cost millions of dollars and pull a lot of power. The token is just what you're counting. The price is for the compute those four steps burn through.

Why are output tokens more expensive than input tokens?

Nearly every provider charges more per output token than per input token, and it's not because they're padding margins. It falls straight out of how generation actually works.

Input gets processed once, in one pass.

Output is a different animal. For every single token it generates, the model has to reprocess the whole conversation so far, predict the next token, feed that token back into the context, and start the cycle over for the one after it.

Ask for 500 output tokens and you're asking for 500 separate trips through that cycle, each one depending on everything that came before, including tokens the model just wrote a moment earlier. That's the real reason output costs more: you're not paying for 500 tokens, you're paying for 500 passes through the model.

Why are newer models more expensive?

A handful of things stack on top of each other here. Newer models tend to have more parameters, so there's more math per token. Many of them reason through intermediate steps before answering, which is its own token cost hiding inside the "thinking." Context windows have gotten enormous, some now handle hundreds of thousands or even millions of tokens, and attention cost climbs with however much context is actually in play. And a lot of the accuracy gains people notice come from spending more compute per response, not just from a bigger training run.

I don't think that makes older or smaller models pointless. For simple, well-defined jobs they're often the cheaper, saner choice. But once you need long context, harder reasoning, or higher accuracy, you're paying for the extra compute that makes it possible. There's no way around that trade.

An analogy that makes it click

Picture asking an expert lawyer a question in person.

Each token is a word you say, or a word the lawyer says back. You're not paying for the words. You're paying for the lawyer to read every one of them, hold the whole conversation in their head, reason through your problem, and hand you a useful answer. Say more, and there's more for them to read, more to hold onto, more to work through before they respond.

That's basically what's happening with AI tokens. The price reflects the work of understanding and producing text, not the text sitting there on the page.

Why this matters if you're building with LLMs

Once tokens stop looking like "units of text" and start looking like "units of computation," a few habits just make sense.

Trim your prompts. Every extra sentence sitting in a system prompt gets reprocessed on every single call, forever, not just once. Watch your context window too. A long conversation history means every new message pays the attention cost of everything said before it. Cap your output length when you can; if you only need three sentences, ask for three sentences, because output is the expensive half of the bill. And pick the model to fit the job. A cheap, smaller model handles classification or extraction just fine. Save the frontier model for the work that actually needs the extra reasoning.

Tokens aren't a billing trick. They're a pretty honest readout of how much work your request actually caused, which is exactly why it's worth understanding them before your next invoice shows up.