What AI costs to run, and how to price it in

Tokens are the unit, and there are two prices

Models are billed per token. A token is a chunk of text, and across ordinary English it averages about three quarters of a word: a thousand tokens is roughly 750 words, or a page and a half.

There are two prices, not one. Input tokens are everything you send: the instruction, the examples, the document, the conversation so far. Output tokens are what comes back. Output is usually the more expensive of the two, often several times over, which matters because the instinct is to worry about the long document you are sending rather than the long answer you asked for.

Prices are quoted per million tokens, which makes them look like nothing. They are nothing, per call. The bill is per call times calls, and calls is the number nobody estimates.

The three things that actually drive the bill

Conversation length. Most chat interfaces resend the whole conversation on every turn, because the model has no memory between calls. A twenty-turn conversation does not cost twenty calls, it costs the sum of a growing document sent twenty times. This is the single most common surprise on a first invoice.

Retries and checks. A pipeline that asks the model to answer, then asks a second call to check the answer, has doubled its cost before anything failed. Every guardrail is a call.

What you paste in. Sending a whole document when the answer lives in one section is paying for the rest of it. Sending a hundred rows to classify one is the same mistake at scale.

Work out the cost of one job, then multiply

Do this on paper before writing code.

Take one real example of the job. Count the words going in, including the instruction and any examples, and the words you expect back. Divide each by 0.75 to get tokens. Multiply by the model's input and output price per million, and add them. That is your cost per job.

Now multiply by how many times a month that job happens. That is your run cost. Compare it against what the job costs today in salary, in delay, or in the sale you lose by being slow.

The arithmetic is deliberately crude because it does not need to be precise. If the answer is close, the project is not worth the risk. If it is off by a factor of ten in your favour, build it.

The levers, in the order they pay

Send less. Trimming what goes in is the cheapest saving there is and it usually improves the answer too: a model given one relevant section beats the same model given forty pages that contain it.

Cache what repeats. If the same long instruction goes out on every call, most providers will let you pay a reduced rate for the repeated part. On a high-volume job this is the difference between a bill you notice and one you do not.

Use a smaller model for the easy half. Classification, routing and extraction rarely need the biggest model. Reserve it for the calls where being wrong is expensive.

Cap the output. Asking for a summary in eighty words instead of leaving it open is a direct cut to the more expensive of the two prices.

Pricing it into what you sell

If the AI sits inside something you charge for, its cost belongs in the same place every other variable cost does: in gross margin, not in overheads.

The mistake is to treat it as a fixed monthly subscription because that is how the invoice arrives. It is not fixed. A customer who uses the feature ten times as much costs ten times as much to serve, and if your price is flat, your margin on that customer is not.

So cost the job, put it in the margin, and look at the spread across your customers rather than the average. If your heaviest user would cost more than they pay, you have a pricing problem to solve now, while it is one customer, rather than at renewal when it is forty.