A single model call costs a fraction of a cent, which makes AI features feel almost free in a demo. Then the feature ships, traffic arrives, and the fraction gets multiplied by a number nobody estimated. The costs that surprise teams are not mysterious — they are just easy to skip when the prototype was cheap.
Here is where the money actually goes, and how to keep it sensible.
Cost scales with usage, and usage grows quietly
Per-request pricing looks trivial per request. Multiplied by every user, every day, retried on failure, it becomes a real line item. The time to estimate this is before launch: how many calls, how large, how often, at the traffic you actually expect — not the traffic you have in testing.
The size of the request matters as much as the number
Cost is driven largely by how much text goes in and comes out. Stuffing a huge amount of context into every call, or letting responses run long, multiplies the bill without necessarily improving the result. Sending only the context a request needs is both cheaper and usually better.
- Cache what repeats — identical or near-identical requests do not need a fresh call each time.
- Right-size the model — many tasks run fine on a smaller, cheaper model, with the largest one reserved for the cases that need it.
- Trim the context — retrieve what is relevant, not everything you have.
The hidden costs are not the model
The obvious cost is the model call. The quieter ones are the infrastructure around it — storing and indexing data for retrieval, the extra calls a single user action can trigger, and the engineering time to keep it all working. These rarely show up in an early estimate and often outweigh the raw model cost.
Cost is a design input for AI features, not a bill you read afterwards. The choices that control it are made before launch.
Decide the trade-offs on purpose
Cheaper, faster, and higher quality pull against each other, and the right balance depends on the feature. A background task can use a cheaper, slower path; a feature a user is waiting on may justify a more expensive one. Making these trade-offs deliberately, per feature, is what keeps an AI product both good and affordable — rather than discovering the balance from an invoice.