Every pricing model I've built for these products started with a spreadsheet and an assumption: “a typical customer will use this about this much, and it'll cost us about this much per customer to serve.” Every one of those spreadsheets was wrong within the first quarter of real usage. The math wasn't bad. The assumption about how customers would actually use the product was a guess dressed up as a forecast.
Across Sustainability Intelligence, Carbon Pre-Audit, Carbon Credit Management, and Sustainability Benchmarking, the cost curve never looked like the demo suggested it would. Getting it under control turned out to be a mix of engineering, pricing structure, and, more than either of those, actually knowing where the money was going in the first place.
The demo doesn't have a cost curve. Production does.
In a demo, cost is invisible. You run the thing a handful of times, it works, everyone's happy. Nobody asks “what does this cost us per run, and does that go up or down as usage grows.” That question doesn't have an answer until real customers are using the product in patterns nobody designed for, which, as we covered in an earlier post, is exactly how one customer selecting “refresh all” against 2,400 regulations in Sustainability Intelligence turned into a cost and latency spike out of nowhere. That was a design problem with a design fix. But it was also the moment cost stopped being a line item we'd think about later and became something we had to actually engineer around, continuously, everywhere.
Not every question deserves the flagship model
Carbon Pre-Audit runs a lot of checks that look similar on the surface but aren't equally hard. Some are genuinely ambiguous. Does this activity belong in Scope 1 or Scope 3 given an unusual ownership structure, is this emission factor still the right one for a recently updated methodology. Others are much more mechanical. Does this value fall inside a known plausible range, is this unit consistent with the rest of the dataset, is this field populated at all. Early on, all of it went through the same model call, because that was simpler to build. It was also expensive in a way that didn't map to the actual difficulty of the work: we were paying flagship-model prices for questions a much smaller, cheaper model could answer just as reliably.
Splitting those apart, routing the mechanical checks to a smaller model and reserving the flagship model for the genuinely ambiguous judgment calls, cut cost substantially without giving anything up on quality, because the smaller model was never the weak link on those questions to begin with. The routing logic was the easy part. The real work was being honest, question by question, about which category something actually belonged to, instead of defaulting to “just use the best model for everything” because that's the easier engineering decision to make once and stop thinking about.
Caching turns a one-time cost into an ongoing asset
We talked in an earlier post about the human checkpoint in Carbon Credit Management and Sustainability Benchmarking, where the model proposes a reconciliation or a peer-normalization decision, a human confirms it, and the system records it. What we didn't talk about is what that unlocks on the cost side. Once a peer company's normalization has been reviewed and confirmed, every future benchmark that includes that peer doesn't need to redo that work. It reuses the confirmed result. The expensive, judgment-heavy model work happens once per peer, not once per comparison. The same logic applies to Carbon Credit Management: a registry's quirks, once understood and encoded, don't need to be re-derived by the model on every reconciliation that touches that registry again.
The mistake we made initially was treating every request as independent, because that's the simplest mental model and it matches how a demo behaves. In production, a meaningful share of what looks like “new” work is actually a repeat of something the system has already figured out. Recognizing that and caching it did more for cost at real volume than any amount of prompt optimization. The trick was caching not the literal repeated API calls but the decisions the model had already made and had confirmed.
Someone has to eat the cost curve, and it needs to be decided before the contract is signed
The conversation we got wrong most often early on was with the client, not with the model. A flat monthly fee is simple to sell and simple for a client to budget against, and it's also a bet that usage stays roughly where you modeled it. When a client's usage pattern turned out to be far heavier than the volume we'd built the pricing around, more regulations tracked, more peers benchmarked, more credits reconciled, we were the ones absorbing the difference, because nothing in the agreement said otherwise.
The fix wasn't complicated once we actually made the decision deliberately instead of by default: know your cost per unit of actual usage, per regulation tracked, per peer benchmarked, per credit reconciled, well enough to price against it, and decide upfront, in the contract, whether usage above a threshold is included, capped, or metered. That's a business decision rather than an engineering one, but it's impossible to make well if engineering can't tell the business what a unit of usage actually costs. The two functions that used to operate almost independently, what we build and what we charge for it, had to start talking to each other continuously, not just at renewal time.
What actually keeps the economics sane
Match model tier to task difficulty, deliberately, question by question. Defaulting to the flagship model for everything is the easy engineering choice and the expensive business one.
Cache decisions, not just responses. A confirmed judgment call, a normalization, a registry mapping, a rule, shouldn't cost the same to reuse as it did to produce the first time.
Decide who absorbs usage above the modeled baseline before the contract is signed. A flat fee without a usage ceiling is an open-ended bet on your own forecast being right.
Treat pricing and engineering as one conversation, not two. The business can't price usage it doesn't understand the cost of, and engineering can't optimize cost without knowing what usage actually needs to be affordable.
The takeaway
The cost curve doesn't show up in a demo because a demo doesn't have real usage patterns, real scale, or a real contract behind it. Production has all three, and each one bends the cost curve in a direction the original pricing spreadsheet didn't anticipate. Getting ahead of it means treating cost as a design constraint from day one: in the model choices you make, in what you cache, in how you price, and in whether you can actually see where the money goes once real customers show up.
That's the fourth lesson from the front line. Last one in the series: a pragmatic architecture for shipping production AI apps for clients, what to build in from day one, what to deliberately leave out of v1, and where to draw the line on scope.
