AI spend management
The invoice should never be the first alert
Every limit here is enforced at the gateway, before the request runs - a monthly ceiling for the account, a daily one for each key, and a per-session stop for the agent loop that goes wrong at 2am. The bill becomes a confirmation, not a discovery.
- Budgets count list price at request time - the same arithmetic the biller uses, so the cap and the invoice can never disagree
- Daily caps can downgrade to the cheapest allowed model instead of refusing - degraded service beats no service
- Alerts at 50, 75, 90 and 100% fire once each, plus a pace projection that warns before the overrun, not after
What is AI spend management?
AI spend management is keeping model costs predictable while usage is not. LLM traffic is spiky by nature - an agent loop, a viral feature, a customer's batch job - and per-provider dashboards only tell you about the spike after it has been billed.
The gateway is the one place every request already passes through, so it is the one place a limit can act before the spend exists. Final Router enforces ceilings at three levels - account, key and session - and pairs them with attribution (which app, which customer, which run) so the number on the invoice decomposes into causes.
Three ceilings and the receipts between them
Each control ends with its concrete behaviour - the status code, the header, the arithmetic.
A monthly budget with teeth
Set a cap and requests are refused once it is reached, so a loop in your code cannot run up an unbounded bill. Counted at list price, at request time.
403 budget_exceeded - distinct from 402 out-of-credits, on purpose
Daily caps that degrade gracefully
A per-key rolling daily cap can refuse - or downgrade to the cheapest model the key allows, keeping the product alive at reduced cost until midnight.
the downgrade is stated in final_router.reason
Session stops for runaway loops
Tag agent runs with X-Session-Id and give each session its own spend and request limits. The retry loop that would have burned the daily budget stops at fifty cents.
per-session caps, enforced mid-run
Alerts that respect your inbox
Budget tiers at 50/75/90/100% each fire exactly once, a pace projection warns when the month is trending past the cap, and a capped key warns at 80% of its limit.
one email per sweep, mirrored to a webhook
Attribution built in
X-Title names the app, sessions group the runs, user and tags label your own customers - so analytics splits the bill by cause instead of presenting one blended number.
cost per app, session, tag and model in analytics
Auto top-up that stays bounded
Running out mid-incident is its own outage, so credit can refill itself - but only after explicit consent naming the threshold and the amount, and never more than ten charges in a day. Budgets still refuse on top.
consent recorded · hard cap of 10 charges a day
Keys that expire
A key for a contractor, a demo or an experiment gets an expiry date and revokes itself - the credential you forgot about stops being a liability on schedule.
expired keys answer 401, immediately
AI spend management
Best practices - the order that works
- 01
Set the monthly budget first
One number, five minutes, and the worst case is bounded forever. Everything else refines the ceiling - this one creates it.
- 02
One key per app, cap included
Per-key daily caps only mean something when a key maps to one workload. Split keys by product and environment, and give each a cap sized to its normal day.
- 03
Prefer downgrade over refusal for user-facing keys
A customer who gets a cheaper model at 5pm is a customer; one who gets a 403 is a ticket. Save hard refusal for batch and internal keys.
- 04
Cap sessions on anything agentic
Agents fail as loops, and loops burn money between alert emails. A per-session limit is the only control fast enough for that failure mode.
- 05
Read the pace alert, not just the tiers
Crossing 50% on the 25th is fine; crossing it on the 5th is the real signal. The projection alert exists for exactly that difference.
Enforced at the gateway vs reconciled afterwards
| Final Router spend controls | Per-provider dashboards | |
|---|---|---|
| When limits act | Before the request runs | After the spend appears in billing |
| Runaway agent loop | Stopped mid-session at the session cap | Discovered on the invoice |
| Granularity | Account, key and session ceilings | Account-level, per provider |
| Attribution | App, session, customer and tag on every request | One number per provider, split by hand |
| Budget arithmetic | Same list-price math as the biller - cap and invoice agree | N providers, N currencies of 'usage' |
FAQ
What happens when a budget is hit - exactly?
The monthly budget returns 403 budget_exceeded, deliberately distinct from 402 insufficient_credits: an empty balance is fixed by topping up, but a budget is your own configured ceiling, and a client that auto-bought credits against it would be spending money your settings still refuse. Daily key caps either refuse or, if you enabled it, downgrade to the cheapest allowed model - stated in the response.
Can spending degrade instead of stopping?
Yes - that is the downgrade-to-cheapest option on per-key daily caps. The chain is rebuilt cheapest-first from the models the key already allows, so every allowlist still holds, and the response says the downgrade happened rather than substituting silently.
How do I stop one agent run from draining the day?
Send an X-Session-Id header per run and set per-session spend and request limits on the key. A retry loop then stops at the session ceiling - typically cents - instead of consuming the whole daily cap between alert emails.
Do alerts spam me as spend climbs?
No. Each tier - 50, 75, 90, 100% - fires exactly once per month, the pace projection fires when trend crosses the cap, and everything in a sweep arrives as one email, optionally mirrored to a Slack-compatible webhook.
Does auto top-up defeat the point of caps?
It is designed not to: auto top-up requires explicit consent naming the amounts, is hard-capped at ten charges per day, and budgets still apply on top - 403 budget_exceeded fires regardless of balance. Credits answer 'can we pay'; budgets answer 'do we want to'.
Give the worst case a ceiling today
Set one monthly number, cap the keys that run unattended, and let the alerts - not the invoice - tell you how the month is going.
Related: AI guardrails, LLM observability, smart LLM routing, pricing.