AI Gateway
One endpoint between your product and every model
An AI gateway is the layer that sits between your applications and the LLM providers: connect once, and routing, fallback, spend control and visibility apply to every request. Final Router is that layer - priced at each provider's own list rate, with the receipts to prove it.
What the gateway does on every request
Not a proxy that forwards traffic - a layer that makes decisions and writes them down. Every response carries a final_router block that says which model answered, why, and what it cost.
Smart routing
Four strategies score every eligible model on price, latency and quality; named policies pin exact chains when you need control.
routing: "cost" | "latency" | "quality" | "balanced"
Automatic fallback
When a provider fails or throttles, the chain walks to the next model - tools and tool history intact - before your user notices.
final_router.fell_back_from names every model tried
Spend caps that degrade gracefully
Monthly budgets, per-key daily caps that can downgrade to the cheapest allowed model instead of refusing, and per-session stops for runaway loops.
402 / 403 with the reason named, or a cheaper model
Guardrails
Prompt-injection detection that decodes base64, hex and letter-scramble evasions before matching; sensitive-data inspection; per-key attachment.
detectors run before any request reaches a provider
Cost visibility
Every request logged with cost, latency and outcome; analytics by app, session, tag and model; prompt text stored only if you opt in.
final_router.cost_cents on every response
Three-layer caching
Provider prompt caching passed through at the provider's discount, an exact-match response cache, and a semantic cache - each with a receipt.
cache.saved_usd is signed - writes show their cost
Connected in the time it takes to read this
If your code speaks OpenAI's API, it already speaks ours. Unknown fields are refused with named 400s rather than silently dropped - a minute of reading an error instead of an afternoon of debugging.
- 1
Create a key
Sign up and mint an fr_live_ key. The free tier gives 25 requests a day before any card.
- 2
Swap the base URL
Point your existing OpenAI SDK at finalrouter.com/api/v1. Nothing else changes.
- 3
Pick a model - or don't
A catalogue id pins a model, "auto" lets the router choose, provider/best never goes stale.
- 4
Ship
Routing, fallback, caps and logging now apply to every request, and each response explains itself.
import OpenAI from "openai";
const client = new OpenAI({
baseURL: "https://finalrouter.com/api/v1", // the only change
apiKey: process.env.FINAL_ROUTER_KEY,
});The numbers behind the claims
8
Providers behind one endpoint, OpenAI to xAI
0%
Markup on tokens - every model billed at its published list price
5 min
Between health probes on every provider, feeding routing and the public status page
17
Chat models, each capability live-tested - not transcribed from docs
Gateway vs wiring providers directly
Direct integration works fine - for the first provider. The gateway earns its place when the second one arrives.
| Through Final Router | Direct to each provider | |
|---|---|---|
| Integrations to maintain | One, OpenAI-compatible | One per provider, each with its own dialect |
| Provider outage | Fallback chain answers from the next healthy model | Your requests fail until you ship a workaround |
| Spend control | Budgets, per-key and per-session caps, enforced before the request | Per-provider dashboards, reconciled by hand |
| New model ships | provider/best resolves to it - no redeploy | Code change and release per application |
| Cost accounting | One log, one currency, cost on every response | N invoices in N formats |
| Token price | Provider list price, 0% markup | Provider list price |
Questions engineers actually ask
Is Final Router an AI gateway or an API gateway?
An AI gateway. An API gateway routes HTTP between services; an AI gateway is purpose-built for LLM traffic - model routing, fallback, token accounting, prompt-level guardrails and caching. Final Router does all of that behind one OpenAI-compatible endpoint.
What does it cost on top of the models?
Tokens are billed at each provider's published list price with no markup. A 5.5% fee (minimum $0.80) applies when you buy credits, and BYOK usage carries a 5% fee after a $25,000 monthly free allowance. Every model's rate is published on the pricing page with the date it was checked.
Which models and providers are behind the endpoint?
Twenty chat models across OpenAI, Anthropic, Google, Meta, Mistral, DeepSeek, Moonshot and xAI, plus two embedding models. GET /v1/models returns each one with its prices, context window and live-tested capabilities.
What happens when a provider goes down?
The routing decision always carries a fallback chain. If the chosen model fails or rate-limits, the gateway walks to the next one - preserving tools and tool history - and the response names what it fell back from. Scheduled probes feed the same signal to routing and to the public status page.
Do you train on or retain our prompts?
We never train on your data. Prompt and response text is not stored unless you switch storage on, and then it is kept 30 days and deletable instantly. Each provider's own trains-on-prompts policy is recorded in the catalogue, and models that train on API traffic ship switched off by default.
Can our agents and AI tools use the gateway too?
Yes - twice over. Your agents call the same endpoint as any client, and AI coding tools can query the public MCP server at /api/mcp for live models, prices and provider health while they write your integration.
One base URL. Every model.
Swap the endpoint, keep your code, and let the gateway carry routing, fallback and spend control from the first request.
Go deeper: smart LLM routing, LLM caching, LLM observability, AI guardrails, AI spend management, the EU gateway, or the API reference.