Prompt library
Stop shipping a deploy to fix a sentence
A system prompt hard-coded into three services is three places to edit and three chances to disagree. Store it once, call it as prompt/name, and fill the parts that change per request - editing it takes effect on the next call, everywhere.
- Call a stored prompt by name in the model field - every service that names it gets the same text
- Variables are filled at request time, and a missing one is an error rather than an empty gap in your instructions
- Expansion happens before moderation, guardrails and the cache, so a stored prompt can never smuggle text past an inspection
// Store the prompt once in the dashboard, then:
{
"model": "prompt/support-triage",
"variables": {
"product": "Checkout",
"tone": "brisk"
},
"messages": [
{ "role": "user", "content": "Card declined on the last step." }
]
}
// The stored messages are prepended, the variables filled,
// and the request routes exactly as any other would.// Fixing a prompt is not a deploy
Before: "Answer in at most three sentences."
After: "Answer in at most three sentences. Never guess a refund amount."
// Every service calling prompt/support-triage picks it up on the
// next request. No redeploy, no version skew between callers.What is a prompt library?
A prompt library moves the instruction text out of your codebase and behind a name. Instead of every service carrying its own copy of a system prompt, each one sends model: "prompt/<name>" and the gateway prepends the stored messages before routing.
The value is not storage, it is a single point of edit. Prompts are the part of an AI product that changes most often and ships slowest, because a one-word fix needs a release. Behind a name, the fix is live on the next request - and the same name works whichever model answers, because routing happens after expansion.
What the library gives you
Small feature, and it removes a whole category of drift.
One text, many callers
A prompt is a stored list of messages prepended to the request. Three services naming the same prompt send byte-identical instructions, which is exactly what they usually do not do today.
model: "prompt/support-triage"
Variables, checked
Fill the parts that differ per request - a product name, a tone, a customer tier. A variable the prompt expects and the request omits is a clear 400, not a sentence with a hole in it.
missing variables are refused, never silently blank
Inspected like anything else
Expansion runs before the content screen, the guardrail detectors and the cache key. The final text - stored prefix and substitutions included - is what gets inspected and what gets cached.
a stored prompt cannot bypass a guardrail
Independent of the model
Naming a prompt does not pin a model. Routing, policies and fallback all run afterwards, so the same prompt works on whichever model answers today and whichever answers next month.
prompt reference first, then the router
Prompt library
Best practices - the order that works
- 01
Move the prompt that changes most first
Not the biggest one - the one someone edits weekly. That is where the deploy tax is actually being paid, and where a single point of edit pays for itself immediately.
- 02
Put the stable text in the prompt, the volatile text in variables
Instructions, tone and refusals belong in the stored messages. Names, tiers and identifiers belong in variables. If you find yourself editing the prompt per request, that part should have been a variable.
- 03
Name prompts after the job, not the model
A prompt called support-triage survives a model change; one called sonnet-triage becomes a lie the first time routing picks something else.
- 04
Let the missing-variable error stay strict
It looks unhelpful the first time it fires and it is the reason you never ship a prompt with a hole where the customer's tier should be.
- 05
Test in the playground, which bills the same way
The playground runs the same dispatch path as the API, so a prompt that behaves there behaves in production - and costs the same to try.
Stored here vs hard-coded in your services
| Prompt library | Prompts in the codebase | |
|---|---|---|
| Changing a sentence | Live on the next request | A release, per service |
| Three services, one prompt | Byte-identical by construction | Three copies that drift |
| Filling per-request values | Checked variables; missing one is a 400 | String interpolation, silently blank |
| Guardrail inspection | Runs on the fully expanded text | Whatever the service happened to send |
| Switching model | Unaffected - routing runs after expansion | Often re-tuned per model |
FAQ
How do I call a stored prompt?
Put its name in the model field: model: "prompt/support-triage". The stored messages are prepended to the messages you send, variables are substituted, and the request then routes normally - so you can still let the router choose the model.
What happens if the prompt does not exist?
A clear error naming the prompt and pointing at your library, exactly like an unknown routing policy. It never silently falls through to an unprefixed request - a prompt you thought was applied and was not is worse than a refusal.
Can a stored prompt get past my guardrails?
No, and the ordering is deliberate. Expansion happens before the content screen, before the injection and sensitive-data detectors, and before the cache key is computed. Everything downstream sees the final text, prefix included.
How many prompts can I store?
Three on a free account, one hundred once you have bought credit. Editing what you already have is never capped - a limit that stops you fixing your own configuration would be the wrong kind of limit.
Does naming a prompt lock me to one model?
No. The prompt reference and the model choice are separate: after expansion the request routes by whatever strategy or policy applies, so a stored prompt works across every model the key can reach.
Edit the prompt, not the deploy pipeline
Store one prompt in the dashboard, call it by name from two services, and change it once to change both.
Related: smart LLM routing, AI guardrails, structured output, the API reference.