Use case
Broker model access
Using Dome's LLM Router
The model is the most volatile part of an agent. Brokering it moves routing, cost and failover below the agent, so switching providers stops being a release. Seven resources do that work, and the agent asks for a capability rather than a vendor.
Composed from
Seven resources, one routing decision
Gateways, Models, Pools, Rules, Guards, Quotas and Audits. The first of those is the same resource the tool gateway grants, so an agent that already reaches tools through Dome reaches models the same way.
The same granted surface the tools use, pointed at model providers instead.
A connection to a provider's model. Credentials, region, and the limits it arrives with.
Several models behind one name, with priorities, weights, caching and failover.
Which pool a call resolves to. Conditions on the agent, the caller, the tier, the context size.
Prompt and completion inspection on the way through.
Spend and token ceilings per agent, per team, per window.
Provider, model, pool, agent, caller, tokens, outcome. Cost becomes a query.
Nothing here belongs only to the LLM Router. One policy language, one audit vocabulary, one access layer, across every composition. See the whole resource set.
Scenario
Three teams, three contracts
Sales, support and ops each signed their own provider contract. Spend lands in three invoices, evaluations are guesswork, and one bad hour at a frontier provider takes out half the estate.
What makes it hard
Nobody owns model spend
Not the way someone owns the cloud bill. Per-team keys and personal accounts make the total unknowable.
The cheap call and the hard call cost the same
Everything routes to whichever frontier model the agent was first written against.
Evaluations are synthetic
A published benchmark is not your workload. The comparison you want runs on your own traffic.
Failover is a code change under pressure
When the provider degrades, every agent that names it degrades with it.
The work, in resources
Models. Register each provider connection once. Keys come from the secrets store you already run.
Pools. Group models behind one name with priorities and weights. A pool resolves to whatever is healthy and cheapest.
Gateways. Grant the surface to the agent. It calls one endpoint, and the provider behind it is configuration.
Rules. Route on the agent, the caller, or the context size. Pin sensitive work to an approved provider.
Quotas. Ceilings per team over a window. The refusal lands in audit like any other decision.
Audits. Cost, latency and tokens per call, attributed to an agent and a person.
Scenario
The model behind your product feature
You ship an agentic feature to customers. A new model lands 30% cheaper at the same quality on your workload, and today switching to it means cutting a release.
What makes it hard
The provider is in the code
An SDK and a model name compiled into the app. Changing either ships a version.
Customers see quality, you see routing
A regression has to be caught on their traffic before it reaches them.
Cost per customer is invisible
Margin per feature needs attribution the provider invoice does not carry.
Prompts leave your boundary
Whatever the agent sends goes to a third party, and whatever comes back goes to your customer.
The work, in resources
Gateways. One endpoint for the feature. Which vendor serves it is a routing decision, not a deploy.
Pools. Shift weight toward the new model, and roll it back with the same edit.
Rules. Send a slice of traffic first. Conditions can name the tenant, the feature, or the plan tier.
Guards. Inspect prompts and completions on the way through. Sensitive fields do not reach the provider.
Quotas. Per-tenant ceilings, so one customer's usage cannot consume the feature's budget.
Audits. Cost telemetry per tenant and per feature. Margin becomes a query instead of a spreadsheet.
Configuration
Pools and routing, not glue code
Pools, priorities, failover and caching are the abstractions you already use for service-to-service routing. The agent does not change when the routing does.
pools:
- name: reasoning-tier
routing: priority-weighted
failover: up to 3
cache: short-context
members:
- provider: anthropic
model: claude-opus-5
weight: 70
- provider: openai
model: gpt-5-thinking
weight: 25
- provider: bedrock
model: claude-opus-5
weight: 5 # regional capacity
rules:
- match:
tier: standard
pool: standard-tier
- match:
tier: premium
context.tokens: ">100k"
pool: long-context-tierWhat changes
The shape of model operations with a broker in place
Before
After
Model in the agent
Each agent holds a provider SDK, an API key, and a chosen model. Changing any of the three means changing the agent. Evaluations are synthetic. Spend lives in monthly invoice PDFs.
Model behind the broker
Agents request a capability. The broker resolves a provider, applies policy, and routes. Evaluation and cost data come from one place, and provider contracts can change without touching agent code.
An outage takes you down
When the upstream provider has a bad hour, every agent that depends on it has a bad hour too. Failover is a code change made under pressure.
The pool routes around failure
Priority-weighted routing with fallback absorbs provider outages and rate-limit storms. Agents stay up when the upstream is not.
Composes with
The enterprise systems you already have
The broker plugs into the systems your team already runs. Provider credentials, telemetry, audit, and configuration all route through their existing home.
Secrets
Provider API keys live in the secrets store you already manage. Rotated centrally, and never in agent code.
Learn moreAPM
Model latency, cost per call, denial rate, and routing decisions stream to the dashboards your platform team already watches.
Learn moreSIEM
Every model call is an audit event. Provider, model, pool, agent, caller. Queryable in the SIEM your SOC already lives in.
Learn moreSource control
Pool configuration and routing rules live in a repo. Versioned, simulated against recent traffic, rolled back like any other deploy.
Learn more