Agent Operations Platform
Model Broker
The Model Broker governs model traffic. It accepts OpenAI- and Anthropic-compatible calls at /v1/*, evaluates policy, routes to a configured provider or pool, and injects the provider credential at egress. The agent's call shape never changes — Dome handles the rest.
Workflow
Connect a provider.Then the next.
The same loop applies to every model upstream you bring under management. Configure once, compose into a pool if you need failover, authorise the agent, and watch the calls flow.
- 01
Configure
Add a model provider — Anthropic, OpenAI, Bedrock and more — and have credentials securely stored.
- 02
Route
Optionally group providers into a pool. Failover, cost routing, region pinning — invisible to the agent.
- 03
Authorize
Define Cedar rules at the right scope. Grant agents access to specific models, IDs, or pools.
- 04
Evaluate
Every model call evaluated against policy, credentials injected at egress, recorded with uniform llm.* audit.
Consistent operations
How every model call lands
Model traffic is where cost, data egress, and provider choice all converge. The Broker is the one place these are governed consistently — regardless of which provider is on the other end of the call.
Agent calls /v1/* with its Dome token. Wire format is OpenAI- or Anthropic-compatible.
Cedar evaluates: which agent, which model, what classification, which scope.
Direct to a provider, or through a pool with match-when predicates.
Provider credential injected at egress. Response normalized back to the agent.
Provider abstraction
OpenAI- and Anthropic-compatible wire formats at /v1/*. Anthropic, OpenAI, Bedrock, Vertex, or a self-hosted server — the agent's client code never changes.
Routing strategy
Direct provider, pool with failover, region-pinned for residency, or cost-cascaded (haiku → sonnet on retry). All invisible to the calling agent.
Spend & posture
Per-agent and per-workspace token ceilings. A uniform llm.* audit vocabulary across every provider — one query reconstructs spend, decisions, and outcomes.
Options
What the Broker lets you do
Beyond a provider connection, the Broker surfaces operational controls — what you can configure via the CLI, API, or Terraform to shape how model traffic flows.
Provider failover
Group providers into a pool with ordered priority. The Broker tries the next provider on rate-limit, outage, or capacity errors — within the same call.
Region pinning
Bind tenants or workspaces to specific provider regions. EU traffic stays on EU endpoints; US workloads route to US.
Model cascading
Configure fallback chains — try haiku, escalate to sonnet on retry. Routing is invisible; the agent always asks for the same model.
Per-agent spend caps
Token ceilings enforced per agent, workspace, or tenant. The Broker denies calls that would exceed the budget, with audit-recorded reason.
Token normalization
Provider tokens, request units, and reasoning tokens normalized into a uniform llm.* vocabulary. Spend reports work across every provider.
Request/response capture
Opt in to capture the full request and response per call, with PII filtering applied. Useful for evals, regression suites, and incident review.
In practice
From provider to governed call
Configure a provider, optionally compose a pool, authorize the agent. The agent's client code is unchanged — it just points at the Broker URL.
dome model add anthropic-prod \
--provider anthropic \
--model claude-sonnet-4-6 \
--auth-method api-key \
--credential-type shared
dome model pool create cheap-reasoning \
--match-when '{"request.model":["claude-haiku-4-5","gpt-5-mini"]}'
dome model pool add cheap-reasoning anthropic-prod
dome rules apply models/cheap-reasoning.cedarclient = anthropic.Anthropic(
base_url="https://broker.dome.example.com/v1",
api_key=os.environ["DOME_AGENT_TOKEN"],
)
response = client.messages.create(
model="claude-sonnet-4-6",
messages=[...],
)