Llama and Dome
Run Llama from three providers under one name. Six of Dome's providers serve Llama, so no one vendor has to hold your support queue. Dome pools Groq, Together AI and SambaNova under llama-3.3-70b, keeps the model to the support team, and puts a daily ceiling on tokens.
Dome provides model brokeringand routing for Llama.
No single vendor
Groq, Together AI and SambaNova each serve Llama 3.3 70B under their own id. The pool gives it one name, and a failed call moves to the next provider.
Llama for the support team
Tag Llama connections with a family attribute. A rule refuses them unless the person the agent acts for is in support.
A daily token ceiling
The pool allows 20M tokens a day, summed across Groq, Together, and SambaNova, and the count resets daily.
Llama with failoverin five steps.
Three providers, three model ids, one pool name, then the support rule and a token cap.
Connect Groq
Groq names it llama-3.3-70b-versatile. Its key stays in Dome; support agents hold Dome keys only.
shelldome models add llama-groq \ --provider groq \ --model llama-3.3-70b-versatile \ --api-key "$GROQ_API_KEY" \ --attributes '{"family":"llama"}' \ --gateway prod-gatewayConnect Together AI and SambaNova
Each provider serves the same model under its own id, so tag them the same way.
shelldome models add llama-together \ --provider together \ --model meta-llama/Llama-3.3-70B-Instruct-Turbo \ --api-key "$TOGETHER_API_KEY" \ --attributes '{"family":"llama"}' \ --gateway prod-gateway dome models add llama-sambanova \ --provider sambanova \ --model Meta-Llama-3.3-70B-Instruct \ --api-key "$SAMBANOVA_API_KEY" \ --attributes '{"family":"llama"}' \ --gateway prod-gatewayPool them for failover
Groq leads and Together and SambaNova follow, while the provider ids stay inside the pool.
shelldome models pool create llama-3.3-70b \ --failover-max all --gateway prod-gateway dome models pool member add llama-3.3-70b llama-groq --priority 0dome models pool member add llama-3.3-70b llama-together --priority 1dome models pool member add llama-3.3-70b llama-sambanova --priority 2Point your agent at the Gateway
All three speak the OpenAI API, so the support agent's client just changes its base URL and key.
pythonfrom openai import OpenAI client = OpenAI(base_url=f"{GATEWAY_URL}/v1", api_key=DOME_AGENT_KEY)client.chat.completions.create( model="llama-3.3-70b", messages=[{"role": "user", "content": "Classify this ticket: billing, bug or how-to."}],)Cap the tokens
Tokens are counted at the pool, not per provider.
shelldome quotas set --subject pool --pool llama-3.3-70b \ --unit tokens --limit 20000000 --window daily
Llamafor the support team.
Put it on an agent with --agent and Llama answers only when the person behind the call is in support. Everything else that agent uses is left alone.
cedar
forbid (principal, action == Dome::Action::"llm:invoke", resource is Dome::LLMModel)when { resource has family && resource.family == "llama" }unless { principal has act_as && principal.act_as.groups.contains("support") };One call,two outcomes.
Switch the caller or the argument and watch the same call decide differently, with every decision landing in audit.
Acting for
agent triage-agent · acting as r.alvarez
llm:invoke(model: "llama-3.3-70b")
- Agenttriage-agent is registered and active
- Callerr.alvarez verified, groups: support
- RuleLlama is open to the support group
- Quota6.2M of 20M tokens today
DecisionAllowed
Bringing it alltogether.
Connecting Llama to registered agents, tools, and identity in Dome completes a governed agent application.
Acting for
Control point
GatewayEvery call decided and audited
RulesGuardsQuotas
Models
Common questionsabout Llama.
More of whatDome works with.
Provider
Anthropic
The Claude API behind the Model Broker: the key held in Dome, every call authorized, metered and audited.
MCP server
GitHub
The GitHub MCP server behind the Tool Gateway, with rules that decide each call on its owner and repository.
Identity
Microsoft Entra ID
Entra access tokens verified on every agent call, so rules read app roles and audit names the person.
Runtime
LangGraph
LangGraph agents with their model calls on the Model Broker and their MCP tools on the Tool Gateway.
Client
Claude Code
Claude Code on a Dome Gateway with per-developer sign-in, rules on every tool call, and audit by name.
Agent service
TinyFish
TinyFish's web agents behind the Tool Gateway, with rules that decide each run on the site it targets.
Talk with ourFDE team.
Or start free, and register your first agent in minutes. Connect it through a gateway, give it rules, and see every call it makes in one audit trail.
Talk with our FDE team
Our engineers design, build and run governed agents for the part of your business you want to change. You see what they deliver.
AI models for enterprise agents: failover, rules and quotas
Every model your agents call goes through the Model Broker. Pool providers for failover, decide who may call each model, and cap what it costs.