Gemma and Dome
Run Gemma 4 from two providers under one name. Gemma is Google's open-weight model, small enough to be the sensible default for people who don't need a frontier one. Dome serves it from DeepInfra with NVIDIA NIM behind, keeps contractors on it, and counts calls per day.
Dome provides model brokeringand routing for Gemma.
DeepInfra first, NVIDIA next
DeepInfra serves google/gemma-4-31B-it and NVIDIA NIM serves google/gemma-4-31b-it. The pool gives them one name.
Contractors use Gemma
Tag Gemma connections with a family attribute. When the person behind a call is a contractor, a rule refuses every other model.
Calls per day
The pool allows 50,000 calls a day. The count is calls, not tokens: each request counts once, however long.
Gemma with failoverin five steps.
DeepInfra and NVIDIA in one pool, a contractor rule, then a daily call cap.
Connect DeepInfra
DeepInfra's id is google/gemma-4-31B-it. Dome keeps the DeepInfra key out of every agent's hands.
shelldome models add gemma-deepinfra \ --provider deepinfra \ --model google/gemma-4-31B-it \ --api-key "$DEEPINFRA_API_KEY" \ --attributes '{"family":"gemma"}' \ --gateway prod-gatewayConnect NVIDIA NIM
The provider id is nvidia. NVIDIA writes the model id in lower case.
shelldome models add gemma-nvidia \ --provider nvidia \ --model google/gemma-4-31b-it \ --api-key "$NVIDIA_API_KEY" \ --attributes '{"family":"gemma"}' \ --gateway prod-gatewayPool them for failover
The two ids differ only in case, and the pool hides both behind gemma-4-31b.
shelldome models pool create gemma-4-31b \ --failover-max all --gateway prod-gateway dome models pool member add gemma-4-31b gemma-deepinfra --priority 0dome models pool member add gemma-4-31b gemma-nvidia --priority 1Point your agent at the Gateway
Contractor-facing agents use the OpenAI SDK pointed at the Gateway, with their own Dome key.
pythonfrom openai import OpenAI client = OpenAI(base_url=f"{GATEWAY_URL}/v1", api_key=DOME_AGENT_KEY)client.chat.completions.create( model="gemma-4-31b", messages=[{"role": "user", "content": "Find the VPN setup steps in the knowledge base."}],)Cap daily calls
One call counts once, on whichever host served it.
shelldome quotas set --subject pool --pool gemma-4-31b \ --unit calls --limit 50000 --window daily
Contractorsuse Gemma.
When the person behind a call is in contractors, any connection not tagged Gemma is refused. Employees using the same agent are untouched.
cedar
forbid (principal, action == Dome::Action::"llm:invoke", resource is Dome::LLMModel)when { principal has act_as && principal.act_as.groups.contains("contractors") }unless { resource has family && resource.family == "gemma" };One call,two outcomes.
Switch the caller or the argument and watch the same call decide differently, with every decision landing in audit.
Model
agent helpdesk-agent · acting as j.smith
llm:invoke(model: "gemma-4-31b")
- Agenthelpdesk-agent is registered and active
- Callerj.smith verified, groups: contractors
- RuleConnection is tagged family: gemma
- Quota18,400 of 50,000 calls today
DecisionAllowed
Bringing it alltogether.
Connecting Gemma to registered agents, tools, and identity in Dome completes a governed agent application.
Acting for
Control point
GatewayEvery call decided and audited
RulesGuardsQuotas
Models
Common questionsabout Gemma.
More of whatDome works with.
Provider
Anthropic
The Claude API behind the Model Broker: the key held in Dome, every call authorized, metered and audited.
MCP server
GitHub
The GitHub MCP server behind the Tool Gateway, with rules that decide each call on its owner and repository.
Identity
Okta
Okta tokens verified on every agent call, so rules and audit name the person each agent acted for.
Runtime
LangGraph
LangGraph agents with their model calls on the Model Broker and their MCP tools on the Tool Gateway.
Client
Claude Code
Claude Code on a Dome Gateway with per-developer sign-in, rules on every tool call, and audit by name.
Agent service
TinyFish
TinyFish's web agents behind the Tool Gateway, with rules that decide each run on the site it targets.
Talk with ourFDE team.
Or start free, and register your first agent in minutes. Connect it through a gateway, give it rules, and see every call it makes in one audit trail.
Talk with our FDE team
Our engineers design, build and run governed agents for the part of your business you want to change. You see what they deliver.
AI models for enterprise agents: failover, rules and quotas
Every model your agents call goes through the Model Broker. Pool providers for failover, decide who may call each model, and cap what it costs.