NVIDIA NIM and Dome
NVIDIA NIM on your GPUs and NVIDIA's, behind one Gateway. Connect hosted NIM and your own NIM containers to Dome. Pool them under one name, with your GPUs first and NVIDIA's API behind them. Rules decide which callers may leave your infrastructure.
Dome provides model brokeringand routing for NVIDIA NIM.
Your GPUs first
Pool a self-hosted NIM with hosted NIM under one name. Calls go to your container, then to NVIDIA's API.
One model name in both places
NIM containers serve the same model ids as NVIDIA's API. Agents send the pool's name and never change.
Sensitive callers stay home
A rule reads the verified user's groups, and clinical users never reach the hosted API.
Connect NVIDIA NIMin three steps.
Add hosted NIM and your own container as connections, pool them, then point agents at the Gateway.
Add hosted NIM
The provider id is nvidia. Dome uses https://integrate.api.nvidia.com/v1 and NVIDIA's model ids.
shelldome models add llama-nim \ --provider nvidia \ --model meta/llama-3.3-70b-instruct \ --api-key "$NVIDIA_API_KEY" \ --attributes '{"hosting":"nvidia-cloud"}' \ --gateway prod-gatewayAdd your NIM container and pool both
A self-hosted NIM speaks the same API. Set its endpoint, and make sure the Gateway can reach it: a Dome-hosted gateway can't reach your private network.
shelldome models add llama-nim-local \ --provider nvidia \ --endpoint http://nim.internal.example.com:8000/v1 \ --model meta/llama-3.3-70b-instruct \ --auth-method none --credential-type none \ --attributes '{"hosting":"self"}' \ --gateway prod-gateway dome models pool create llama-3.3-70b \ --failover-max all --gateway prod-gatewaydome models pool member add llama-3.3-70b llama-nim-local --priority 0dome models pool member add llama-3.3-70b llama-nim --priority 1Point your agent at the Gateway
Any OpenAI-compatible client works. The agent sends its Dome key and the pool's name.
pythonfrom openai import OpenAI client = OpenAI(base_url=f"{GATEWAY_URL}/v1", api_key=DOME_AGENT_KEY)client.chat.completions.create( model="llama-3.3-70b", messages=[{"role": "user", "content": "Draft a visit summary from these notes."}],)
Clinical usersstay on your GPUs.
Applied to an agent, this refuses hosted NIM whenever the verified user is in the clinical group. Everyone else can still fail over to NVIDIA's API.
cedar
forbid (principal, action == Dome::Action::"llm:invoke", resource is Dome::LLMModel)when { resource has hosting && resource.hosting == "nvidia-cloud" && principal has act_as && principal.act_as.groups.contains("clinical")};Bringing it alltogether.
Connecting NVIDIA NIM to registered agents, tools, and identity in Dome completes a governed agent application.
Acting for
Control point
GatewayEvery call decided and audited
RulesGuardsQuotas
Models
Common questionsabout NVIDIA NIM.
More of whatDome works with.
Model
Claude Fable
Fable 5.1 from Anthropic and Amazon Bedrock in one failover pool, open to one group and capped by quota.
MCP server
GitHub
The GitHub MCP server behind the Tool Gateway, with rules that decide each call on its owner and repository.
Identity
Okta
Okta tokens verified on every agent call, so rules and audit name the person each agent acted for.
Runtime
LangGraph
LangGraph agents with their model calls on the Model Broker and their MCP tools on the Tool Gateway.
Client
Claude Code
Claude Code on a Dome Gateway with per-developer sign-in, rules on every tool call, and audit by name.
Agent service
TinyFish
TinyFish's web agents behind the Tool Gateway, with rules that decide each run on the site it targets.
Talk with ourFDE team.
Or start free, and register your first agent in minutes. Connect it through a gateway, give it rules, and see every call it makes in one audit trail.
Talk with our FDE team
Our engineers design, build and run governed agents for the part of your business you want to change. You see what they deliver.
LLM providers for AI agents: one governed path to every model
Connect a provider once. Its key stays in Dome, and every agent call to it is authorized, metered and audited.