NVIDIA NIM and Dome

1. 01

   Add hosted NIM

   The provider id is nvidia. Dome uses https://integrate.api.nvidia.com/v1 and NVIDIA's model ids.

   $ dome models add llama-nim \
     --provider nvidia \
     --model meta/llama-3.3-70b-instruct \
     --api-key "$NVIDIA_API_KEY" \
     --attributes '{"hosting":"nvidia-cloud"}' \
     --gateway prod-gateway

2. 02

   Add your NIM container and pool both

   A self-hosted NIM speaks the same API. Set its endpoint, and make sure the Gateway can reach it: a Dome-hosted gateway can't reach your private network.

   $ dome models add llama-nim-local \
     --provider nvidia \
     --endpoint http://nim.internal.example.com:8000/v1 \
     --model meta/llama-3.3-70b-instruct \
     --auth-method none --credential-type none \
     --attributes '{"hosting":"self"}' \
     --gateway prod-gateway
    
   $ dome models pool create llama-3.3-70b \
     --failover-max all --gateway prod-gateway
   $ dome models pool member add llama-3.3-70b llama-nim-local --priority 0
   $ dome models pool member add llama-3.3-70b llama-nim --priority 1

3. 03

   Point your agent at the Gateway

   Any OpenAI-compatible client works. The agent sends its Dome key and the pool's name.

   from openai import OpenAI
    
   client = OpenAI(base_url=f"{GATEWAY_URL}/v1", api_key=DOME_AGENT_KEY)
   client.chat.completions.create(
       model="llama-3.3-70b",
       messages=[{"role": "user", "content": "Draft a visit summary from these notes."}],
   )

Commands and rules tested against a Dome workspace on October 1, 2026. For anything about NVIDIA NIM itself, see NVIDIA's documentation.

forbid (principal, action == Dome::Action::"llm:invoke", resource is Dome::LLMModel)
when {
  resource has hosting && resource.hosting == "nvidia-cloud" &&
  principal has act_as && principal.act_as.groups.contains("clinical")
};