Llama and Dome

1. 01

   Connect Groq

   Groq names it llama-3.3-70b-versatile. Its key stays in Dome; support agents hold Dome keys only.

   $ dome models add llama-groq \
     --provider groq \
     --model llama-3.3-70b-versatile \
     --api-key "$GROQ_API_KEY" \
     --attributes '{"family":"llama"}' \
     --gateway prod-gateway

2. 02

   Connect Together AI and SambaNova

   Same model, each provider's own id. Tag them the same way.

   $ dome models add llama-together \
     --provider together \
     --model meta-llama/Llama-3.3-70B-Instruct-Turbo \
     --api-key "$TOGETHER_API_KEY" \
     --attributes '{"family":"llama"}' \
     --gateway prod-gateway
    
   $ dome models add llama-sambanova \
     --provider sambanova \
     --model Meta-Llama-3.3-70B-Instruct \
     --api-key "$SAMBANOVA_API_KEY" \
     --attributes '{"family":"llama"}' \
     --gateway prod-gateway

3. 03

   Pool them for failover

   Groq leads, Together and SambaNova follow. The provider ids stay inside the pool.

   $ dome models pool create llama-3.3-70b \
     --failover-max all --gateway prod-gateway
    
   $ dome models pool member add llama-3.3-70b llama-groq --priority 0
   $ dome models pool member add llama-3.3-70b llama-together --priority 1
   $ dome models pool member add llama-3.3-70b llama-sambanova --priority 2

4. 04

   Point your agent at the Gateway

   All three speak the OpenAI API, so the support agent's client just changes its base URL and key.

   from openai import OpenAI
    
   client = OpenAI(base_url=f"{GATEWAY_URL}/v1", api_key=DOME_AGENT_KEY)
   client.chat.completions.create(
       model="llama-3.3-70b",
       messages=[{"role": "user", "content": "Classify this ticket: billing, bug or how-to."}],
   )

5. 05

   Cap the tokens

   Tokens are counted at the pool, not per provider.

   $ dome quotas set --subject pool --pool llama-3.3-70b \
     --unit tokens --limit 20000000 --window daily

Commands and rules tested against a Dome workspace on October 1, 2026. For anything about Llama itself, see Meta's documentation.

forbid (principal, action == Dome::Action::"llm:invoke", resource is Dome::LLMModel)
when { resource has family && resource.family == "llama" }
unless { principal has act_as && principal.act_as.groups.contains("support") };