Rate limits vs. quotas and budgets

Rate limiting constrains how quickly requests or units of work may be consumed. Usage quotas limit a quantity within a defined scope or period. A budget expresses a spending allowance or target, which may or may not be connected to a hard enforcement control. Concurrency limits constrain simultaneous work. Budget enforcement connects a defined financial allowance to a mechanism that restricts consumption.

Products sometimes use these names differently. Check what is counted, whose usage is counted, over what period, and what happens when the limit is reached.

Explore Dome: Model Broker · Docs: Quota scopes and behavior.

Choose a limit for the problem you're trying to solve

Suppose a research agent overwhelms a document service during bursts and also consumes too much model budget over a month. Those are two different problems.

The research agent may need both a rate limit to protect the document service and a quota to control monthly usage. An alert can notify the owner, but stopping consumption requires an enforcement action.

For Model Context Protocol (MCP) deployments, OWASP's security guidance includes resource controls among deployment protections. The appropriate control still depends on the workload.

Explore Dome: Tool Gateway · Docs: Quota scopes and behavior.

Specify how the limit works

For each limit, record what is counted, whose usage counts, when the allowance resets or replenishes, and what happens when it runs out. Check how simultaneous requests and delayed usage reports affect the count.

“1,000 calls per agent per day” is a different limit from “1,000 calls shared by the whole team.” Specify the unit and scope just as clearly for tokens, estimated provider cost, or billed platform usage.

Dome's quotas reference documents its supported subjects, cumulative scopes, windows, and distinctions between reported usage measures. Distinguish provider-cost estimates from the final provider invoice.

If you need an exact spending ceiling, check how the implementation accounts for in-flight work and delays in usage reporting.

Explore Dome: Model Broker · Docs: Quota scopes and behavior.

Decide what the application should do at the limit

When a limit is reached, the application needs to know whether to stop, wait, do less work, or request an approved exception. Retrying immediately can worsen an overload without making progress.

If a workload switches models or credentials to continue, ask whether the new path has an independently approved allowance. Otherwise, switching routes may let the workload evade the limit.

Test requests below the limit, at the limit, after it has been reached, and after it resets or an approved recovery takes effect. Include multiple active callers if the budget is shared. Record the observed behavior alongside the configuration.

See spend governance for approving exceptions, and failover for choosing alternative model destinations.

Explore Dome: Model Broker · Docs: Quota scopes and behavior.