Guide
Test agent authorization beyond the requests in your logs
A policy replay reports no changed decisions. The team is ready to deploy a new production restriction. But the replay contains no production requests.
The replay may be correct, but it hasn't tested the change the team cares about. To know whether production access is denied, you need to try a production request.
Write expected decisions before running the test
Use a hypothetical deployment agent with a simple requirement: it may deploy to staging, but production deployment requires additional authority. The authorization foundation supplies the shared decision model. Here, we'll turn that model into a test plan.
| Synthetic case | Expected decision | Why it earns a row |
|---|---|---|
| Approved agent, staging tool, required context present | Allow | Positive control |
| Same identity, production tool, no production authority | Deny | The new boundary |
| Same request, required represented user absent | Deny for the delegated workflow | Missing-identity path |
| Same request, unexpected environment value | Deny | Unrecognized input must not broaden access |
| Workspace permit conflicts with a higher-scope forbid | Deny | Effective policy differs from one edited file |
| Previously refused unrelated tool | Remains denied | Guard against accidental widening |
These are intended outcomes, not measured results. Record actual decisions and the policy identity when executing them.
Know which rules the tool evaluates
Dome supports different simulation paths. A synthetic evaluation can test the active effective rules without performing the action. The rules editor can compare deployed and unsaved draft content, with the draft replacing the workspace or agent scope being edited for that evaluation.
An ordinary synthetic command does not automatically load a nearby candidate file:
# Illustrative: configure these example names first.
# This evaluates the active effective rules, not an unsaved candidate file.
dome rules simulate \
--agent deployment-agent \
--action mcp:call \
--resource staging-deploy \
--resource-type mcp_toolFor an unsaved candidate, use the documented Draft path and verify the selected scope. Retain the candidate files and identify the unchanged scopes that still contribute to the effective decision.
Check what the evaluator actually receives
If the policy reads arguments, confirm how the evaluator receives them. Dome exposes supported top-level arguments through resource.arguments; the documented conversion drops values it cannot represent, including fractional floats, nulls, and mixed-type lists.
For example, don't write a spending rule around a decimal argument until you've checked how that value reaches the evaluator. The JSON you send and the attribute the rule sees may differ.
Likewise, simulation accepts act-as claims as test inputs. That tests claim-aware policy, not token validation. Exercise the live delegated identity path separately with synthetic users in a controlled environment.
Read historical differences in both directions
Replay the candidate against an identified historical window. Keep the window, event count, filters, candidate, and effective-policy context with the result.
For each allow-to-deny change, ask whether the release intentionally stops that operation or breaks legitimate work. For each deny-to-allow change, identify the newly granted authority and its approver. Both directions need review.
Zero differences means no observed outcome changed in the replayed set. It does not validate absent request classes. If there were no production requests, the production rows in the synthetic matrix remain essential.
Do not turn a historical frequency into an authorization requirement. A rare operation can be necessary; a frequent one can be over-permitted.
Make three checks before release
Before releasing the change, check that:
- Each synthetic case returns the expected result, including requests that should be refused.
- Someone has reviewed every changed historical decision and explained whether it is intended.
- Live integration tests cover what simulation can't: identity verification, routing, and the relevant backend behavior.
Keep a failure at any gate visible. A policy simulation does not execute the remote business action, so it cannot prove that a deployment commits correctly or that a rejected request left no unintended backend effect.
Keep those results together with the candidate version and any untested cases. The next reviewer should be able to see why you're confident about production access—even when production requests are missing from history. The simulation guide covers the tools for running these checks.