Policy tests
Executable specifications for the access model — selectors, scenarios, assertions, and the runs that keep policies honest.
Policy tests are the access model's safety net. Each one is a first-class artifact that turns a security requirement into an executable specification: given this user and this world, access must be granted — or must stay denied. Tests run independently of the application, and they are what lets humans and coding agents change policies without guessing what else moved. This page closes the Concepts group: tests exercise both action policies and data policies.
Anatomy
A test answers three questions: what is under test, in what world, and what must happen. An action test points at its subject through a selector over actions, and a dataset test names one dataset by its code. The scenario says who the user is, what the request carries, and what the data looks like. The assertion says what must happen. One of the seeded tests, piece by piece:
| Piece | In Manager cannot issue an invoice |
|---|---|
| Selector | ACTION_SET with a single code, invoices:issue |
| Scenario | user attributes {"role": "manager"} and nothing else |
| Assertion | expectedAccess: "DENIED" |
A negative test — and in a permit-only model these carry real weight: adding a policy can only widen access, and negative tests are what pin the boundaries that must stay closed.
Selectors come in three types: ACTION_SET names specific action codes, and ALL_ACTIONS sweeps every action in the workspace. A formula-based selector exists in the API but does not filter yet — until it ships, it selects every action, so prefer the other two.
The scenario is the whole world
Nothing outside the scenario influences the outcome. It carries user attributes, context attributes, an optional request.* payload, and — for dataset-backed rules — dataset fixtures. Attributes are fed to evaluation exactly as a live check would receive them. An attribute the scenario does not provide evaluates as unknown, and unknown never grants — a test cannot pass by accident of a missing input.
Assertions
expectedAccess is GRANTED or DENIED. An optional provenance check, mustBeGrantedByPolicyIds, tightens a GRANTED assertion further: the test passes only if every listed policy was among those that granted. That distinguishes "access was granted" from "access was granted for the right reason" — useful when several policies overlap.
Dataset tests
Data policies filter rows rather than granting actions, so their tests assert row sets. A dataset test supplies the table's entire content as a fixture and expects exactly these primary keys to be visible (READ) or writable (WRITE):
| Piece | In Manager sees only their region's invoices |
|---|---|
| Dataset | demo_billing.public.invoice |
| Scenario | a manager from region eu, with a two-row fixture — one invoice in eu, one in us |
| Assertion | expectedVisiblePks holds exactly the eu invoice |
Two rows in, one expected back. DATASET_WRITE is identical with expectedWritablePks. The fixture is the whole table — an empty fixture with an empty expectation is the idiom for "this user sees nothing", including the deny-by-default case. There are no request attributes on the data side, because data policies cannot reference request.*.
Dataset tests run through the production pipeline: the same rendering that hands your application its SQL produces the condition here, applies it to the fixture rows, and compares the resulting key set by equality. The result records that rendered condition, so a failing test shows the exact SQL the SDK would have received.
Lifecycle
| Status | Meaning | Runs in the suite |
|---|---|---|
| GENERATED | created by automation, awaiting review | no |
| DRAFT | still being configured | no |
| ENABLED | active | yes |
| DISABLED | kept, but excluded | no |
Only ENABLED tests participate. GENERATED is the review gate for machine-authored tests: a coding agent can file its tests there, and approving one simply means a human moves it to ENABLED. The lifecycle encodes the same division of labor as the rest of the model — agents author, humans approve.
Runs and traces
arkveil tests run <testId> executes one test, arkveil tests run-all executes every ENABLED test — each produces its own run, and one failure does not stop the rest. Runs are historical: they record what the policies looked like at that moment, and arkveil tests history walks them back.
A run holds one result per evaluated action (or the tested dataset), and every result carries a full evaluation trace: the candidate policies, how each target matched, and how every sub-expression of every condition evaluated, attribute values included. Dataset results add a per-policy filter breakdown alongside the combined rendered condition. The trace is the same machinery as arkveil eval explain --json, recorded at run time — when a test fails, the answer to "why" is already attached. Arkveil Studio shows the same traces visually.
Next steps
This page completes the concepts. From here:
- Getting started — run the seeded suite and protect an endpoint.
- Coding agents — tests as part of an agent's authoring loop.
- CLI —
tests create,run,run-all,history, and friends.