Open tools that check claims about AI agents against evidence you pinned: an offline verifier today, with the observer, admission rules, vocabulary, conformance vectors and Rust crates joining this repository.
It's for auditors, incident reviewers and relying parties who need a decision they can replay, and for tool authors who want their records checked the same way.
Install the released verifier: uv add probity-verify==0.1.1, then pip install probity-verify==0.1.1.
Verified by Probity (github.com/probityai/probity). The probityai command line is planned, not yet released: uv add probityai, then pip install probityai.
Check a sample claim with the released package. You need Python 3.11 or newer and uv; there is nothing to clone:
samples="$(uvx --from probity-verify==0.1.1 python -c 'import importlib.resources as r; print(r.files("probity_verify") / "samples")')"
uvx probity-verify==0.1.1 "$samples/case.json" --policy "$samples/policy-june.json"It prints a decision packet that opens:
Probity decision packet v1
Case: "source-coverage-demo"
Claim: source_text_coverage/v1
Decision: contradicted
Reason: selected_span_missing_from_record
A selected passage is in the source capture but missing from the stored report. Swap in policy-april.json for not_established, or case-complete.json with policy-complete.json for supported.
| member | what it is | released |
|---|---|---|
| packages/probity-verify | offline verifier: supported, contradicted or not_established for a claim against pinned bytes |
0.1.1 on PyPI |
| packages/probityai | the probity command; probity verify today, each member adds its own subcommand |
not yet released |
uv sync --all-packages --all-extras
uv run python -m pytest| page | read it for |
|---|---|
| probity-verify | the verifier, its adapters and its recipes |
| Repository layout | the workspace, the probity command seam, CI and releases |
| Agent instructions | commands to run before a change |
| Contributing | reporting a problem and opening a pull request |
Apache-2.0, per directory: each member carries its own LICENSE.