An offline verifier that decides whether a claim holds against evidence bytes the consumer pinned, with adapters for source passages, operand lineage, authority captures, bounded absence and spend journals.
It's for auditors, incident reviewers and relying parties who receive a claim about an AI system and need a decision they can replay, not a report they have to trust.
There is no package release yet, so pin a commit. Python 3.10 or newer and uv:
git clone https://github.com/probityai/probity-verify && cd probity-verify
git checkout 588db582c50558f3e2a928be89ffc135188afcdb
uv run probity-verify examples/source-coverage/case.json \
--policy examples/source-coverage/policy-june.jsonIt prints a decision packet that opens:
Probity decision packet v1
Case: "source-coverage-demo"
Claim: source_text_coverage/v1
Decision: contradicted
Reason: selected_span_missing_from_record
Policy SHA-256: dfda1551714f92702169d1411af6b0bfeee54732f934ce71265534af775fa388
A selected passage is in the source capture but missing from the stored report. The packet goes on to list every artifact binding and check. uv run pytest runs the test suite.
Version 0.1.0, unreleased. Six adapters ship: source_text_coverage/v1, operand_lineage/v1, authority_anchor/v1, event_absence/v1, event_absence/v2 and spend_reservation/v1. Each decision is supported, contradicted or not_established. The verifier checks supplied bytes only: it does not fetch URLs or authenticate the authority a policy names.
| page | read it for |
|---|---|
| Install a pinned version | install outside the source checkout and replay three sample decisions |
| Choose a claim check | select a task from checked recipes and retrieve exact commands and input digests |
| Select a consumer adapter | supply exact adapter versions through adjudication, bundle creation and replay |
| Check a customer offer | compare a draft with raw company records and an exact manager approval |
| Running the verifier | every example command, the case and policy inputs, and what each decision means |
| Replayable decisions and the index | ZIP bundles, replay, and the local index of decisions, challenges and supersessions |
| Adapters | what operand lineage, authority anchor and event absence check, and what they leave to the consumer |
| Spend reservations | replay a pinned budget journal, refuse dispatch without an affordable reservation, and keep pending calls charged |
| Observation vantage | select the producer and observation scope, replay bounded absence, and operate a witness with separate keys and retained heads |
| Adapter plan | the architecture and the adapters planned next |
| A2A server and client | send raw case bytes to an operator-configured verifier, or read a pinned Lab record |
| Source coverage vectors and operand-lineage vectors | conformance cases with pinned bytes and expected decisions |
Apache-2.0.