feat(verification): Add transparent audits - #739
Closed
kotevcode wants to merge 22 commits into
Closed
Conversation
…nting Add an on-chain verifier network that detects model substitution. Whitelisted verifiers probe sellers advertising the same model, compare response distributions across the cohort, and attest SAME/DIFF verdicts on-chain backed by ResponseAuth-signed evidence. Contracts (packages/contracts/verification/): - AntseedVerifierRegistry: owner-approved verifier whitelist, probe-set commit-reveal, per-audit attestations, per-(agent,service) and per-agent verification stats, cooldown-gated per-epoch audit credits. - AntseedVerifierRewards: emissions-gate bucket controller for VERIFICATION_MINTER_ID; verifiers claim each finalized epoch's bucket pro rata to credited audits. - AntseedVerifierPointsPolicy: swappable IAntseedPointsPolicy that discounts a DIFF-flagged seller's recognized-usage points (never reverts). @antseed/fingerprints (new package): KBF verifier math (Clopper-Pearson + binomial verdicts), cross-seller cohort consensus, deterministic probe bank, canonical evidence hashing, and a stealth request engine that makes probes read like organic chat with free-text answer extraction. @antseed/node: verifier contract clients, per-(peer,model) verification reputation scoring surfaced on PeerInfo during discovery, getResponseAuth accessor, chain-config + payments wiring. @antseed/cli: antseed verifier start|status|claim daemon and a verifier config section. Stacked on the recognized-usage emissions stack.
Add a ProbeSource abstraction with a large compositional entity/attribute generator as the default probe origin, demoting the static bank to a test fixture. Deterministic randomness now uses RFC 5869 HKDF-SHA256 seed derivation, an SP 800-90A HMAC_DRBG stream, rejection sampling, and Fisher-Yates; canonical JSON hashing is documented against RFC 8785 (JCS). Stealth framings gain task-shaped variants to widen the request silhouette.
The daemon now auto-discovers every advertised service from one wildcard buyer discovery per round when verifier.services is empty, rotates revealed probes via a per-service log, and tracks the epoch audit budget across services. New `antseed verifier reference build` enrolls certified KBF references through a trusted OpenAI-compatible upstream (consistency filtering across temperatures + hold-out self-test, per arXiv:2605.29524); with verifier.upstream configured the daemon enrolls automatically and audits referenced services down to a single seller. Spec 07 gains research grounding and standards citations.
A single approved verifier's DIFF attestation permanently zeroed a seller's usage emissions: diffCount is monotonic and the policy gated on diffCount > 0 with no clearing path, while the buyer-side routing score already required two distinct verifiers. The registry now tracks activeDiffVerifierCount — distinct verifiers whose latest verdict is DIFF, retractable by the same verifier re-attesting SAME on the same service, with agent-level standing held while ANY service carries the DIFF so an honestly served second model cannot launder a substituted one. The points policy gates on an owner-configurable minDistinctDiffVerifiers (default 2).
Verifiers announce a probe-delegation capability on the DHT and dispatch verifier-crafted probe jobs to opt-in buyers, who relay them verbatim over their paid buyer path so probe traffic is indistinguishable from organic usage. The verifier re-verifies each seller-signed ResponseAuth itself, and carriers earn a delegateShareBps cut of the verification emissions bucket via creditDelegates/claimDelegateReward at their operator address.
The delegate payout address is now always the operator registered for the buyer in AntseedDeposits: the buyer daemon resolves it on-chain (config payoutAddress is only a cross-check), and the verifier verifies the binding before accepting a delegate registration, which also filters sybils without a funded, operator-bound deposit account.
Move delegation muxes, the delegate roster, hello/operator-binding validation, and job dispatch/serving out of node.ts into verification/delegation-manager.ts. AntseedNode keeps only connection lifecycle wiring and thin API forwarders.
Replace verifier-submitted creditDelegates with EIP-712 DelegateVouchers claimed by the buyer's deposits operator. The contract now enforces the operator binding at claim time and caps voucher credits per probe commitment at the probeCount of credited attestations, anchoring grants to real commit-reveal audit work.
Contracts: points-policy survives permissive-fallback staking targets (raw staticcall + returndata checks), rewards freeze epoch credit totals alongside budget/split, deploy script asserts registry wiring and warns on controller-flip stranding, submitAttestation documents the whitelist-trust model, and de-whitelist/controller-flip behaviors are pinned by tests. Fingerprints: probe commitments bind full probe content (not just ids), nonce derivation folds a hash of the exclude set (no reuse across rotations, no hkdf info-size limit), the free-text extractor handles decimals, list ordinals, preamble numerals, and predicate cues, cohort consensus dedupes seller identities and scores support against the emitted median, match vectors are validated at the API boundary, and ambiguous dataset entries are replaced. Node: routing excludes sellers with an active (unretracted) DIFF flag and scores honor activeDiffVerifierCount, findPeer and background discovery get verification enrichment, DelegationManager fails closed until an accepted welcome and enforces delegate/concurrency caps, the delegation codec validates sizes and voucher shapes on both sides, and delegation hosts announce their capability DHT topic. CLI: hard per-epoch attempt caps with per-seller backoff, minProbeCount enforced before spend, rotation-pool recycling, round-level RPC retry with backoff, credited-vs-submitted attestation accounting, strict config/flag validation, --api-key precedence, reference hold-out quality gate, per-seller failure isolation in delegated probing, offline re-verifiable evidence bundles, safe service slugs, official bootstrap merging, delegate worker revocation/abort/stop/schema enforcement, and durable voucher persistence.
Contracts: voucher replay guard keyed by recovered signer (two verifiers signing identical fields no longer collide), owner-only clearVerifierStanding to retract a rogue de-whitelisted verifier's standing DIFF, and corrected late-credit natspec that overstated the frozen-claim-set guarantee. Fingerprints: nonce derivation binds service/count/source/domains so sibling artifacts from one seed keep independent blinding, positional extraction prefers the last window and suppresses question echoes, same-entity probes never share a request chunk, canonical JSON rejects sparse arrays, evidence schema declares exchanges/fullyAuthenticated, and ambiguous dataset entries are replaced with verified facts. Node: delegate voucher signatures normalize to canonical 65-byte form (compact EIP-2098 signatures passed every off-chain check but reverted on-chain), a dropped connection during delegate registration no longer crashes the process with an unhandled rejection, welcome acceptance flips synchronously so a coalesced voucher is not dropped, and the voucherClaimed client mirrors the new two-arg ABI. CLI: reference enrollment sends the advertised model spelling upstream, new `antseed delegate vouchers|claim` commands let operators actually claim voucher credits before they expire, reward claiming is fault isolated per epoch across status/start/claim, the delegate job schema allowlists body fields (n, best_of, max_completion_tokens can no longer multiply paid output), and stop() is re-checked after the approval await. Buyer proxy: pinned peers carrying an active substitution flag are refused (/v1/models stays exempt) and verification stats persist across restarts. Spec 07 documents the delegation protocol, the delegate reward split, the stealth request construction, and the evidence hash.
Routing: hasModelSubstitutionFlag now keys solely on the on-chain activeDiffVerifierCount, so owner clearVerifierStanding lifts the buyer-side routing block (DefaultRouter and the CLI 502 gate) in lockstep with the on-chain points penalty — the lastVerdict fallback was unreachable via RPC and kept a cleared seller blocked forever. CLI: persisted modelVerification carries a freshness stamp and expires after 30 minutes, so a stale DIFF flag can no longer block a retracted-accuser peer indefinitely while enrichment is degraded; delegated probes now relay a sized max_tokens bound and the delegate worker rejects budget-less jobs; the /v1/models substitution-gate exemption is scoped to GET on the exact path. Fingerprints: element_bp drops carbon and arsenic (which sublime at 1 atm and have no well-defined boiling point), matching the round-2 melting- point fix, replaced with neodymium and gadolinium. Docs: spec 07 corrects the delegate maxConcurrentJobs default (2) and the KBF package file listing.
Contributor
Author
|
Claude finished @kotevcode's task in 6m 33s —— View job Review completeThe trigger comment was just Todo list
Overall this is a well-engineered, thoroughly-tested PR — signature verification, EIP-712 handling, reentrancy guards, reward accounting, and routing/gating logic are all careful and adversarially tested. A few concrete gaps surfaced: Medium
Low / nit
Confirmed solid (no issues found)
No changes were made — this was review-only as requested. Let me know if you'd like me to implement any of the fixes above. |
Exempt legitimately un-attested sellers from audit-verify MISMATCH, resolve anchor verifiers from the event topic, chunk delegate credit discovery, age out stale substitution flags in SDK routing, cap the router's body parse, debounce hot-path peer persistence, and fix length-changing lowercase offsets in stealth answer extraction.
Key delegate accrual and budgets by the audited (agentId, serviceHash) so a credited attestation only backs its own carriers, and add a global anchored-exchange registry so a seller-signed exchange anchors at most once network-wide. Updates the TS client, CLI claim flow, spec, and the audit e2e (also rewired to the renamed fetchMatchingAttestations).
Dedupe agent-owner/stats/claim scaffolding in the verifier contracts and drop derivable anchor storage; consolidate the substitution-flag gate, service extraction, and audit-verify chain fetchers into @antseed/node; parallelize delegate scans and carrier probing; remove dead fields, wrappers, and exports across node and cli.
kotevcode
force-pushed
the
refactor/contracts-domain-architecture
branch
2 times, most recently
from
July 26, 2026 09:09
ad08d75 to
feef079
Compare
Collaborator
|
Superseded by #969, which implements the revised KBF verification architecture on top of the corrected |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Adds a transparent, publicly recomputable model-audit pipeline across the verifier contracts, Node SDK, fingerprints package, and CLI. Verifiers commit probe sets before sending traffic, anchor seller-signed exchanges before reveal, bind attestations to the anchored batch root, publish evidence packs, and expose
antseed audit verifyfor independent chain-and-pack verification.This replaces the voucher-based carrier-credit design with chain-discovered accruals and adds production limits, strict configuration validation, hardened opt-in remote pack fetching, and a real Anvil lifecycle test. It supersedes the approach proposed in #720; #720 intentionally remains open for comparison.
Release Notes
antseed audit verifyto independently reconstruct and verify audits from public chain data and optional evidence packs.Types of Changes
Checklist
CHANGELOG.mdincludes the user-facing changesValidation
pnpm run typecheckpnpm run buildpnpm --filter @antseed/node run test— 824 testspnpm --recursive --filter '!@antseed/node' run test— all remaining workspace suites, including 362 CLI tests and 65 existing E2E testsforge test— 832 testspnpm --filter @antseed/e2e run flow:transparent-audit— 3 signed exchanges, 3 attestations, 9 delegate credits, independent verification passedgit diff --check