[chanchall26]

📌 Description

Adds agentwatch/replay/shadow.py: a shadow replay engine that isolates whether an observed agent failure came from model stochasticity or from the recorded tool states. A recorded session is re-driven against a model while every tool call is answered from the recorded evidence. Nothing is executed, so replay has no side effects and needs no HTTP mock server. The shadow trace is then compared with the historical one and classified. Full description: docs/v3/SHADOW_REPLAY.md.

Fixes #751

• ToolEvidenceStore: read-only lookup built from the recorded events. Calls are matched on tool name plus canonical arguments (key order never matters), a repeated identical call gets the first and then the second recorded answer in order, parallel calls are paired by tool_id, and orphan results are counted. Everything served is a deep copy, so a replay cannot change the recording or poison a later run.
• ShadowReplayRunner / run_shadow_replay: asks a ShadowModel for each decision (temperature 0 and an optional seed in ShadowConfig), serves tool results from evidence, and stops when a call has no recorded answer, because everything after it would rest on a made-up value. A terminal event ends the replay, and a bounded number of extra decisions lets a shadow that keeps going show up as a divergence.
• Differential comparison on event signatures (event type, tool, argument hash, result hash, status, planner text hash; ids, timestamps and durations ignored), reporting the first divergence, the fields that changed, and the normalized tool-sequence distance from replay.divergence.
• Classification: IDENTICAL_REPLAY, DETERMINISTIC_CORROBORATION (exact match and the recorded run failed, so the failure reproduces from the recorded tool states), STOCHASTIC_MODEL_DIVERGENCE (the model branched although every tool result matched), and INSUFFICIENT_EVIDENCE (a call had no recorded answer, or evidence was not served faithfully).
• Reuse, per the lazy-engineering constraint: built on AgentEvent, canonical_json / sha256_hex, and replay.divergence. from_step_fn adapts an existing counterfactual StepFn, and PromptCompletionModel re-sends recorded prompts through any complete(prompt, config) callable.
• Exported from agentwatch.replay. CHANGELOG.md and docs/v3/SHADOW_REPLAY.md updated.

---

🏛️ Architectural Plane & Classification

• Plane A: Evidence Plane (Sensors, RawObservation, Ingest, Redaction)
• Plane B: Interpretation Plane (Normalizers, ComputationalEvent, Entity Resolution)
• Plane C: Structure Plane (Hyperedge Store, Execution/Information/Causal Graphs, Provenance)
• Plane D: Model Plane (Latent States, Motifs, Behavioral Genome, Drift)
• Plane E: Experiment Plane (Replay L0–L4, Timelines, Counterfactuals)
• Plane F: Interface Plane (API, Typer CLI, Next.js Frontend)
• Documentation / CI Maintenance

---

🧪 Verification & Evidence of Testing

pytest tests/replay/test_shadow_replay.py
# 42 passed

pytest tests --cov=agentwatch --cov-fail-under=70
# 2106 passed, 7 skipped; total coverage 80.36%

ruff check agentwatch tests/replay    # All checks passed
ruff format --check agentwatch/       # 286 files already formatted
bandit -r agentwatch                  # 0 issues
mypy agentwatch/replay/shadow.py      # Success: no issues found

ruff check . over the whole repo reports 33 findings in panopti/ and tests/sensors/; none are in files this PR touches.

The tests cover: each of the four classifications; a recorded failure that reproduces versus one that does not; a model that diverges on the first decision, stops early, or keeps going; missing evidence (an unrecorded tool, the same tool with different arguments, a call whose result was never captured, more calls than were recorded); argument key order; repeated identical calls served in order; parallel calls paired by tool_id; orphan results; unused evidence; no network access during replay (sockets are blocked in the test); a model that mutates its history, the recorded decision or the served result cannot change the recording or a later run; repeatability; the temperature and seed reaching the model; PromptCompletionModel and from_step_fn; signature behaviour; the divergence cap; JSON-serialisable reports; and the public exports.

• New unit tests or regression tests were added covering this logic.
• Existing test suite passes with zero regressions (pytest tests/).
• Test coverage does not decrease (satisfies >= 70% coverage gate).

---

🛡️ Security & Linter Quality Checklist

• Bandit Scan Passed: Ran bandit -r agentwatch with 0 issues found.
• No Baseline File: Confirmed bandit-baseline.json was NOT created or edited.
• Inline Suppressions (If applicable): none were added.
• Ruff Format: Ran ruff format --check agentwatch/ (Clean pass).
• Ruff Check: Ran ruff check agentwatch/ (Clean pass).
• Plane Boundaries: Verified lower planes do not import from higher planes (the module imports core, evidence.canonical and replay.divergence only).

---

✍️ Contributor Integrity Declaration

The declaration is left for the account owner to confirm after reviewing the code.

1. I have been formally assigned to this issue by a maintainer.
2. I personally understand and wrote or thoroughly verified every line of this code.
3. This PR contains no unreviewed, hallucinated, or unverified AI-generated slop.
4. My branch is cleanly rebased on the latest origin/main.

Discord Handle: @chanchalguptacse_70080

[github-actions]

🧪 PR Test Results

Check	Result
Tests (pytest tests/)	✅ success
Lint (ruff check .)	❌ failure
Coverage (agentwatch)	80.36%

_{Python 3.12 · commit ec30304}

[SHAURYASANYAL3]

NSoC '26 Maintainer Verification & Level 3 Approval 🚀

Exceptional engineering work @chanchall26!

• Architecture & Determinism: Pure Plane E shadow replay isolating model stochasticity from recorded tool states without side effects or external HTTP mocks.
• Evidence Integrity: Immutable deep copying in ToolEvidenceStore with canonical JSON and argument order invariance.
• Classification Engine: Precise categorization across IDENTICAL_REPLAY, DETERMINISTIC_CORROBORATION, STOCHASTIC_MODEL_DIVERGENCE, and INSUFFICIENT_EVIDENCE.
• Quality Gate: 100% green CI across all matrix environments (12/12 successful checks), 42 dedicated unit tests in tests/replay/test_shadow_replay.py, and comprehensive documentation in docs/v3/SHADOW_REPLAY.md.
• Accreditation: Scored for NSoC '26 (Level 3, soc26).

Merging into main! 🛡️