Repository navigation
feat(sensors): harden LangChain and LangGraph callback handler (#759) - #787
Conversation
…aya-Tech-Labs#759) Never raise into the host chain, release per-run state on terminal callbacks, add duration and token usage, and add require_langchain() with an install hint. Signed-off-by: chanchall26 <190047591+chanchall26@users.noreply.github.com>
🧪 PR Test Results
Python 3.12 · commit 9f5cca9 |
SHAURYASANYAL3
left a comment
There was a problem hiding this comment.
NSoC '26 Maintainer Verification & Approval 🚀
- Architecture & Plane Isolation: Confirmed clean Plane A (Sensors) isolation with defensive exception boundaries and memory leak prevention via monotonic per-run state eviction.
- Concurrency & Resilience: Lock guarding across asynchronous/concurrent LangGraph branches verified.
- CI / Quality Gate: 100% green pass across all matrix builds, security scanners, linters, and test suites.
- Accreditation: Scored for NSoC '26 (Level 2).
Merging into main! Excellent work @chanchall26.
|
Hi! Thank you for merging this contribution. If this PR is eligible for NSoC 2026, could you please add the exact |
|
Hi @chanchall26! 👋 Thank you for checking! We have created the official Your contributions are now fully labeled and indexed for NSoC 2026 scoring! 🚀 |
Fixes #759
Description
agentwatch/sensors/langchain/already contained a callback handler (AgentWatchLangChainSensor) that emits raw callback payloads withrun_id,parent_run_idandroot_run_id. Theagentwatch/sensors/langchain.pypath in the guidance cannot coexist with that package (the package would shadow it), so I hardened the existing handler in place instead of adding a second one. The tests are attests/sensors/test_langchain_sensor.pyas requested.Changes:
on_*callback is wrapped so nothing raised while building or emitting an observation can reach the user's chain. Failures are counted inctx.errors.root_run_idmap plus a monotonic start time) is released on each terminal callback (*_end,*_error), so a long-lived sensor no longer grows without bound.agent_actionandagent_finishshare the chain's run id and do not touch that state.duration_ms, measured per run with a monotonic clock.llm_endcarries a normalizedtoken_usage(fromllm_outputor messageusage_metadata). The LangChain normalizer falls back to it (one line).LANGCHAIN_AVAILABLEandrequire_langchain()give a clear install hint (pip install "agentwatch-ai[langchain]"). Importing the module and constructing the sensor still need no LangChain.Plane: A (Evidence, sensors). The normalizer change is a one-line additive fallback.
Testing
Run locally on Python 3.12.15 (Windows),
langchain-corenot installed, no live LangChain or model calls (duck-typed fakes):pytest tests/sensors tests/v3: 366 passed, 5 skipped. The new file has 22 test cases: llm, chat model, tool, chain and retriever success; error paths; LangGraph nesting and root ids; agent action and finish; token usage; failing sink; malformed events; 8 concurrent threads; state cleanup; missinglangchain_core(run in a subprocess).agentwatch.sensors.langchainfrom those tests: 98%.pytest tests(the whole suite; I did not use the CI flags-v,--covor--cov-fail-under): 1919 passed, 17 skipped, 4 deselected. I deselectedtests/test_cli_server.pybecause I did not want to start servers.ruff check agentwatch/: clean.ruff format --check agentwatch/: clean.bandit -r agentwatch: 0 issues.mypyon the two changed source files only: clean.Not verified / not done
make typecheck(full),pip-auditand the frontend jobs were not run.langchain-coreor LangGraph install.--cov-fail-under=70figure.agentwatch/sensors/langchain.pymodule was created (see Description).