Skip to content

feat(sensors): harden LangChain and LangGraph callback handler (#759) - #787

Merged
SHAURYASANYAL3 merged 1 commit into
Shinraya-Tech-Labs:mainfrom
chanchall26:feat/759-langchain-callback-handler
Oct 26, 2026
Merged

SHAURYASANYAL3 merged 1 commit into
Shinraya-Tech-Labs:mainfrom
chanchall26:feat/759-langchain-callback-handler

Conversation

@chanchall26

@chanchall26 chanchall26 commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #759

Description

agentwatch/sensors/langchain/ already contained a callback handler (AgentWatchLangChainSensor) that emits raw callback payloads with run_id, parent_run_id and root_run_id. The agentwatch/sensors/langchain.py path in the guidance cannot coexist with that package (the package would shadow it), so I hardened the existing handler in place instead of adding a second one. The tests are at tests/sensors/test_langchain_sensor.py as requested.

Changes:

  • Every on_* callback is wrapped so nothing raised while building or emitting an observation can reach the user's chain. Failures are counted in ctx.errors.
  • Per-run state (root_run_id map plus a monotonic start time) is released on each terminal callback (*_end, *_error), so a long-lived sensor no longer grows without bound. agent_action and agent_finish share the chain's run id and do not touch that state.
  • State is guarded by a lock for concurrent LangGraph branches.
  • End and error payloads carry duration_ms, measured per run with a monotonic clock.
  • llm_end carries a normalized token_usage (from llm_output or message usage_metadata). The LangChain normalizer falls back to it (one line).
  • LANGCHAIN_AVAILABLE and require_langchain() give a clear install hint (pip install "agentwatch-ai[langchain]"). Importing the module and constructing the sensor still need no LangChain.

Plane: A (Evidence, sensors). The normalizer change is a one-line additive fallback.

Testing

Run locally on Python 3.12.15 (Windows), langchain-core not installed, no live LangChain or model calls (duck-typed fakes):

  • pytest tests/sensors tests/v3: 366 passed, 5 skipped. The new file has 22 test cases: llm, chat model, tool, chain and retriever success; error paths; LangGraph nesting and root ids; agent action and finish; token usage; failing sink; malformed events; 8 concurrent threads; state cleanup; missing langchain_core (run in a subprocess).
  • Coverage of agentwatch.sensors.langchain from those tests: 98%.
  • pytest tests (the whole suite; I did not use the CI flags -v, --cov or --cov-fail-under): 1919 passed, 17 skipped, 4 deselected. I deselected tests/test_cli_server.py because I did not want to start servers.
  • ruff check agentwatch/: clean. ruff format --check agentwatch/: clean.
  • bandit -r agentwatch: 0 issues.
  • mypy on the two changed source files only: clean.

Not verified / not done

  • The integration tests, make typecheck (full), pip-audit and the frontend jobs were not run.
  • Not tested against a real langchain-core or LangGraph install.
  • I did not check the combined repo-wide --cov-fail-under=70 figure.
  • No new agentwatch/sensors/langchain.py module was created (see Description).

…aya-Tech-Labs#759)

Never raise into the host chain, release per-run state on terminal callbacks, add duration and token usage, and add require_langchain() with an install hint.

Signed-off-by: chanchall26 <190047591+chanchall26@users.noreply.github.com>
@github-actions

Copy link
Copy Markdown
Contributor

🧪 PR Test Results

Check Result
Tests (pytest tests/) ✅ success
Lint (ruff check .) ❌ failure
Coverage (agentwatch) 79.88%

Python 3.12 · commit 9f5cca9

@SHAURYASANYAL3 SHAURYASANYAL3 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NSoC '26 Maintainer Verification & Approval 🚀

  • Architecture & Plane Isolation: Confirmed clean Plane A (Sensors) isolation with defensive exception boundaries and memory leak prevention via monotonic per-run state eviction.
  • Concurrency & Resilience: Lock guarding across asynchronous/concurrent LangGraph branches verified.
  • CI / Quality Gate: 100% green pass across all matrix builds, security scanners, linters, and test suites.
  • Accreditation: Scored for NSoC '26 (Level 2).

Merging into main! Excellent work @chanchall26.

@SHAURYASANYAL3
SHAURYASANYAL3 merged commit 01bd6e8 into Shinraya-Tech-Labs:main Oct 26, 2026
14 checks passed
@chanchall26

chanchall26 commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor Author

Hi! Thank you for merging this contribution. If this PR is eligible for NSoC 2026, could you please add the exact
nsoc26 label alongside its existing level label? The program rules require
nsoc26 and a level label for the PR to be counted. If NSoC 26' is the project’s official equivalent, please let me know. Thank you!

@SHAURYASANYAL3 SHAURYASANYAL3 added the soc26 Nexus Spring of Code 2026 label Oct 28, 2026
@SHAURYASANYAL3

Copy link
Copy Markdown
Collaborator

Hi @chanchall26! 👋 Thank you for checking!

We have created the official soc26 label in the repository and attached both soc26 and NSoC 26' alongside the corresponding level labels across all your merged PRs (#776, #777, #779, #780, #781, #782, #787).

Your contributions are now fully labeled and indexed for NSoC 2026 scoring! 🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

level2 NSoC 26' soc26 Nexus Spring of Code 2026

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Sensors] LangChain & LangGraph Native Callback Handler [Level 2]

2 participants