Skip to content

fix: assert DingTalk empty-config Gateway responses - #477

Merged
vincentkoc merged 2 commits into
mainfrom
fix/dingtalk-response-expectations-20261015
Oct 15, 2026
Merged

vincentkoc merged 2 commits into
mainfrom
fix/dingtalk-response-expectations-20261015

Conversation

@vincentkoc

@vincentkoc vincentkoc commented Oct 15, 2026 •

Copy link
Copy Markdown
Member

What Problem This Solves

Crabpot cannot declare exact method-specific Gateway responses, so intentional empty-config rejections appear as unexpected failures.

User Impact

Fixture-owned expectations execute each retained handler and require its exact response. Undeclared rejections, mismatches, missing methods, blocked expected methods, and handler failures remain failures. Both captured and entrypoint execution routes use the same declarations.

Why This Change Was Made

Validate the optional expectation map in the manifest and schema, forward it through both synthetic routes, and declare DingTalk's twelve existing credential-free rejection contracts. Adopt landed Inspector source 9df499d8884883bfaaeb1eeaac0da5ffe9b9e80b from openclaw/plugin-inspector#110. The published CLI remains pinned to 0.3.26, which does not contain the new API; source and locally packed delivery provide it without publication.

Adds 35 net runtime lines, 25 schema lines, 16 fixture lines and 67 net test lines. No submodule or package version changes are included.

Evidence

  • Independent source review is clear. The complete branch at a61680cbdd46390be713df7bd2da11eea7ddafd2 received a fresh P0–P2 automated review after adopting the landed source pin.
  • The changed exact source-pin test passes. Inspector's landed tree 9468d9a33927720de29dacd1c4c56aad62548315 is byte-identical to the reviewed and executed source head 955863f12ec3f552cbeb8a7d7b9d03d7ab93b7e4.
  • Prior isolated AWS run run_20ebfd050066 used Node 24.19.0, Crabpot b9f6fcd06a1dead9dbac567f696e463fccbeb740 and that Inspector source. Consumer tests passed 68/68, including both execution routes and source-loader cases; Inspector focused/sibling tests passed 98/98. All 61 unpublished package files matched source.
  • That run built actual DingTalk 5fef12d37377e299e26d18b0145baf646cd17a8b (0.8.26) with its declared OpenClaw 2026.8.1 dependency, then exercised it under the mock SDK. Baseline Inspector produced 3 pass / 12 fail / 1 blocked; source and packed candidates each produced 15 pass / 0 fail / 1 blocked. It verified twelve exact negative responses and three exact diagnostics. A deliberate mismatch produced 14 pass / 1 fail / 1 blocked; the channel prerequisite remained blocked.
  • Source and packed native CLI smoke each passed with one fixture and zero breakages. Generic runtime smoke preserved its intentional Gateway failure; resource checks cover the collector contract only. Exit 0; the owned lease was released.

DingTalk has no tracked lockfile. That run generated one and installed with npm ci; lock SHA256: 22e2d47f5f7ff6e4888c2b8b93edc9aec9868258b8e0a95e45451bd6a5bef8f8. This is qualification of that resolved build and the assertion path under the mock SDK, not live DingTalk or full host compatibility. The prior run is not a fresh execution of this final commit; final CI checks the landed source-pin integration. The initial local source-loader attempt failed three cases before the Inspector commit was pushed; the isolated run passed those cases.

Related: #327 remains open and held. This PR retains the current DingTalk gitlink 39bdb2d8a7468237858906d255a717cd03b037ad; it does not adopt 5fef12d37377e299e26d18b0145baf646cd17a8b, supersede the dependency PR, or claim its upgrade is ready. The installed mandatory review helper cannot admit a gitlink diff, so that dependency change is outside this PR's explicit scope.

Fixture Impact

  • Updates fixture manifest
  • Updates submodule pins
  • Changes contract or smoke logic

Notes

No credentials, network-backed plugin operation, package publication, tag, deployment or release is included. Published CLI remains pinned to 0.3.26.

Final Review Follow-through

The fresh ClawSweeper review covers a61680cbdd46390be713df7bd2da11eea7ddafd2: no findings or before-merge requirements. It independently verified the twelve guards at the retained DingTalk revision.

The optional rank-up suggestion to link raw output is acknowledged. The existing proof summary and private-log decision apply: coordinator and executor directly inspected the source, unpublished packed, and deliberate-mismatch results. Raw lease logs remain private; no new artifact upload or repeat run is needed. This does not extend the reported target-revision proof to a fresh execution of the retained revision.

Fresh CI passed Ubuntu, macOS, Windows, and container static checks at this head. The Ubuntu source-and-packed Inspector smoke, runtime smoke, and resource collector checks passed. The changed-fixture lane is correctly unselected because no submodule pin changes.

@vincentkoc

Copy link
Copy Markdown
Member Author

@clawsweeper review

@clawsweeper

clawsweeper Bot commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

🦞👀
Exact review queued.

Re-review progress:

@clawsweeper clawsweeper Bot added P2 Normal priority bug or improvement with limited blast radius. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. labels Oct 15, 2026
@clawsweeper

clawsweeper Bot commented Oct 15, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs maintainer review before merge.

What this changes

This PR declares DingTalk’s twelve empty-config Gateway rejection contracts and forwards exact expectations through both synthetic execution routes.

Example: An empty-config probe invokes dingtalk-connector.sendToUser.

  • Before: The intended response {ok:false,payload:{error:"DingTalk not configured"}} fails qualification.
  • After: That exact declared response passes; a different response or handler failure still fails.

Review scores

Measure Result What it means
Overall readiness 🦐 gold shrimp (3/6) Useful, focused implementation with meaningful integration coverage and no actionable finding; independently inspectable behavior evidence is limited.
Proof confidence 🦐 gold shrimp (3/6) This MEMBER-authored PR is outside the ordinary contributor-proof gate. Its body reports source and packed mock-SDK execution improving DingTalk from 3 pass/12 fail/1 blocked to 15 pass/0 fail/1 blocked, with a deliberate mismatch still failing. The linked author discussion confirms raw logs are private; those reported runs used DingTalk 5fef12d rather than the retained 39bdb2d, although all twelve retained-source guards were independently verified.
Patch quality 🐚 platinum hermit (4/6) No actionable review findings were identified.

Product

Kind: Feature · Worth it: Yes · Fix scope: Complete
User problem: Credential-free qualification reports intentional DingTalk rejection contracts as unexpected failures.
Reason: Exact fixture assertions add useful negative-contract coverage, and the maintainer-authored direction is supported by the landed Inspector implementation.

Merge readiness

✅ Ready for maintainer review

Keep open: this maintainer-authored PR adds useful fixture assertions absent from main, and review found no concrete introduced defect.

Priority: P2
Reviewed head: a61680cbdd46390be713df7bd2da11eea7ddafd2

Before merge

None.

Findings

None.

Tests

  • Missing end-to-end proof: No publicly viewable trace shows the changed fixture assertions through the shipped entry point at the retained DingTalk revision; supplied qualification reports a newer revision under the mock SDK.
Agent review details

How this fits together

Crabpot supplies fixture declarations and retained plugin handlers to Plugin Inspector, which executes synthetic probes and returns pass, fail, or blocked results.

flowchart LR
  A[Fixture manifest] --> B[Crabpot synthetic runner]
  C[Captured or entrypoint handlers] --> B
  B --> D[Plugin Inspector]
  D --> E[Exact response comparison]
  E --> F[Probe report]
Loading

Technical review

Best possible solution:

Keep exact response comparison in Inspector and fixture-owned declarations in Crabpot while preserving undeclared failures and separate dependency qualification.

Do we have a high-confidence way to reproduce the issue?

Source inspection establishes that the retained DingTalk handlers intentionally reject empty requests while main lacks fixture declarations to assert those responses; no target code was executed during this read-only review.

Is this the best way to solve the issue?

The implementation uses the existing response owner and fixture-selection path, providing exact negative-contract coverage without skipping handlers or broadly suppressing rejections.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning medium; reviewed against d300f1bf92fb.

Provenance checked

Testing

Proof path: in-process harness.

Security

None.

Evidence

What I checked:

  • Introduced change and main boundary: The pinned diff adds manifest validation, fixture declarations, and forwarding through both existing execution routes; main has no fixture-owned response expectation support. (scripts/synthetic-probes.mjs:197, a61680cbdd46)
  • Repository policy applied: Read the complete root AGENTS.md; no nested policy or maintainer-notes entries exist. The change remains fixture-driven, updates the source pin and test expectations, and preserves credential-free checks. (AGENTS.md:1, a61680cbdd46)
  • Authoritative assertion implementation and dependency policy: Crabpot executes this pinned implementation, making its contract and complete root AGENTS.md applicable. The runner compares serialized frames exactly and retains failures for mismatches, unavailable expected methods, malformed frames, and unsettled handlers; undeclared rejections still fail. (src/synthetic-probes.js:714, 9df499d88848)
  • Retained fixture contract verified: The retained revision contains the nine DingTalk-not-configured guards and the probe rejection declared by this PR; card-bridge.ts separately matches both declared argument guards. These runtime contracts are the test oracle; the repository tree contains no AGENTS.md. (src/gateway-methods.ts:60, 39bdb2d8a746)
  • Landed source identity and supplied qualification: The landed pin and reported executed head share tree 9468d9a33927720de29dacd1c4c56aad62548315. The author’s discussion at fix: verify expected Gateway rejection responses plugin-inspector#110 (comment) confirms that raw qualification logs remain private and limits claims to mock-SDK execution. (9df499d88848)
  • Distinct dependency update: chore(deps): bump plugins/dingtalk-connector from 39bdb2d to 5fef12d #327 updates the DingTalk gitlink; this PR retains 39bdb2d8a7468237858906d255a717cd03b037ad and explicitly leaves that dependency qualification separate. (crabpot.config.json:1295, a61680cbdd46)

Likely related people:

  • Vincent Koc: Raw commit 5f5e290 adds scripts/synthetic-probes.mjs:193 relative to its recorded parents. This identifies author metadata, not feature responsibility or a PR merger. (role: source-line author; confidence: high; commits: 5f5e29042745; files: scripts/synthetic-probes.mjs)
  • Peter Steinberger: Raw commit e4a6cfd adds scripts/synthetic-probes.mjs:195 relative to its recorded parents. This identifies author metadata, not feature responsibility or a PR merger. (role: source-line author; confidence: medium; commits: e4a6cfdcbc57; files: scripts/synthetic-probes.mjs)

Review metrics

Metric Value Why it matters
Implementation and regression coverage Production JavaScript +36/-1; tests +69/-2; fixture/schema +41 Focused growth adds declaration validation and forwarding, with coverage for fixture isolation, explicit overrides, mismatches, and both execution routes.

Labels

Label changes:

  • add P2: This improves credential-free plugin qualification with limited blast radius and no demonstrated urgent runtime regression.
  • add rating: 🦐 gold shrimp: Overall readiness is 🦐 gold shrimp; proof is 🦐 gold shrimp and patch quality is 🐚 platinum hermit.
  • add status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR.

Label justifications:

  • P2: This improves credential-free plugin qualification with limited blast radius and no demonstrated urgent runtime regression.
  • rating: 🦐 gold shrimp: Overall readiness is 🦐 gold shrimp; proof is 🦐 gold shrimp and patch quality is 🐚 platinum hermit.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR.

Rank-up moves

Optional improvements that raise the rating; they are not merge blockers.

  • Optionally link existing redacted output showing exact declarations and the deliberate mismatch through the actual fixture execution path.

Rating scale

6/6 🦀 challenger crab · 5/6 🦞 diamond lobster · 4/6 🐚 platinum hermit · 3/6 🦐 gold shrimp · 2/6 🦪 silver shellfish · 1/6 🧂 unranked krab. Overall follows the weaker of proof and patch quality; ✨ marks media proof (a screenshot, video, or linked artifact) that directly shows the changed behavior.

Workflow

ClawSweeper edits this one comment on every review. Comment @clawsweeper re-review for a fresh review only. Repair and merge execution lanes are retired.

Reviewed October 14, 2026, 11:26 PM ET / October 15, 2026, 03:26 UTC.

@vincentkoc
vincentkoc marked this pull request as ready for review October 15, 2026 03:29
@vincentkoc
vincentkoc requested a review from a team as a code owner October 15, 2026 03:29
@clawsweeper

clawsweeper Bot commented Oct 15, 2026

Copy link
Copy Markdown
Contributor

ClawSweeper reviews are opt-in for maintainers. If a review is required, finish the work and validation first, then comment @clawsweeper review. Please avoid repeated requests while a review is queued or running; agents should not request reviews automatically.

@vincentkoc
vincentkoc merged commit 5fbe914 into main Oct 15, 2026
25 of 26 checks passed
@vincentkoc
vincentkoc deleted the fix/dingtalk-response-expectations-20261015 branch October 15, 2026 03:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

P2 Normal priority bug or improvement with limited blast radius. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant