fix(sdk): strip leading 思考 block from LLM-generated title - #4534
fix(sdk): strip leading 思考 block from LLM-generated title#4534santhiprakash wants to merge 1 commit into
Conversation
|
👋 This PR needs a couple of things fixed before OpenHands can review it:
Push an update once this is addressed and this check re-runs automatically. This is an automated check - no AI was used to generate this comment. |
90911af to
e7133f1
Compare
|
🚦 CI is currently failing on this PR's latest commit. Please fix the failing checks before OpenHands reviews it - this is re-checked automatically once you push a new commit. (A maintainer can also request This is an automated check - no AI was used to generate this comment. |
Some providers (notably Qwen3 behind Nebius) return reasoning inline in content as `思考...思考` instead of the normalized `reasoning_content` field. The transparent leak surfaced in Agent Canvas 1.14.0 (openhands/OpenHands#16687) as the autogenerated conversation title containing the raw 思考 reasoning markup. `generate_title_with_llm()` now peels a leading 思考 block from the LLM response text before using it as the title. The peel is conservative: - A leading closed block `思考…思考` is stripped and the remainder is returned. - An unterminated leading block (response begins with 思考 but never closes) consumes the entire response, and the LLM title path returns None so the existing user-message truncation fallback supplies the title. - Mid-text literal 思考 occurrences in a legitimate title are preserved. Fixes OpenHands#4530.
e7133f1 to
44c81c7
Compare
|
📁 PR Artifacts Notice This PR contains a |
|
Rebased onto current Verified locally on the rebased head:
Heads-up: every workflow run on this PR (including the original Aug 18 head) has been sitting in action_required — "awaiting workflow approval" — so CI has never actually executed. The bot's "CI failing" notice reflects that gate, not a test failure. Could a maintainer approve the workflow runs so the suite can run? |
|
I think this is probably a fix on the litellm side. The "think" blocks should be parsed out into the |
HUMAN:
Santhi Prakash — small correctness fix for the SDK title-leak bug class cross-referenced from #4541; see AGENT section for evidence.
AGENT:
Why
Some providers (notably Qwen3 behind Nebius) return reasoning inline
in
contentas思考...思考instead of the normalizedreasoning_contentfield. The transparent leak surfaced in AgentCanvas 1.14.0 (software-agent-sdk#4541)
as the autogenerated conversation title containing the raw
思考reasoning markup — the SDK ships the reasoning into the title, and Agent
Canvas just renders it.
This is the SDK-side root-cause fix: peeling the leading
思考block inthe SDK means every consumer of
Conversation.generate_title()and theagent-server auto-title path benefits, not just Agent Canvas.
Summary
generate_title_with_llm()now peels a leading思考block from theLLM title response before using it as the title.
思考block (closed or unclosed) yieldsno LLM title and falls back to the existing user-message truncation.
思考occurrences in a legitimate title arepreserved.
Issue Number
Fixes #4530
How to Test
Expected: 12 passed (4 new regression tests + 8 existing).
The 4 new tests cover:
test_generate_title_strips_leading_think_block— closed思考…思考block is peeled, the real title after the block is returned.test_generate_title_unclosed_think_block_falls_back— unterminatedleading
思考…returns no LLM title; truncation fallback supplies thetitle.
test_generate_title_think_only_response_falls_back— response thatis only a
思考block yields no LLM title; truncation fallbacksupplies the title.
test_generate_title_preserves_mid_text_think_literal— a mid-textliteral
思考occurrence in a legitimate title is preserved.I verified the regression coverage by reverting
generate_title_with_llm()to its pre-fix behavior and re-running thenew tests: 3/4 fail on the unfixed code (the literal-preservation test
correctly passes either way), proving the tests do catch the bug class.
Type
Notes
it does not touch the Agent Canvas frontend rendering issues (plaintext
thoughts / duplicate thought block) covered separately in
[Bug]: Weird <think> behavior with Qwen3-32B #4541.
block is stripped, so a legitimate title that happens to contain the
characters
思考mid-text is preserved.returning the raw reasoning string, so the LLM path surfaces no title
for the truncation fallback to take over.