feat(sdk): support custom title generation prompts - #4564
Open
DevinVinson wants to merge 7 commits into
Open
Conversation
Co-authored-by: openhands <openhands@all-hands.dev>
Contributor
|
This was referenced Aug 20, 2026
Contributor
Python API breakage checks — ✅ PASSEDResult: ✅ PASSED |
Contributor
REST API breakage checks (OpenAPI) — ✅ PASSEDResult: ✅ PASSED |
5 tasks
DevinVinson
added
integration-test
Contributor
|
Hi! I started running the integration tests on your PR. You will receive a comment with the results shortly. |
Contributor
Coverage Report •
|
|||||||||||||||||||||||||||||||||||
Contributor
🧪 Integration Tests ResultsOverall Success Rate: 97.7% 📁 Detailed Logs & ArtifactsClick the links below to access detailed agent/LLM logs showing the complete reasoning process for each model. On the GitHub Actions page, scroll down to the 'Artifacts' section to download the logs.
📊 Summary
📋 Detailed Resultslitellm_proxy_deepseek_deepseek_v4_flash
Skipped Tests:
litellm_proxy_anthropic_claude_sonnet_4_6
Failed Tests:
litellm_proxy_gemini_3.1_pro_preview
litellm_proxy_openai_gpt_5.5
litellm_proxy_minimax_MiniMax_M2.7
Skipped Tests:
|
DevinVinson
marked this pull request as ready for review
August 21, 2026 10:47
Collaborator
|
🚦 CI is currently failing on this PR's latest commit. Please fix the failing checks before OpenHands reviews it - this is re-checked automatically once you push a new commit. (A maintainer can also request This is an automated check - no AI was used to generate this comment. |
VascoSch92
reviewed
Aug 21, 2026
VascoSch92
reviewed
Aug 21, 2026
Comment on lines
+99
to
+116
| if title_generation_prompt and title_generation_prompt.strip(): | ||
| prompt = title_generation_prompt.strip() | ||
| includes_conversation = CONVERSATION_CONTENT_PLACEHOLDER in prompt | ||
| user_prompt = prompt.replace( | ||
| CONVERSATION_CONTENT_PLACEHOLDER, truncated_message | ||
| ).replace(MAX_LENGTH_PLACEHOLDER, str(max_length)) | ||
| if not includes_conversation: | ||
| user_prompt = f"{user_prompt}\n\nConversation content:\n{truncated_message}" | ||
| else: | ||
| user_prompt = ( | ||
| f"Generate a title (maximum {max_length} characters) " | ||
| f"for a conversation that starts with this message:\n\n" | ||
| f"{truncated_message}." | ||
| "Also make sure to include ONE most relevant emoji at " | ||
| "the start of the title." | ||
| f" Choose the emoji from this list:{emojis_descriptions} " | ||
| ) | ||
|
|
Member
There was a problem hiding this comment.
template = (title_generation_prompt or "").strip()
if template:
user_prompt = template.replace(
CONVERSATION_CONTENT_PLACEHOLDER, truncated_message
).replace(MAX_LENGTH_PLACEHOLDER, str(max_length))
if CONVERSATION_CONTENT_PLACEHOLDER not in template:
user_prompt += f"\n\nConversation content:\n{truncated_message}"
else:
...
VascoSch92
reviewed
Aug 21, 2026
| {"emoji": "♻️", "name": "refactor", "description": "Code refactoring"}, | ||
| ] | ||
|
|
||
| CONVERSATION_CONTENT_PLACEHOLDER = "{conversation_content}" |
Member
There was a problem hiding this comment.
Final[str] in both from typing.
VascoSch92
reviewed
Aug 21, 2026
VascoSch92
reviewed
Aug 21, 2026
VascoSch92
reviewed
Aug 21, 2026
Co-authored-by: Vasco Schiavo <115561717+VascoSch92@users.noreply.github.com>
github-actions
Bot
removed
the
release-note-required
Co-authored-by: Vasco Schiavo <115561717+VascoSch92@users.noreply.github.com>
…vice.py Co-authored-by: Vasco Schiavo <115561717+VascoSch92@users.noreply.github.com>
Co-authored-by: Vasco Schiavo <115561717+VascoSch92@users.noreply.github.com>
Co-authored-by: openhands <openhands@all-hands.dev>
Member
Author
|
Thank you @VascoSch92 ! I've updated to |
VascoSch92
approved these changes
Aug 21, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
HUMAN: Will need some 👀 to check but hopefully this is the right direction.
AGENT:
Implemented and exercised the complete Agent Server path: request validation and
conversation metadata persistence, AutoTitleSubscriber forwarding, prompt rendering,
and the LLM completion message. A live provider-backed run was not performed because
this environment has no LLM credentials; the PR remains a draft for human E2E testing.
Why
Users can choose a title LLM profile, but cannot customize the prompt used to generate
automatic conversation titles. This adds the Agent Server contract required by
#4561 and OpenHands/OpenHands#16761.
Summary
title_generation_promptconversation setting with a 2,000-character limit.{conversation_content}and{max_length}placeholders while safely appending conversation content when its placeholder is omitted.REST API contract changes
Compared with base OpenAPI
1de2e6d1bfcffor public/api/**paths.Issue Number
Closes #4561
How to Test
uv run pytest tests/sdk/conversation/test_generate_title.py tests/agent_server/test_conversation_service.py tests/agent_server/test_event_service.py.autotitle: trueand a prompt such asWrite a title under {max_length} characters for: {conversation_content}; after the first user message, confirm the completion receives the rendered custom user prompt andmeta.jsonretains the template.Validation performed:
make buildVideo/Screenshots
Not applicable to this backend-only change. Live LLM E2E evidence is pending human testing.
Design Doc
Not included; the API behavior and fallback rules are documented in the companion docs PR linked below.
Type
Notes
🐳 Agent Server images for this PR — GHCR package, pull/run commands, and all pushed tags (click to expand)
• GHCR package: https://github.com/OpenHands/agent-sdk/pkgs/container/agent-server
Variants & Base Images
eclipse-temurin:17-jdknikolaik/python-nodejs:python3.13-nodejs22-slimgolang:1.21-bookwormPull (multi-arch manifest)
# Each variant is a multi-arch manifest supporting both amd64 and arm64 docker pull ghcr.io/openhands/agent-server:4afe9a9-pythonRun
All tags pushed for this build
About Multi-Architecture Support
4afe9a9-python) is a multi-arch manifest supporting both amd64 and arm644afe9a9-python-amd64) are also available if needed