Skip to content

fix: preserve seller reasoning profiles across API adaptation - #976

Open
alexanderludwig wants to merge 4 commits into
mainfrom
fix/seller-reasoning-profile
Open

fix: preserve seller reasoning profiles across API adaptation#976
alexanderludwig wants to merge 4 commits into
mainfrom
fix/seller-reasoning-profile

Conversation

@alexanderludwig

@alexanderludwig alexanderludwig commented Sep 1, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • preserve explicit reasoning effort across Chat Completions → canonical request → Responses API adaptation
  • preserve completed Responses output items when CatGPT streams a valid answer but sends an empty response.completed.response.output
  • verify both changes against the real CatGPT backend with minimal authenticated smoke requests

Root cause

This investigation found two separate defects at different layers.

1. Request profile loss — buyer protocol adapter

The buyer sent reasoning_effort: "none", but the canonical request representation did not carry reasoning effort. During Chat Completions → Responses adaptation, the field was silently discarded and the seller therefore received no equivalent reasoning object.

Historical authenticated audit evidence showed this consistently across the affected CatGPT run: the buyer request contained reasoning_effort: "none", while the adapted seller request omitted reasoning and CatGPT reported its default effort instead.

2. Blank response — seller Responses SSE collapse

After the request mapping was fixed, a real CatGPT smoke request proved that CatGPT honored reasoning: { effort: "none" } and generated the requested text. However, CatGPT's stream ended with this event pattern:

response.output_item.done  → complete assistant message containing "42"
response.completed         → response.output: []

The OpenAI Responses seller ignored response.output_item.done and collapsed the stream using only the terminal response.completed.response. That discarded the valid output item and returned a blank non-stream response, which caused verifier batches to remain UNDETERMINED.

Fix

Reasoning profile preservation

The canonical adapter now carries reasoning effort losslessly:

Chat Completions                       Canonical                    Responses API
reasoning_effort: "none"  <->  reasoningEffort: "none"  <->  reasoning: { effort: "none" }

The canonical value remains model-agnostic rather than using a global hard-coded allowlist. The adapter preserves the requested profile; the selected upstream model remains responsible for accepting or rejecting it.

Specification-aligned output preservation

For forced-stream requests serving non-stream clients, the seller now collects complete response.output_item.done items by output_index. If the terminal response already contains output, it is returned unchanged. If the terminal response has an empty or missing output, the seller restores the completed streamed items.

This keeps the upstream item IDs, types, statuses, content, tool calls, refusals, and ordering intact. It does not synthesize message IDs or add the SDK-only top-level output_text convenience field.

OpenAI API references

These references support both sides of the fix: preserve the client's reasoning setting while adapting request protocols, and preserve completed standard output items while collapsing a Responses stream. The final terminal response remains authoritative whenever it already contains output; reconstruction is only the compatibility fallback for CatGPT's observed empty terminal output.

Scope

This PR contains:

  • a buyer/shared-adapter fix for request-profile preservation
  • a seller-provider fix for CatGPT's empty terminal output behavior
  • focused regression coverage and a changelog update

It does not change routing, verifier classification, or audit thresholds. It also does not claim to fix unrelated malformed responses such as refusals, non-numeric prose, or failures from other seller implementations.

Verification

Local validation:

  • pnpm --filter @antseed/api-adapter test — 120 tests passed
  • pnpm --filter @antseed/provider-openai-responses test — 29 tests passed
  • pnpm --filter @antseed/api-adapter typecheck
  • pnpm --filter @antseed/provider-openai-responses typecheck
  • pnpm --filter @antseed/api-adapter build
  • pnpm --filter @antseed/provider-openai-responses build

Real CatGPT smoke verification on September 1, 2026:

{
  "statusCode": 200,
  "reasoningEffort": "none",
  "reasoningTokens": 0,
  "outputCount": 1,
  "outputItemType": "message",
  "outputItemStatus": "completed",
  "contentType": "output_text",
  "contentText": "42",
  "hasTopLevelOutputText": false
}

This was a minimal funded smoke test, not a full-network verification audit. A separate funded audit is still required to measure the reduction in UNDETERMINED results across the network.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant