Skip to content

[bugfix] Streaming: answer malformed client frames with invalid_message instead of dropping the session - #2073

Merged
mergify[bot] merged 2 commits into
hao-ai-lab:mainfrom
stayinalive181:streaming-malformed-frames
Nov 5, 2026
Merged

mergify[bot] merged 2 commits into
hao-ai-lab:mainfrom
stayinalive181:streaming-malformed-frames

Conversation

@stayinalive181

@stayinalive181 stayinalive181 commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

Purpose

On /v1/stream, one client frame that is not JSON text ends the session without an error frame.

_read_init_message and _run_segment_loop read frames with WebSocket.receive_json(). For text that is not JSON it raises json.JSONDecodeError, and for a binary frame it raises KeyError: 'text'. Neither is caught at those call sites, so the exception reaches the catch-all in _stream, which logs unhandled error and drops the socket. A frame that is valid JSON but fails schema validation already gets invalid_message and the session continues, and docs/design/server_contracts/streaming.md says validation errors are reported with error and non-fatal ones keep the connection open.

Measured over a real connection (uvicorn + a websockets client, with the mock generator from test_server.py):

Client sends main (f886424) This PR
{not json as the opening frame no frame; close 1006 error invalid_message (retryable: false); close 1003 invalid_init
a binary opening frame no frame; close 1006 same as the row above
{not json mid-session ltx2_stream_complete stop_requested; close 1006 error invalid_message (retryable: true); the session keeps serving (snapshot_state is answered)
a binary frame mid-session same as the row above same as the row above

Changes

  • _receive_client_json() reads one frame and decodes it. A binary frame or text that is not JSON raises _MalformedClientFrame.
  • Opening frame: rejected like an opening frame that fails validation (invalid_message, then close 1003 invalid_init).
  • Mid-session: invalid_message with retryable: true, and the loop continues, as for a schema-invalid frame.
  • streaming.md: one paragraph after the client-to-server table says client frames are JSON text and how a malformed frame is answered.

Test Plan

pytest fastvideo/tests/entrypoints/streaming/ -q
pre-commit run --files fastvideo/entrypoints/streaming/server.py fastvideo/tests/entrypoints/streaming/test_server.py

Test Results

  • Four new tests in test_server.py: a non-JSON and a binary opening frame, and both kinds mid-session followed by a snapshot_state round trip. Each runs under a 10 s SIGALRM deadline, so a missing frame fails the test instead of hanging the unit lane. On main all four fail: the two opening-frame tests with TimeoutError: no server frame within 10 s (the server sends nothing), and the two mid-session tests because they get ltx2_stream_complete instead of error.
  • fastvideo/tests/entrypoints/streaming/: 176 passed (172 on main), CPU only, macOS, Python 3.12, Starlette 1.10. The fMP4 tests ran with ffmpeg 9.0.
  • pre-commit on the changed files: yapf, ruff, codespell and mypy pass.
Real-connection output, main
{"case": "init_not_json", "received": [], "close": {"code": 1006, "reason": ""}}
{"case": "init_binary", "received": [], "close": {"code": 1006, "reason": ""}}
{"case": "mid_not_json", "received": [{"type": "queue_status"}, {"type": "gpu_assigned"}, {"type": "ltx2_stream_start"}, {"type": "ltx2_stream_complete", "reason": "stop_requested"}], "close": {"code": 1006, "reason": ""}}
{"case": "mid_binary", "received": [{"type": "queue_status"}, {"type": "gpu_assigned"}, {"type": "ltx2_stream_start"}, {"type": "ltx2_stream_complete", "reason": "stop_requested"}], "close": {"code": 1006, "reason": ""}}

ERROR [server.py:164] session bc131943: unhandled error
  ...
json.decoder.JSONDecodeError: Expecting property name enclosed in double quotes: line 1 column 2 (char 1)
ERROR [server.py:164] session c0f03f7b: unhandled error
  ...
KeyError: 'text'
Real-connection output, this PR
{"case": "init_not_json", "received": [{"type": "error", "code": "invalid_message", "retryable": false, "message": "opening frame failed validation: client frame is not valid JSON: Expecting property name e..."}], "close": {"code": 1003, "reason": "invalid_init"}}
{"case": "init_binary", "received": [{"type": "error", "code": "invalid_message", "retryable": false, "message": "opening frame failed validation: client frames must be JSON text, got a binary frame"}], "close": {"code": 1003, "reason": "invalid_init"}}
{"case": "mid_not_json", "received": [{"type": "queue_status"}, {"type": "gpu_assigned"}, {"type": "ltx2_stream_start"}, {"type": "error", "code": "invalid_message", "retryable": true, "message": "client frame is not valid JSON: ..."}, {"type": "continuation_state_snapshot", "state_payload": {"schema_version": 1, "segment_index": 3}}]}
{"case": "mid_binary", "received": [{"type": "queue_status"}, {"type": "gpu_assigned"}, {"type": "ltx2_stream_start"}, {"type": "error", "code": "invalid_message", "retryable": true, "message": "client frames must be JSON text, got a binary frame"}, {"type": "continuation_state_snapshot", "state_payload": {"schema_version": 1, "segment_index": 3}}]}

#2072 fixes the reason in the session's last frame: it fixes the stop_requested above and adds the documented session_timeout. Both PRs touch _run_segment_loop, so whichever merges second needs a small rebase. I will do it.

Checklist

  • I ran pre-commit run --all-files and fixed all issues (ran on the changed files)
  • I added or updated tests for my changes
  • I updated documentation if needed (streaming.md: client frames are JSON text, and how a malformed one is answered)
  • I considered GPU memory impact of my changes (none)

@mergify

mergify Bot commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

Merge Protections

🟢 Merge protection satisfied — ready to merge.

Show 1 satisfied protection

🟢 PR merge requirements

  • #approved-reviews-by>=1
  • check-success=fastcheck-passed
  • check-success=full-suite-passed
  • check-success~=pre-commit
  • title~=(?i)^\[(feat|feature|bugfix|fix|refactor|perf|ci|doc|docs|misc|chore|kernel|new.?model|skill|skills|infra)\]

@mergify mergify Bot added type: bugfix Bug fix scope: inference Inference pipeline, serving, CLI scope: infra CI, tests, Docker, build labels Nov 5, 2026
@greptile-apps

greptile-apps Bot commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] The PR appears safe to merge; no blocking issues remain.

Summary

Malformed client frames now receive invalid_message instead of silently ending the connection.

  • The streaming server answers malformed client frames with an error.

No new actionable issues were found.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Malformed client frame] --> B{Opening frame?}
    B -->|Yes| C[Send non-retryable invalid_message]
    C --> D[Close with 1003 invalid_init]
    B -->|No| E[Send retryable invalid_message]
    E --> F[Wait for next client message]
Loading

Reviews (2) · Last reviewed commit: "[bugfix] Streaming: bound the new tests ..." · Reviewed by Greptile

@fv-superagent fv-superagent left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fv-superagent score: 4/5

Read the diff and the call sites; the fix is right. _receive_client_json correctly treats a missing text as a binary frame, the disconnect branch mirrors Starlette's, and the two stages now match docs/design/server_contracts/streaming.md (:78 for the opening frame, :98 for non-fatal errors keeping the connection open). Nothing here blocks merge; two non-blocking points:

  1. fastvideo/tests/entrypoints/streaming/test_server.py:104-121, 168-195 — the new tests call receive_json() / receive() with no timeout, so a regression in the path under test hangs the unit lane (.buildkite/scripts/unit_test.sh) instead of failing it; pyproject.toml configures no pytest-timeout. The PR body already notes this for the before-change comparison. A bounded receive helper would make a future regression fail fast. The same blocking pattern already exists in this file, so this is a nit.

  2. docs/design/server_contracts/streaming.md:98 — the error row lists code, message, but ErrorMessage also carries retryable, which this change relies on, and "client to server frames are JSON text" is now a tested contract. One line in the catalogue would close that gap; the doc header only requires edits when protocol.py/session.py change, so "not needed" is defensible.

CI already collects this directory, and the four tests cover both frame kinds at both stages, including a snapshot_state round trip that proves the session survives.

— fv-superagent · run by @aryan5v

@stayinalive181

Copy link
Copy Markdown
Contributor Author

Thanks. Addressed in 9e40574:

  1. The four new tests now run under a 10 s SIGALRM deadline. On main, the two opening-frame tests now fail with TimeoutError instead of blocking the unit lane. I used a signal rather than a TestClient internal because fastapi>=0.129 still allows Starlette 0.40, whose websocket test session has different internals.
  2. streaming.md has a paragraph after the client-to-server table: client frames are JSON text, and a malformed frame gets invalid_message. As the opening frame, the socket then closes with 1003 invalid_init; later in the session the error is retryable and the session continues. I put the missing retryable field on the error row in [bugfix] Streaming: end sessions with the documented segment_cap, session_timeout and error frames #2072, which already edits that row, to keep the two PRs from conflicting on it.
  3. I removed the comment Greptile flagged.

@mergify mergify Bot added the scope: docs Documentation label Nov 5, 2026

@fv-superagent fv-superagent left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fv-superagent score: 5/5 (was 4/5)

Re-reviewed only the delta in 9e40574; server.py is unchanged.

  • Point 1 (tests could hang on regression) is addressed: _deadline() wraps the three websocket tests with a 10 s SIGALRM that raises TimeoutError, and no-ops where SIGALRM is unavailable. A regression now fails the unit lane instead of stalling it.
  • Point 2 (contract) is addressed: a paragraph after the client to server table in streaming.md states that client frames are JSON text and that a malformed frame is answered with invalid_message — 1003 invalid_init on the opening frame, retryable and session-continuing mid-session. Moving the retryable field on the error row to #2072 is a fair call to avoid a conflict in that table.
  • The comment Greptile flagged is gone.

Nothing blocking from my side.

— fv-superagent · run by @aryan5v

@fv-superagent fv-superagent mentioned this pull request Nov 5, 2026

@fv-superagent fv-superagent left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fv-superagent score: 5/5 — malformed-frame handling matches the streaming contract; Greptile clean and the new tests are bounded.

@fv-superagent

Copy link
Copy Markdown
Collaborator

/merge

@github-actions github-actions Bot added the ready PR is ready to merge label Nov 5, 2026
@mergify
mergify Bot merged commit 5894099 into hao-ai-lab:main Nov 5, 2026
24 checks passed
@mergify

mergify Bot commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

This pull request does not currently match the merge queue conditions, so it cannot be queued from here. The box comes back if it matches again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready PR is ready to merge scope: docs Documentation scope: inference Inference pipeline, serving, CLI scope: infra CI, tests, Docker, build type: bugfix Bug fix

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants