Skip to content

[bugfix] Streaming: end sessions with the documented segment_cap, session_timeout and error frames - #2072

Open
stayinalive181 wants to merge 3 commits into
hao-ai-lab:mainfrom
stayinalive181:streaming-end-of-session-messages
Open

stayinalive181 wants to merge 3 commits into
hao-ai-lab:mainfrom
stayinalive181:streaming-end-of-session-messages

Conversation

@stayinalive181

@stayinalive181 stayinalive181 commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

Purpose

The last frame a /v1/stream client receives does not say why the session ended.

_handle_session sends ltx2_stream_complete with reason: "stop_requested" from a finally, whatever stopped the segment loop. protocol.py defines the segment_cap and error reasons and a SessionTimeout frame, and docs/design/server_contracts/streaming.md says session_timeout is sent when the session hits session_timeout_seconds and is "immediately followed by close". The server never sends either the timeout frame or those two reasons.

Clients built to that contract read the frames differently. The DreamVerse web client sets sessionExpired: true on session_timeout and treats ltx2_stream_complete as a normal finish, and DreamVerse's own backend sends session_timeout and then closes with 1000. Against this server, an idle timeout therefore looks like a stream that finished normally.

Measured over a real connection (uvicorn + a websockets client, with the mock generator from test_server.py, generation_segment_cap=2, session_timeout_seconds=2):

How the session ends main (f886424) This PR
reaches generation_segment_cap ltx2_stream_complete stop_requested; close 1006 ltx2_stream_complete segment_cap; close 1000 segment_cap; session state COMPLETE
idle for session_timeout_seconds ltx2_stream_complete stop_requested; close 1006 session_timeout {"timeout_seconds": 2}; close 1000 session_timeout
no opening frame within session_timeout_seconds no frame; close 1006 session_timeout; close 1000 session_timeout
unexpected exception in the segment loop (unit test only) ltx2_stream_complete stop_requested error internal_error (retryable: false); close 1011 internal_error

Changes

  • _run_segment_loop returns why it stopped (segment_cap, idle_timeout or disconnected), and _handle_session sends the matching last frame and closes:
    • segment_cap: ltx2_stream_complete segment_cap, close 1000, transition to COMPLETE (the state streaming.md gives a capped session).
    • idle_timeout (and the opening-frame timeout in _read_init_message): session_timeout, close 1000.
    • disconnected: nothing. Before, the finally tried to send to a closed socket.
  • An unexpected exception from the loop sends an error frame (internal_error, not retryable), closes with 1011, and is re-raised to the existing catch-all in _stream, which logs it as before. It sends no ltx2_stream_complete: the DreamVerse client clears its error notice on stream/completed.
  • streaming.md: the lifecycle diagram puts segment_cap under COMPLETE. The ltx2_stream_complete, session_timeout and error rows give their fields and close codes, and mark stop_requested and error as reserved reasons.

Test Plan

pytest fastvideo/tests/entrypoints/streaming/ -q
pre-commit run --files fastvideo/entrypoints/streaming/server.py fastvideo/tests/entrypoints/streaming/test_server.py docs/design/server_contracts/streaming.md

Test Results

  • Four new tests in TestSessionEnd (test_server.py) cover the segment cap, the idle timeout, the opening-frame timeout and an unexpected failure. Each runs under a 10 s SIGALRM deadline, so a missing frame fails the test instead of hanging the unit lane. On main all four fail: three get ltx2_stream_complete stop_requested instead of the expected frame, and the opening-frame one fails with TimeoutError: no server frame within 10 s.
  • fastvideo/tests/entrypoints/streaming/: 176 passed (172 on main), CPU only, macOS, Python 3.12, Starlette 1.10. The fMP4 tests ran with ffmpeg 9.0.
  • pre-commit on the changed files: yapf, ruff, codespell, mypy and PyMarkdown pass.
Real-connection output
main
{"case": "segment_cap", "received": [..., "<segment 1: [... 'ltx2_segment_complete']>", {"type": "ltx2_stream_complete", "reason": "stop_requested"}], "close": {"code": 1006, "reason": ""}}
{"case": "idle_timeout", "received": [{"type": "queue_status"}, {"type": "gpu_assigned"}, {"type": "ltx2_stream_start"}, {"type": "ltx2_stream_complete", "reason": "stop_requested"}], "close": {"code": 1006, "reason": ""}}
{"case": "init_timeout", "received": [], "close": {"code": 1006, "reason": ""}}

this PR
{"case": "segment_cap", "received": [..., "<segment 1: [... 'ltx2_segment_complete']>", {"type": "ltx2_stream_complete", "reason": "segment_cap"}], "close": {"code": 1000, "reason": "segment_cap"}}
{"case": "idle_timeout", "received": [{"type": "queue_status"}, {"type": "gpu_assigned"}, {"type": "ltx2_stream_start"}, {"type": "session_timeout", "timeout_seconds": 2}], "close": {"code": 1000, "reason": "session_timeout"}}
{"case": "init_timeout", "received": [{"type": "session_timeout", "timeout_seconds": 2}], "close": {"code": 1000, "reason": "session_timeout"}}

#2073 makes malformed client frames return invalid_message instead of ending the session. Both PRs touch _run_segment_loop, so whichever merges second needs a small rebase. I will do it.

Checklist

  • I ran pre-commit run --all-files and fixed all issues (ran on the changed files)
  • I added or updated tests for my changes
  • I updated documentation if needed
  • I considered GPU memory impact of my changes (none)

@mergify

mergify Bot commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

Merge Protections

🟢 Merge protection satisfied — ready to merge.

Show 1 satisfied protection

🟢 PR merge requirements

  • #approved-reviews-by>=1
  • check-success=fastcheck-passed
  • check-success=full-suite-passed
  • check-success~=pre-commit
  • title~=(?i)^\[(feat|feature|bugfix|fix|refactor|perf|ci|doc|docs|misc|chore|kernel|new.?model|skill|skills|infra)\]

@mergify mergify Bot added type: bugfix Bug fix scope: inference Inference pipeline, serving, CLI scope: infra CI, tests, Docker, build scope: docs Documentation labels Nov 5, 2026
@fv-superagent
fv-superagent requested a review from aryan5v November 5, 2026 16:06
@greptile-apps

greptile-apps Bot commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] The PR appears safe to merge, with no outstanding findings.

Summary

The server now tells clients why a session ended and sends the matching WebSocket close code.

  • Streaming sessions now end with a frame that explains why they stopped.

The latest documentation change correctly removes the claim that a client disconnect moves the session to COMPLETE. Both earlier comments were addressed: the tests now have deadlines, and the flagged code comments were removed. No new issues found.

Reviews (3) · Last reviewed commit: "[docs]: correct the streaming example-fl..." · Reviewed by Greptile

@fv-superagent fv-superagent left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fv-superagent score: 3/5

Thanks — the segment-cap and idle-timeout paths now match the contract, ACTIVE -> COMPLETE is a legal transition, and the four new tests pin the frames and close codes. One must-fix and two doc nits.

Must fix

  1. fastvideo/entrypoints/streaming/server.py:247 — the unexpected-failure path sends ltx2_stream_complete(reason="error") + close 1011 but never the error frame. The client this PR is written for reads ltx2_stream_complete as a normal finish and only error as a session error, and ignores reason (apps/dreamverse/web/src/lib/ws/protocol.ts:30-31, apps/dreamverse/web/src/lib/ws/reducer.ts:470-491). A server failure therefore still looks like a clean end — the same misread as the timeout case. docs/design/server_contracts/streaming.md:98 also says fatal errors precede a close with an error (code, message) frame. Please send ErrorMessage(code="internal_error", ...) before closing; whether you keep the ltx2_stream_complete there too is your call.

While you're there (docs)

  1. docs/design/server_contracts/streaming.md:95 — stop_requested is documented for "client requested clean shutdown", but nothing can emit it now: ClientMessage has no stop message, disconnected deliberately sends nothing, and the only constructions are error/segment_cap. Drop it from the row and from protocol.py:170's Literal/default, or mark it reserved.
  2. docs/design/server_contracts/streaming.md:44 — the lifecycle diagram still routes segment_cap to TIMEOUT; server.py:250 now transitions to COMPLETE, matching the prose below. Line 96 also says session_timeout carries reason, but the frame carries timeout_seconds (protocol.py:173) and now closes 1000/session_timeout.

— fv-superagent, automated maintainer run by @aryan5v

@stayinalive181

Copy link
Copy Markdown
Contributor Author

Thanks. Addressed in 3f6a9dd:

  1. Unexpected failure: the server now sends error internal_error (retryable: false) and closes with 1011 internal_error. I dropped the ltx2_stream_complete there. The DreamVerse reducer clears sessionNotice on stream/completed, which would wipe the error it had just shown.
  2. stop_requested: streaming.md now marks it reserved, together with error. I left the Literal and default in protocol.py unchanged, so the server-side model keeps its shape in this PR.
  3. The lifecycle diagram now puts segment_cap under COMPLETE. The session_timeout row gives timeout_seconds and close 1000 session_timeout, and the error row lists retryable.

Greptile's notes: I removed the _SessionEnd comment and the one next to range(2). The four TestSessionEnd tests now run under a 10 s SIGALRM deadline, so on main the opening-frame test fails with TimeoutError instead of hanging. I used a signal rather than a TestClient internal because fastapi>=0.129 still allows Starlette 0.40, whose websocket test session has different internals.

@fv-superagent fv-superagent left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fv-superagent score: 4/5 (was 3/5)

The must-fix is fixed: an unexpected loop failure now sends ErrorMessage(code="internal_error", retryable=False) and closes 1011 internal_error, and dropping the ltx2_stream_complete there was right — the client clears the notice on stream/completed. The doc rows and lifecycle diagram match the code, and the _deadline SIGALRM guard is sound. Nothing blocking; one leftover:

  • docs/design/server_contracts/streaming.md:163 — the example-flow footer still ends (close) → (session → COMPLETE), but a client close returns "disconnected" with no frame and no transition (server.py:353; _handle_session has no branch for it). Row 95 now says only segment_cap is emitted, so this is the last place still claiming a clean leave reaches COMPLETE.

Two optional follow-ups in the same area: streaming.md:54-55 says non-terminal sessions transition to TIMEOUT, but the opening-frame timeout runs while the session is INITIALIZING, which has no INITIALIZING -> TIMEOUT edge, so the transition at server.py:276 is a swallowed no-op (low impact — the session is dropped right after). And server.py:246 wraps only _run_segment_loop, so a fatal exception earlier in _handle_session still reaches _stream's catch-all without an error frame; worth a follow-up if you want streaming.md:98 to hold universally.

— fv-superagent · run by @aryan5v

@mergify mergify Bot added the needs-rebase PR has merge conflicts label Nov 5, 2026

@fv-superagent fv-superagent left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fv-superagent score: 5/5 — the doc nit is fixed in fdfa12b; all checks green, Greptile 5/5.

@fv-superagent

fv-superagent commented Nov 5, 2026 •

Copy link
Copy Markdown
Collaborator

Heads up @stayinalive181: please rebase on main — #2073 landed and this PR now conflicts in _run_segment_loop. Everything else is green and approved.

— fv-superagent · run by @aryan5v

@github-actions github-actions Bot added the ready PR is ready to merge label Nov 5, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs-rebase PR has merge conflicts ready PR is ready to merge scope: docs Documentation scope: inference Inference pipeline, serving, CLI scope: infra CI, tests, Docker, build type: bugfix Bug fix

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants