connect-node: RST_STREAM(NO_ERROR) mid-response yields misleading incomplete envelope / missing status instead of a transport error #1751

Describe the bug

When a proxy closes a gRPC stream with RST_STREAM code=0 (NO_ERROR) while the response tail (final DATA and/or trailers) has not yet reached the client — e.g., Envoy resetting right after the upstream completes, while downstream bytes are still flow-control-blocked (envoyproxy/envoy#30149, envoyproxy/envoy#38206; grpc/grpc-node#2569) — Node's http2 client (pre nodejs/node#63249) surfaces this as a clean, error-free end-of-stream with rstCode 0. connect-node's h2Request() close handler correctly maps NO_ERROR to "no error" (connectErrorFromH2ResetCode), so the truncation goes unnoticed and the protocol layer then fails with:

ConnectError: [invalid_argument] protocol error: incomplete envelope   // cut mid-message
ConnectError: [internal] protocol error: missing status               // cut before trailers

Neither error names the stream reset, both point at the server's wire format, and invalid_argument reads as non-retryable — a transport-level abort is misreported as a server protocol violation. RST_STREAM(NO_ERROR) after a complete response is explicitly legal (RFC 9113 §8.1 — which also states "Clients MUST NOT discard responses as a result of receiving such a RST_STREAM"), so any connect-node client behind Envoy/Istio can hit this under page-load-style concurrency.

Note: when the complete response including trailers does reach the client before the RST, Node 24 preserves buffered data and the call succeeds — verified experimentally. The failing case is specifically the reset overtaking the response tail.

Environment

• @connectrpc/connect-node 2.1.2, @connectrpc/connect 2.1.2 (current latest); createGrpcTransport, unary calls
• Node.js 24.11.1, macOS (darwin/arm64)
• Default session options (no nodeOptions, default 65535-byte windows), one shared connection per process, up to ~12 concurrent streams in our traces; interceptors present (auth header, tracing, logging — none consume or transform bodies); no retry layer
• Server path: Istio/Envoy ingress → Go (grpc-go) services

Evidence

Observed in a production-like environment and bracketed from both ends:

• Server side (Envoy edge debug logs, per stream): the upstream completed the response; Envoy's codec accepted headers, data, and trailers and then reset the stream with code=0 (doEndStream() path) while request-body bytes were still flow-control-outstanding (65535-byte stream window).
• Client side (instrumented Connect interceptor, 799 calls): the same streams failed with the two errors above — 34 of 95 calls (~36%) for the largest/slowest unary response (median 654 ms), 13 of them mid-message; near zero for small trailers-only responses; sequential traffic never fails.
• Control: grpc-go, grpcurl (1000/1000 at 100-way parallelism), and curl ran the identical wire pattern against the same gateway with zero failures (grpc-go's large/BDP-tuned windows leave no undelivered tail for the reset to overtake).
• Proposed fix trialed as a local patch: running the trailers-tracking fix (the companion PR) as a temporary pnpm patch in the same environment, the identical live failures surfaced as [unavailable] http/2 stream closed with error code NO_ERROR (0x0) before trailers were received — client-side confirmation that the failing streams close with rstCode 0 without trailers ever arriving. In our experiments the graceful END_STREAM-without-trailers shape deterministically still yields missing status, so this signature specifically indicates the reset shape.
• We have not yet captured client-side frames for a failing stream (planned: NODE_DEBUG=http2/pcap); attribution of the drop to the proxy's downstream leg follows from the verified Node behavior above plus the Envoy issue family cited.

Reproduction (verified on Node 24.11.1)

Node http2 server; client keeps the request open (request body > 64 KiB that the server never reads, so flow control blocks it). The large unread body is the repro's mechanism for holding the stream in the reset-prone state; our Envoy logs show the same precondition (request bytes flow-control-outstanding at reset time). Server: write DATA, then stream.close(NGHTTP2_NO_ERROR) either mid-DATA or after all DATA but before trailers. (When reproducing with a Node http2 server and a fully-sent request instead, respond() must pass { waitForTrailers: true } — otherwise Node closes gracefully with END_STREAM and no RST reaches the wire; the companion PR's spec tests use this to reproduce through the transport.) The client observes a clean end with rstCode 0 and no error event; through createGrpcTransport this yields exactly incomplete envelope / missing status. Sending the RST after sendTrailers() does not reproduce — data and trailers survive — which is what points the data loss at the proxy hop rather than Node.

// Variant 3: the RST(NO_ERROR) lands while the response is INCOMPLETE from the
// client's perspective (mid-DATA, or after DATA but before trailers).
// Expectation per pre-#63249 Node: readable ends CLEANLY, no 'error' event,
// rstCode = 0 -> silent truncation. connect-node would then report
// "incomplete envelope" (mid-message cut) or "missing status" (no trailers).
const http2 = require('node:http2');

const mode = process.argv[2]; // "midData" | "beforeTrailers"
const MB = 1024 * 1024;
const log = (...a) => console.log(...a);

const server = http2.createServer();
server.on('stream', (stream) => {
  stream.respond({ ':status': 200, 'content-type': 'application/grpc' });
  if (mode === 'midData') {
    stream.write(Buffer.alloc(300 * 1024, 0x61), () => {
      stream.close(http2.constants.NGHTTP2_NO_ERROR); // cut mid-message, no trailers, no END_STREAM
    });
  } else {
    // whole body flushed, then RST instead of trailers
    stream.write(Buffer.alloc(1 * MB, 0x61), () => {
      stream.close(http2.constants.NGHTTP2_NO_ERROR);
    });
  }
});

server.listen(0, () => {
  const port = server.address().port;
  const session = http2.connect(`http://127.0.0.1:${port}`, {
    settings: { initialWindowSize: 4 * MB },
  });
  const req = session.request({ ':path': '/x', ':method': 'POST' }, { endStream: false });
  let received = 0, sawTrailers = null, sawEnd = false, sawError = null;
  req.on('response', () => {});
  req.on('trailers', (t) => { sawTrailers = t; });
  req.on('error', (e) => { sawError = `${e.code}: ${e.message}`; });
  req.on('data', (c) => { received += c.length; });
  req.on('end', () => { sawEnd = true; });
  req.on('close', () => {
    log(`mode=${mode} received=${received} end=${sawEnd} trailers=${!!sawTrailers} error=${sawError} rstCode=${req.rstCode}`);
    log(sawError ? 'RESULT: surfaced as ERROR' : 'RESULT: SILENT clean EOF (truncation invisible to consumer)');
    process.exit(0);
  });
  req.write(Buffer.alloc(256 * 1024, 0x62)); // request kept in flight
  setTimeout(() => { log('timeout'); process.exit(1); }, 10000);
});

Expected behavior

A stream that closes with rstCode 0 before the response completed is a transport-level abort, not a server wire-format violation. Concretely: extend #1747's approach past response headers — track response completeness, and when the stream closes NO_ERROR with the response started but incomplete, reject with Code.Unavailable (or Aborted) and a message naming the reset instead of surfacing invalid_argument/internal protocol errors. This mirrors grpc-js, which reports this exact case as Received RST_STREAM with code 0 (Call ended without gRPC status) and ignores NO_ERROR resets only when the status already arrived. Secondarily: consider exposing http2 window settings (connect-node currently runs Node defaults, 65535) — downstream flow-control backpressure under concurrency is what widens the race; grpc-js 1.14.3 shipped an adjacent mitigation (prompt half-close) for the same Envoy interop.

A companion PR implementing this accompanies the issue.

Additional context — references

connect-es, active work in this code region:

• Reject unary calls when the http/2 stream closes with NO_ERROR before the response #1747 / Unary Promise never settles — sentinel.error never called after premature stream termination #1678 — the complementary case: RST(NO_ERROR) before the response (hang → reject). This report is the post-headers continuation of the same gap in h2Request()'s close handler.
• Fix Http2SessionManager.verify() never settling when the connection closes during verification #1746 — session verify() never settling on connection close during PING; Add connectTimeoutMs option to bound connection establishment #1748 — unbounded connection establishment. Same subsystem, distinct failures.
• [connect-node] Transport does not recover from server restarts - connections remain in zombie state #1561 — zombie connections after server restart: related family, but distinct — that is failure to reconnect; this is misreported truncation on a live connection.

Upstream and spec:

• Fix HTTP/2 RST_STREAM behaviour, add auto-drain, deprecate 'aborted', fix related compat API issues nodejs/node#63249 — merged 2026-06-01 (semver-major, in no released Node): makes a non-error RST before readable completion an explicit 'error'; its text confirms current behavior: "we currently truncate cancelled responses with no warning whatsoever". (http2: closing an HTTP2Stream on the other side causes data to be lost nodejs/node#16880 was the 2017 original, closed via http2: refactor close/destroy for Http2Stream and Http2Session nodejs/node#17406; the silent-truncation flavor persisted until #63249, tracked in http2: confusion with how aborted ClientHttp2Stream is reported nodejs/node#56627.)
• @grpc/grpc-js throw 'Received RST_STREAM with code 0' with retry enabled grpc/grpc-node#2569 (closed) — same family behind Istio/Envoy; resolved via a half-close ordering fix in grpc-js 1.14.3 plus the Envoy issues below.
• RST_STREAM sent to downstream client instead of remaining DATA, HEADERS on end stream envoyproxy/envoy#30149 (closed) — "RST_STREAM sent to downstream client instead of remaining DATA, HEADERS on end stream"; gRPC server issues with envoy proxy on RST_STREAM envoyproxy/envoy#38206 (closed).
• RFC 9113 §8.1 — legality of post-response RST_STREAM(NO_ERROR), and "Clients MUST NOT discard responses as a result of receiving such a RST_STREAM."