Skip to content

spec: document optimistic multistream-select (issue #643) #721

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Devguru-codes wants to merge 3 commits into
base: master
Choose a base branch
from
Open

spec: document optimistic multistream-select (issue #643) #721

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
f5eba72
spec: document optimistic multistream-select (issue #643)
Devguru-codes May 22, 2026
548f837
address review: apply all feedback from MarcoPolo
Devguru-codes Jun 16, 2026
7f2227e
addressing remaining comments
Devguru-codes Jun 16, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions connections/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ and spec status.
- [Definitions](#definitions)
- [Protocol Negotiation](#protocol-negotiation)
- [multistream-select](#multistream-select)
- [Optimistic Protocol Negotiation](#optimistic-protocol-negotiation)
- [Upgrading Connections](#upgrading-connections)
- [Opening New Streams Over a Connection](#opening-new-streams-over-a-connection)
- [Practical Considerations](#practical-considerations)
Expand Down Expand Up @@ -183,6 +184,15 @@ traffic over the channel will adhere to the rules of the agreed-upon protocol.
If a peer receives a `"na"` response to a proposed protocol id, they can either
try again with a different protocol id or close the channel.

### Optimistic Protocol Negotiation

When the dialer has prior knowledge that the responder supports a given
protocol (e.g., via the [identify protocol][identify/push]), it can use an
optimistic variant of multistream-select that saves one round trip by sending
the protocol proposal and application data without waiting for the responder's
echo. For details, see the [Optimistic Protocol Negotiation][optimistic-ms]
specification.


## Upgrading Connections

Expand Down Expand Up @@ -427,3 +437,4 @@ updated to incorporate the changes.
[resource-manager-issue]: https://github.com/libp2p/go-libp2p/issues/635
[hole-punching]: ./hole-punching.md
[inlined-muxer-selection]: ./inlined-muxer-negotiation.md
[optimistic-ms]: ./optimistic-multistream.md
307 changes: 307 additions & 0 deletions connections/optimistic-multistream.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,307 @@
# Optimistic Protocol Negotiation <!-- omit in toc -->

| Lifecycle Stage | Maturity | Status | Latest Revision |
|-----------------|---------------|--------|-----------------|
| 1A | Working Draft | Active | r0, 2026-05-22 |

Authors: [@Devguru-codes]

Interest Group: [@marcopolo], [@marten-seemann]

[@Devguru-codes]: https://github.com/Devguru-codes
[@marcopolo]: https://github.com/MarcoPolo
[@marten-seemann]: https://github.com/marten-seemann

See the [lifecycle document][lifecycle-spec] for context about the maturity level
and spec status.

[lifecycle-spec]: https://github.com/libp2p/specs/blob/master/00-framework-01-spec-lifecycle.md

## Table of Contents <!-- omit in toc -->

- [Overview](#overview)
- [Applicability](#applicability)
- [Wire Format](#wire-format)
- [Standard Multistream-Select](#standard-multistream-select)
- [Optimistic Multistream-Select](#optimistic-multistream-select)
- [Prerequisites](#prerequisites)
- [Requirements](#requirements)
- [Dialer (Initiator) Requirements](#dialer-initiator-requirements)
- [Listener (Responder) Requirements](#listener-responder-requirements)
- [Known Limitations](#known-limitations)
- [Protocol Confusion on Negotiation Failure](#protocol-confusion-on-negotiation-failure)
- [Security Considerations](#security-considerations)
- [Interaction with Inlined Muxer Negotiation](#interaction-with-inlined-muxer-negotiation)
- [Implementation References](#implementation-references)

## Overview

Also known as "lazy multistream-select" or "lazy negotiation".

In standard [multistream-select][mss] negotiation, the dialer (initiator) sends
its protocol proposal and **waits** for the listener (responder) to echo the
protocol ID back before sending any application data. This costs two full

@MarcoPolo MarcoPolo Jun 15, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just one round trip:

A -> <multistream-header> <protocol> -> B
A <- <multistream-header> <protocol> <- B 
A -> Application data

@Devguru-codes Devguru-codes Jun 16, 2026

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was my main doubt. I have now updated it to 1 RTT count

round trips before application data can flow.

Optimistic protocol negotiation eliminates one round trip by allowing the dialer
to send the multistream-select header, the protocol proposal, **and** the initial
application data all at once, without waiting for the listener's echo response.
The listener's echo response arrives asynchronously while the dialer is already
sending application data.

This optimization is critical for latency-sensitive use cases such as
[Kademlia DHT][kad-dht] operations, which use a "one stream per RPC" pattern
and would otherwise pay the full two-round-trip cost on every request.

Both [go-libp2p] and [rust-libp2p] use optimistic negotiation in production,
but it has not previously been documented in the libp2p specifications. This
document formalizes the optimization and documents the requirements
implementations must follow to use it correctly.

@MarcoPolo MarcoPolo Jun 15, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Both [go-libp2p] and [rust-libp2p] use optimistic negotiation in production,
but it has not previously been documented in the libp2p specifications. This
document formalizes the optimization and documents the requirements
implementations must follow to use it correctly.

## Applicability

Optimistic protocol negotiation can be applied in two contexts:

1. **Stream-level negotiation** (primary use case): When opening a new stream
over an existing connection, the dialer can optimistically propose a protocol
and begin sending application data immediately. This is the most common use
case and provides the greatest latency benefit for protocols that use
short-lived streams (e.g., DHT RPCs, Bitswap requests).

2. **Connection upgrade negotiation**: When negotiating security or stream
multiplexing protocols during [connection establishment][connections]. This
use case is less common because [inlined muxer negotiation][inlined-muxer]
already reduces the round-trip cost of the connection upgrade process.

## Wire Format

The wire format for individual messages is unchanged from standard
[multistream-select][mss]. Messages are UTF-8 strings, newline-terminated, and
prefixed with their length as an [unsigned varint][uvarint]. The difference is
purely in the **sequencing** of messages.

@MarcoPolo MarcoPolo Jun 15, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
[multistream-select][mss]. Messages are UTF-8 strings, newline-terminated, and
prefixed with their length as an [unsigned varint][uvarint]. The difference is
purely in the **sequencing** of messages.
[multistream-select][mss].

### Standard Multistream-Select

In the standard (non-optimistic) flow, the dialer waits for the listener's echo
before sending application data:

```
Dialer Listener
| |
|--- /multistream/1.0.0 ---------------->|
|<-- /multistream/1.0.0 -----------------|

@MarcoPolo MarcoPolo Jun 15, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dialer does not have have to wait to hear back /multistream/1.0.0

@Devguru-codes Devguru-codes Jun 16, 2026

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated it.

| |
|--- /my-protocol/1.0.0 --------------->|
|<-- /my-protocol/1.0.0 ---- (echo) ----| <- Dialer waits for echo
| |
|--- [application data] --------------->| <- Only then sends data
| |
```

**Cost**: 2 round trips before application data flows.

### Optimistic Multistream-Select

In the optimistic flow, the dialer sends the multistream header, protocol
proposal, and application data without waiting:

```
Dialer Listener
| |
|--- /multistream/1.0.0 ---------------->|
|--- /my-protocol/1.0.0 ---------------->| <- No wait for header echo
|--- [application data] ---------------->| <- Sent immediately
| |
|<-- /multistream/1.0.0 -----------------| <- Echo arrives later
|<-- /my-protocol/1.0.0 -----------------|
| |
```

**Cost**: 1 round trip saved. Application data is delivered to the listener
alongside the protocol proposal. The listener processes the negotiation and then
delivers the application data to the protocol handler.

## Prerequisites

Optimistic protocol negotiation is inherently **best-effort**. The dialer sends

@MarcoPolo MarcoPolo Jun 15, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Optimistic protocol negotiation is inherently **best-effort**. The dialer sends
Optimistic protocol negotiation is inherently optimistic. The dialer sends

@Devguru-codes Devguru-codes Jun 16, 2026

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated this

application data before receiving confirmation that the listener supports the
requested protocol. If the listener does not support the protocol, the
negotiation will fail, and the application data sent optimistically will be
wasted.

To minimize the risk of failed negotiations, dialers typically use the
[identify protocol][identify] to learn which protocols a peer supports before
using optimistic negotiation. The `protocols` field of the identify message
(see [identify spec][identify]) contains the list of protocols the remote peer
supports.

However, it is important to note that this is still *optimistic* — a peer's
supported protocols can change dynamically at any time (e.g., via
[identify/push][identify-push] updates, or due to configuration changes).
In practice, protocol support is stable enough that optimistic negotiation
succeeds in the vast majority of cases.

Implementations SHOULD NOT use optimistic negotiation on the **first** stream
to a peer when no prior protocol knowledge is available.

@MarcoPolo MarcoPolo Jun 15, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Implementations SHOULD NOT use optimistic negotiation on the **first** stream
to a peer when no prior protocol knowledge is available.
Implementations MAY skip optimistic negotiation when no prior protocol knowledge is available.

@Devguru-codes Devguru-codes Jun 16, 2026

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated this


## Requirements

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD",
"SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be
interpreted as described in [RFC 2119].

### Dialer (Initiator) Requirements

1. **Ordering**: Implementations MUST send the multistream protocol ID
(`/multistream/1.0.0`) and the application protocol ID before any
application data on the stream, without waiting for responses between
them.

2. **Handshake completion on close**: Implementations SHOULD finish reading the
handshake response before closing the stream. If the dialer writes
application data and then closes the stream before the listener has echoed
back the protocol ID, the listener's echo write may fail on a closed stream,
potentially causing the listener to reset the stream and discard data that
was already received.

If the application does not need to read any data from the stream (i.e., it
is a pure write/fire-and-forget pattern), the implementation MAY skip reading
the handshake response.

3. **Prior knowledge**: Implementations SHOULD NOT use optimistic negotiation

@MarcoPolo MarcoPolo Jun 15, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SHOULD NOT is too strong I think. This can be OPTIONAL or MAY

@Devguru-codes Devguru-codes Jun 16, 2026

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I updated it now

without prior knowledge that the peer supports the requested protocol (e.g.,
via a preceding [identify][identify] exchange). Using optimistic negotiation
without prior knowledge risks triggering the protocol confusion issue
described in [Known Limitations](#known-limitations).

### Listener (Responder) Requirements

1. **Tolerating echo write failures**: Implementations SHOULD ignore write
errors when echoing back the protocol ID during negotiation. With optimistic
negotiation, the dialer may have already closed its write side of the stream
after sending application data. The echo write failing does not indicate a
negotiation failure — the negotiated protocol has already been identified.

2. **Stream delivery**: Implementations MUST still deliver the stream to the
application protocol handler even if the echo write fails. The handler can
then `Read()` any application data the dialer already sent.

3. **Data boundary**: After a protocol has been successfully matched during
negotiation, implementations MUST NOT interpret subsequent bytes on the
stream as multistream-select messages. All bytes after the negotiated
protocol ID are application data and MUST be forwarded to the protocol
handler.
Comment thread
Devguru-codes marked this conversation as resolved.
Outdated

## Known Limitations

### Protocol Confusion on Negotiation Failure

There is a known soundness issue with optimistic multistream-select that arises
when the negotiation **fails** — i.e., the listener does not support the
proposed protocol.

Consider the following scenario:

- Peer A supports `protocolA`
- Peer B supports `protocolB` (but **not** `protocolA`)

Peer A sends optimistically:

```
<len>/multistream/1.0.0\n <len>/protocolA/1.0.0\n
```

Then immediately sends application data that happens to look like a valid
multistream-select protocol proposal:

```
<len>/protocolB/1.0.0\n [other data]
```

Peer B does not support `protocolA`, so it responds with `"na"` and reads the
next message. It sees `/protocolB/1.0.0` — which it **does** support — and
interprets it as a new protocol proposal. It echoes back `/protocolB/1.0.0` and
starts interpreting `[other data]` as `protocolB` traffic.

**Result**: Peer B is now speaking `protocolB` with data that was actually
`protocolA` application data. This is a protocol confusion vulnerability.

This issue is extensively documented in [go-multistream#20]. As stated by the
original author:

> "I don't know of a good fix that won't break everything (without upping
> the version and adding a mandatory round trip)."

**Mitigations**:

- The prerequisite of having prior protocol knowledge (via identify) makes this
scenario unlikely in practice, since the dialer would not propose a protocol
the listener does not support.
- Application protocol designers SHOULD ensure that their wire format cannot be
confused with multistream-select messages (i.e., application data should not
begin with a valid varint-prefixed, newline-terminated protocol path).
- Implementations SHOULD log or track negotiation failures to detect potential
protocol confusion.

## Security Considerations

@MarcoPolo MarcoPolo Jun 15, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This section doesn't add value. Consider cutting it.

@Devguru-codes Devguru-codes Jun 16, 2026

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have also removed the security considerations section here. If you want me to keep - i will re-write it again. Thank you.


The protocol confusion issue described in [Known Limitations](#known-limitations)
represents a potential security concern. If a malicious or misconfigured peer
provides incorrect information about which protocols it supports (e.g., via
identify responses), the dialer may optimistically propose a protocol the peer
does not actually support, triggering the protocol confusion scenario.

Implementations SHOULD:

- Validate identify responses and consider them advisory, not authoritative.
- Track and limit the rate of negotiation failures per peer. A high failure rate
may indicate a misconfigured peer or an attack.
- Ensure that protocol handlers validate incoming data before processing it,
regardless of how the stream was negotiated.

The security impact is limited by the fact that libp2p connections are
authenticated and encrypted (via [Noise][noise] or [TLS 1.3][tls]). An attacker
cannot inject data into an existing connection. The protocol confusion risk only
applies within an already-authenticated peer relationship, where the remote peer
provides incorrect identify information.

## Interaction with Inlined Muxer Negotiation

@MarcoPolo MarcoPolo Jun 15, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This section should simply say Inlined Muxer Negotiation is preferred.

@Devguru-codes Devguru-codes Jun 16, 2026

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.


For connection upgrades (as opposed to stream-level negotiation), the latency
benefit of optimistic negotiation is reduced by [inlined muxer negotiation][inlined-muxer],
which moves the stream multiplexer selection into the security handshake. When
inlined muxer negotiation is in use, the connection upgrade already avoids the
extra round trip for muxer selection, so the additional savings from optimistic
negotiation during the connection upgrade are minimal.

Optimistic negotiation during connection upgrades is therefore most useful when
inlined muxer negotiation is **not** available (e.g., when connecting to older
peers that do not support the optimization).

## Implementation References

The following implementation artifacts motivated and informed this specification:

| Reference | Description |
|-----------|-------------|
| [go-multistream#115] | Fix: finish reading handshake on `lazyConn` close |
| [go-multistream#87] | Fix: ignore error if can't write back echoed protocol |
| [go-multistream#20] | Issue: lazy negotiation soundness problem (protocol confusion) |
| [go-libp2p#3038] | Bug: WebTransport `StopSending` error caused by incomplete lazy handshake |

[mss]: https://github.com/multiformats/multistream-select
[uvarint]: https://github.com/multiformats/unsigned-varint
[connections]: https://github.com/libp2p/specs/tree/master/connections
[inlined-muxer]: ./inlined-muxer-negotiation.md
[identify]: ../identify/README.md
[identify-push]: ../identify/README.md#identifypush
[kad-dht]: ../kad-dht/README.md
[noise]: ../noise/README.md
[tls]: ../tls/tls.md
[go-libp2p]: https://github.com/libp2p/go-libp2p
[rust-libp2p]: https://github.com/libp2p/rust-libp2p
[go-multistream#115]: https://github.com/multiformats/go-multistream/pull/115
[go-multistream#87]: https://github.com/multiformats/go-multistream/pull/87
[go-multistream#20]: https://github.com/multiformats/go-multistream/issues/20
[go-libp2p#3038]: https://github.com/libp2p/go-libp2p/issues/3038
[RFC 2119]: https://www.ietf.org/rfc/rfc2119.txt