Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
134 changes: 134 additions & 0 deletions docs/messaging/raw/chat-framework.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,134 @@
# CHAT-FRAMEWORK

| Field | Value |
| --- | --- |
| Name | A modular framework for defining chat protocols |
| Status | raw |
| Type | RFC |
| Category | Standards Track |
| Tags | chat |
| Editor | Jazz Alyxzander <jazz@status.im> |
Comment on lines +1 to +10

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the previous repo the folder structure helped to separate:

  • Standards track vs Informational
  • Application (Chat) vs Core(Delivery)

WIthout these its harder to navigate.

[Sand] Consider separating Delivery and Chat. The specifications are mutually exclusive; combining only adds confusion.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fbarbu15 what do you think about this?

The current folder structure is per-team (e.g. there's anoncomms, messaging and research), but it doesn't make much sense for an external reader. Maybe indeed we should rather make dir-structure follow logical areas of application?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(I'll keep it as is in this PR anyway though, this is a separate discussion)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@igor-sirotin I'm ok with that, blockchain team asked for a similar folder structure like they had in notion and it looks like this now in LIPS
image

So if you guys need to categorize specs in any other way feel free to do it or lmk and I can look into it.


# Abstract

This specification defines a modular communication protocol framework for describing chat protocols. It introduces abstraction boundaries, and a component model for describing chat protocol functionality.


# Background / Rationale / Motivation

Chat protocols specifications can be long and dense documents. To fully describe a chat protocol there are many layers and operations which are required to be documented. This includes payloads, message transport, encryption as well as user level features such as account registration, typing indicators, content formatting.

With the vast amount of information required to maintain compatibility between applications - protocol documentation is either comprehensive which leads to large monolithic specifications or lacking the required details for interop between implementors. A suitable solution would provide both the specificity while also remaining lean and focused.

# Theory / Semantics

This specification defines an abstract framework for building a chat protocol. Its purpose is to name the distinct components/phases, and define modular boundaries between them to promote reuse. The end result is that a chat protocol implementation can be described by listing its approach to 5 things.

The lifecycle of a protocol instance is divided into three phases:

- **Discovery:** How does a Sender learn of other clients.
- **Initialization:** How does a Recipient learn a client wants to communicate with them.
- **Operation:** How do participants exchange content.

and transport details are divided into 2 components:

- **Delivery Service:** How are payloads routed and delivered to a client.
- **Framing Strategy:** How are payloads encoded.

Defining these 5 parameters allows for chat protocol implementations to be fully defined, which allows clients from different applications to exchange messages.
While also dividing documents by their focus area.

## Abstract Transport

### Delivery service

A Delivery Service (DS) is the service or method that distributes payloads to clients. A DS accepts payloads with a delivery_address and delivers them to all subscribers of that delivery_address. Protocols use delivery_addresses to establish delivery contracts between senders and recipients. The mapping of delivery_addresses to DS-level concepts is implementation-specific.

#### Requirements

- A DS MUST provide a method for clients to subscribe to messages from a delivery_address
- Payloads sent to a delivery_address are delivered by a DS to all subscribers of that delivery_address
- A DS SHOULD handle segmentation if the underling transport limits message sizes
- A DS MAY NOT guarantee message delivery
- A DS MAY NOT guarantee message ordering
- A DS MAY reject payloads


### Framing Strategy

In this protocol framework, payloads from multiple protocols are potentially multiplexed over the same channel. This requires that clients are able to associate a given payload to a given instance of a protocol.

A framing strategy should define a common payload type as well as a method to determine which state machine a receiving client must use to decode it.


## Protocol Phases

In order to exchange content clients must be able to learn of each others existence, gather the pre-requisite information/parameters and, remain synchronized over time.

The lifecycle of a protocol instance is divided into three phases, which are described by a corresponding protocol.

- **Discovery Phase:** Discovery Protocol
- **Initialization Phase:** Initialization Protocol
- **Operation Phase:** Conversation Protocol

```mermaid
sequenceDiagram
participant D as ???
participant S as Saro
participant R as Raya


Note over D,S: Discovery
D -->> S: <IntroBundle>

Note over R,S: Initialization
S ->> R: Invite

Note over R,S: Operation

loop
par
R->> S: Send Message
and
S->> R: Send Message
end
end
```



### Discovery Protocol

A discovery protocol defines how clients gather the prerequisite information to contact another client.

The input requirements of the discovery protocol are not defined here, and largely determined by the desired user experience.

The output requirements of the discovery protocol are very implementation specific, and will depend on the initialization protocol requirements and DS chosen by the implementation. The data provided by the discovery protocol and required by the initialization protocol is called the `IntroductionBundle`.

- The discovery protocol MUST provide all data required for the initialization protocol.

Note: There is no requirement that the Discovery protocol be neither a complicated nor interactive process. Hypothetically If all required values to construct a IntroductionBundle could be statically defined, that would be sufficient for this definition.

### Initialization Protocol

A initialization protocol specifies how two clients can initiate communication. The input to this process is the `IntroductionBundle` and the output is an established instance of a `Conversation` between the participants.

The core of an initialization protocol is a defined location and procedure for receiving initial messages.

Many chat protocols choose to define the initialization protocol within the conversation protocol. This tight coupling produces two negative artifacts.
- New conversation protocols must define and deploy there own initialization channels. Increasing overhead and adding complexity.
- Protocol upgrades then create partitions in the communication network, as older clients have no means of communicating with new clients.

Separating channel initialization from conversation flow allows multiple conversations to reuse the same initialization channel. This reduces effort for new conversation protocols, and is especially valuable when upgrading existing ones. Being independent the initialization pathway can persist across conversation versions. Even if an older client cannot parse new message types, it can still recognize their presence, adding observability.

### Conversation Protocol

A conversation protocol defines how messages flow between participants, and subsequently determines the properties of that channel.

- A Conversation protocol MUST define the payloads it uses and how to handle them.
- A Conversation protocol SHOULD outline the cryptographic properties provided
- A Conversation protocol SHOULD describe bidirectional communication.

## Copyright

Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
180 changes: 180 additions & 0 deletions docs/messaging/raw/contentframe.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,180 @@
# CONTENTFRAME

| Field | Value |
| --- | --- |
| Name | Chat Content Frames |
| Status | raw |
| Type | RFC |
| Category | Standards Track |
| Editor | Jazzz <jazz@status.im> |

## Abstract

This specification defines ContentFrame, a self-describing message format for decentralized chat networks.
ContentFrame wraps content payloads with metadata identifying their type and governing specification repository.
Using a `(domain, tag)` tuple, applications can uniquely identify message types and locate authoritative documentation for parsing unfamiliar content.
This approach enables permissionless innovation while maintaining the context needed for interoperability, allowing applications to gracefully handle messages from sources they don't explicitly know about.

## Motivation

In an interoperable chat network, participants cannot be assumed to use the same software to send and receive messages.
Users may employ different versions of the same application or different applications entirely.
This heterogeneity creates a fundamental challenge: how can applications support extensible message types without prior knowledge of every possible format?

Two naive approaches each have significant drawbacks:

**Developer-defined types** would allow flexibility but create fragmentation.
When developers define their own message types, the context for parsing these messages remains tightly coupled to the software that created them.
Other applications receiving these messages lack the necessary context to interpret them correctly.
This leads to multiple definitions of basic types such as `Text` and `Image` that are not compatible across applications.

**Fixed type systems** would ensure universal understanding but restrict innovation.
A predetermined set of message types eliminates ambiguity but adds friction for developers who want to extend functionality.
In a permissionless, decentralized protocol, requiring centralized approval for new message types contradicts core design principles.

The core challenge is managing fragmentation in a decentralized protocol while preserving developer freedom to innovate.

**Solution:** A self-describing message format that encodes both the payload and the metadata needed to parse it.
This approach directs application developers on how a message should be parsed while providing a clear path to learn about unfamiliar content types they encounter.
By decoupling the encoded data from the specific software that created it, applications can gracefully handle messages from diverse sources without sacrificing extensibility.


## Theory / Semantics

### ContentFrame

A ContentFrame provides a self-describing format for payload types by encoding both the type identifier and its administrative origin.
The core principle is that each payload should declare which entity is responsible for its definition and provide a unique type discriminator within that entity's namespace.

A ContentFrame consists of two key components:

- **Domain**: Points to a specification repository that defines and governs a collection of types
- **Tag**: A unique identifier within that domain that specifies which type the payload conforms to

Together, the tuple `(domain, tag)` serves two purposes:

1. **Identification**: Uniquely identifies the payload type without ambiguity
2. **Discovery**: Provides a path for developers to learn how to parse and support unfamiliar types

**Benefits:**

This approach provides several advantages for decentralized interoperability:

- **No naming collisions**: Developers can independently create types without coordinating with others, as each domain manages its own namespace
- **Type reuse**: Well-defined, established types can be shared across applications, reducing fragmentation
- **Graceful extensibility**: Applications encountering unknown types can direct developers to the authoritative specification
- **Decentralized governance**: No central authority is required to approve new types; domains manage their own specifications

By separating the "who defines this" (domain) from the "what is this" (tag), ContentFrame enables permissionless innovation while maintaining the context needed for interoperability.


### Concept Mapping

The following diagram illustrates the relationship between ContentFrame components and their specifications:
```mermaid
flowchart TD
d[Domain ID] -->|references| D
D[Domain] -->|Defines| T[Tag]
T -->|References| Specification
```

### Domain

A domain identifies the authority responsible for defining and governing a set of content types.
By including the domain, receiving applications can locate the authoritative specification for a type, regardless of which application originally sent it.

**Requirements:**

- A domain MUST be a valid URL as defined in [RFC 3986](https://datatracker.ietf.org/doc/html/rfc3986)
- A domain MUST host or reference definitions for all content types within its namespace
- A domain SHOULD be a specification repository or index that developers can reference

**Specification Format:**

Domains are responsible for describing their types in whatever format is most appropriate.
The only requirement is that the information needed to parse and understand each type is accessible from the domain URL.


**Domain ID Mapping:**

To minimize payload size, domains are mapped to integer identifiers.
Each domain is assigned a unique `domain_id` which is used in the wire format instead of the full URL.

- A `domain_id` MUST be a positive integer value
- A `domain_id` MUST correspond to exactly one unique domain
- The canonical mapping of `domain_id` to domains can be found in [Appendix A: Domains](#appendix-a-domains)

### Tag

A tag is a numeric identifier that uniquely specifies a content type within a domain's namespace.
After resolving the domain and tag, application developers have all the information needed to locate the definition and parse the payload.

**Requirements:**

- A tag MUST be a positive integer value
- A tag MUST uniquely identify a single type within its domain
- Two payloads with the same `(domain, tag)` tuple MUST conform to the same type specification
- A tag's meaning MUST NOT change after it has been assigned within a domain

**Domain Responsibility:**

Each domain is responsible for:
- Assigning and managing tag values within its namespace
- Documenting how each tag maps to a type specification
- Ensuring tag assignments remain stable and unambiguous

Tags are scoped to their domain, meaning the same tag value can represent different types in different domains without conflict.



## Wire Format Specification / Syntax
```protobuf
message ContentFrame {
uint32 domain_id = 1;
uint32 tag = 2;
bytes payload = 3;
}
```

**Field Descriptions:**

- **domain_id**: Identifies the domain that governs this content type
- **tag**: Identifies the specific content type within the domain's namespace
- **payload**: The encoded content data

All fields are required.


## Implementation Suggestions

### Tags to Specifications

Where possible, tag values should directly correspond to specification identifiers.
Using specification IDs as tags removes the need to maintain a separate mapping between tags and specifications.

### Fragmentation

This protocol allows multiple competing definitions of similar content types.
Having multiple definitions of `Text` or `Image` increases fragmentation between applications.
Where possible, reusing existing types will reduce burden on developers and increase interoperability.

Domains should focus on providing types unique to their service or use case.


# Appendix A: Domains

![TODO] Find appropriate home for this registry.
Comment thread
fbarbu15 marked this conversation as resolved.

Domain IDs are assigned sequentially on a first-come, first-served basis.
New domains are added via pull request.

**Registry Rules:**

- A domain MUST only appear once in the table
- A domain MAY be updated by the original submitter if the repository has been moved

**Registry:**

| domain_id | specification repository |
|-----------|--------------------------------------|
| 1 | https://github.com/logos-co/logos-lips |
Loading
Loading