-
Notifications
You must be signed in to change notification settings - Fork 4
docs(messaging): move remaining specs from logos-messaging/specs #346
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
4 commits
Select commit
Hold shift + click to select a range
20a352e
docs(messaging): copy remaining specs verbatim from logos-messaging/s…
igor-sirotin 2884ef3
docs(messaging): adapt remaining specs to logos-lips conventions
igor-sirotin c3b8096
fix: address pr comments
igor-sirotin 347b4a5
fix: chat-framework category
igor-sirotin File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,134 @@ | ||
| # CHAT-FRAMEWORK | ||
|
|
||
| | Field | Value | | ||
| | --- | --- | | ||
| | Name | A modular framework for defining chat protocols | | ||
| | Status | raw | | ||
| | Type | RFC | | ||
| | Category | Standards Track | | ||
| | Tags | chat | | ||
| | Editor | Jazz Alyxzander <jazz@status.im> | | ||
|
|
||
| # Abstract | ||
|
|
||
| This specification defines a modular communication protocol framework for describing chat protocols. It introduces abstraction boundaries, and a component model for describing chat protocol functionality. | ||
|
|
||
|
|
||
| # Background / Rationale / Motivation | ||
|
|
||
| Chat protocols specifications can be long and dense documents. To fully describe a chat protocol there are many layers and operations which are required to be documented. This includes payloads, message transport, encryption as well as user level features such as account registration, typing indicators, content formatting. | ||
|
|
||
| With the vast amount of information required to maintain compatibility between applications - protocol documentation is either comprehensive which leads to large monolithic specifications or lacking the required details for interop between implementors. A suitable solution would provide both the specificity while also remaining lean and focused. | ||
|
|
||
| # Theory / Semantics | ||
|
|
||
| This specification defines an abstract framework for building a chat protocol. Its purpose is to name the distinct components/phases, and define modular boundaries between them to promote reuse. The end result is that a chat protocol implementation can be described by listing its approach to 5 things. | ||
|
|
||
| The lifecycle of a protocol instance is divided into three phases: | ||
|
|
||
| - **Discovery:** How does a Sender learn of other clients. | ||
| - **Initialization:** How does a Recipient learn a client wants to communicate with them. | ||
| - **Operation:** How do participants exchange content. | ||
|
|
||
| and transport details are divided into 2 components: | ||
|
|
||
| - **Delivery Service:** How are payloads routed and delivered to a client. | ||
| - **Framing Strategy:** How are payloads encoded. | ||
|
|
||
| Defining these 5 parameters allows for chat protocol implementations to be fully defined, which allows clients from different applications to exchange messages. | ||
| While also dividing documents by their focus area. | ||
|
|
||
| ## Abstract Transport | ||
|
|
||
| ### Delivery service | ||
|
|
||
| A Delivery Service (DS) is the service or method that distributes payloads to clients. A DS accepts payloads with a delivery_address and delivers them to all subscribers of that delivery_address. Protocols use delivery_addresses to establish delivery contracts between senders and recipients. The mapping of delivery_addresses to DS-level concepts is implementation-specific. | ||
|
|
||
| #### Requirements | ||
|
|
||
| - A DS MUST provide a method for clients to subscribe to messages from a delivery_address | ||
| - Payloads sent to a delivery_address are delivered by a DS to all subscribers of that delivery_address | ||
| - A DS SHOULD handle segmentation if the underling transport limits message sizes | ||
| - A DS MAY NOT guarantee message delivery | ||
| - A DS MAY NOT guarantee message ordering | ||
| - A DS MAY reject payloads | ||
|
|
||
|
|
||
| ### Framing Strategy | ||
|
|
||
| In this protocol framework, payloads from multiple protocols are potentially multiplexed over the same channel. This requires that clients are able to associate a given payload to a given instance of a protocol. | ||
|
|
||
| A framing strategy should define a common payload type as well as a method to determine which state machine a receiving client must use to decode it. | ||
|
|
||
|
|
||
| ## Protocol Phases | ||
|
|
||
| In order to exchange content clients must be able to learn of each others existence, gather the pre-requisite information/parameters and, remain synchronized over time. | ||
|
|
||
| The lifecycle of a protocol instance is divided into three phases, which are described by a corresponding protocol. | ||
|
|
||
| - **Discovery Phase:** Discovery Protocol | ||
| - **Initialization Phase:** Initialization Protocol | ||
| - **Operation Phase:** Conversation Protocol | ||
|
|
||
| ```mermaid | ||
| sequenceDiagram | ||
| participant D as ??? | ||
| participant S as Saro | ||
| participant R as Raya | ||
|
|
||
|
|
||
| Note over D,S: Discovery | ||
| D -->> S: <IntroBundle> | ||
|
|
||
| Note over R,S: Initialization | ||
| S ->> R: Invite | ||
|
|
||
| Note over R,S: Operation | ||
|
|
||
| loop | ||
| par | ||
| R->> S: Send Message | ||
| and | ||
| S->> R: Send Message | ||
| end | ||
| end | ||
| ``` | ||
|
|
||
|
|
||
|
|
||
| ### Discovery Protocol | ||
|
|
||
| A discovery protocol defines how clients gather the prerequisite information to contact another client. | ||
|
|
||
| The input requirements of the discovery protocol are not defined here, and largely determined by the desired user experience. | ||
|
|
||
| The output requirements of the discovery protocol are very implementation specific, and will depend on the initialization protocol requirements and DS chosen by the implementation. The data provided by the discovery protocol and required by the initialization protocol is called the `IntroductionBundle`. | ||
|
|
||
| - The discovery protocol MUST provide all data required for the initialization protocol. | ||
|
|
||
| Note: There is no requirement that the Discovery protocol be neither a complicated nor interactive process. Hypothetically If all required values to construct a IntroductionBundle could be statically defined, that would be sufficient for this definition. | ||
|
|
||
| ### Initialization Protocol | ||
|
|
||
| A initialization protocol specifies how two clients can initiate communication. The input to this process is the `IntroductionBundle` and the output is an established instance of a `Conversation` between the participants. | ||
|
|
||
| The core of an initialization protocol is a defined location and procedure for receiving initial messages. | ||
|
|
||
| Many chat protocols choose to define the initialization protocol within the conversation protocol. This tight coupling produces two negative artifacts. | ||
| - New conversation protocols must define and deploy there own initialization channels. Increasing overhead and adding complexity. | ||
| - Protocol upgrades then create partitions in the communication network, as older clients have no means of communicating with new clients. | ||
|
|
||
| Separating channel initialization from conversation flow allows multiple conversations to reuse the same initialization channel. This reduces effort for new conversation protocols, and is especially valuable when upgrading existing ones. Being independent the initialization pathway can persist across conversation versions. Even if an older client cannot parse new message types, it can still recognize their presence, adding observability. | ||
|
|
||
| ### Conversation Protocol | ||
|
|
||
| A conversation protocol defines how messages flow between participants, and subsequently determines the properties of that channel. | ||
|
|
||
| - A Conversation protocol MUST define the payloads it uses and how to handle them. | ||
| - A Conversation protocol SHOULD outline the cryptographic properties provided | ||
| - A Conversation protocol SHOULD describe bidirectional communication. | ||
|
|
||
| ## Copyright | ||
|
|
||
| Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/). | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,180 @@ | ||
| # CONTENTFRAME | ||
|
|
||
| | Field | Value | | ||
| | --- | --- | | ||
| | Name | Chat Content Frames | | ||
| | Status | raw | | ||
| | Type | RFC | | ||
| | Category | Standards Track | | ||
| | Editor | Jazzz <jazz@status.im> | | ||
|
|
||
| ## Abstract | ||
|
|
||
| This specification defines ContentFrame, a self-describing message format for decentralized chat networks. | ||
| ContentFrame wraps content payloads with metadata identifying their type and governing specification repository. | ||
| Using a `(domain, tag)` tuple, applications can uniquely identify message types and locate authoritative documentation for parsing unfamiliar content. | ||
| This approach enables permissionless innovation while maintaining the context needed for interoperability, allowing applications to gracefully handle messages from sources they don't explicitly know about. | ||
|
|
||
| ## Motivation | ||
|
|
||
| In an interoperable chat network, participants cannot be assumed to use the same software to send and receive messages. | ||
| Users may employ different versions of the same application or different applications entirely. | ||
| This heterogeneity creates a fundamental challenge: how can applications support extensible message types without prior knowledge of every possible format? | ||
|
|
||
| Two naive approaches each have significant drawbacks: | ||
|
|
||
| **Developer-defined types** would allow flexibility but create fragmentation. | ||
| When developers define their own message types, the context for parsing these messages remains tightly coupled to the software that created them. | ||
| Other applications receiving these messages lack the necessary context to interpret them correctly. | ||
| This leads to multiple definitions of basic types such as `Text` and `Image` that are not compatible across applications. | ||
|
|
||
| **Fixed type systems** would ensure universal understanding but restrict innovation. | ||
| A predetermined set of message types eliminates ambiguity but adds friction for developers who want to extend functionality. | ||
| In a permissionless, decentralized protocol, requiring centralized approval for new message types contradicts core design principles. | ||
|
|
||
| The core challenge is managing fragmentation in a decentralized protocol while preserving developer freedom to innovate. | ||
|
|
||
| **Solution:** A self-describing message format that encodes both the payload and the metadata needed to parse it. | ||
| This approach directs application developers on how a message should be parsed while providing a clear path to learn about unfamiliar content types they encounter. | ||
| By decoupling the encoded data from the specific software that created it, applications can gracefully handle messages from diverse sources without sacrificing extensibility. | ||
|
|
||
|
|
||
| ## Theory / Semantics | ||
|
|
||
| ### ContentFrame | ||
|
|
||
| A ContentFrame provides a self-describing format for payload types by encoding both the type identifier and its administrative origin. | ||
| The core principle is that each payload should declare which entity is responsible for its definition and provide a unique type discriminator within that entity's namespace. | ||
|
|
||
| A ContentFrame consists of two key components: | ||
|
|
||
| - **Domain**: Points to a specification repository that defines and governs a collection of types | ||
| - **Tag**: A unique identifier within that domain that specifies which type the payload conforms to | ||
|
|
||
| Together, the tuple `(domain, tag)` serves two purposes: | ||
|
|
||
| 1. **Identification**: Uniquely identifies the payload type without ambiguity | ||
| 2. **Discovery**: Provides a path for developers to learn how to parse and support unfamiliar types | ||
|
|
||
| **Benefits:** | ||
|
|
||
| This approach provides several advantages for decentralized interoperability: | ||
|
|
||
| - **No naming collisions**: Developers can independently create types without coordinating with others, as each domain manages its own namespace | ||
| - **Type reuse**: Well-defined, established types can be shared across applications, reducing fragmentation | ||
| - **Graceful extensibility**: Applications encountering unknown types can direct developers to the authoritative specification | ||
| - **Decentralized governance**: No central authority is required to approve new types; domains manage their own specifications | ||
|
|
||
| By separating the "who defines this" (domain) from the "what is this" (tag), ContentFrame enables permissionless innovation while maintaining the context needed for interoperability. | ||
|
|
||
|
|
||
| ### Concept Mapping | ||
|
|
||
| The following diagram illustrates the relationship between ContentFrame components and their specifications: | ||
| ```mermaid | ||
| flowchart TD | ||
| d[Domain ID] -->|references| D | ||
| D[Domain] -->|Defines| T[Tag] | ||
| T -->|References| Specification | ||
| ``` | ||
|
|
||
| ### Domain | ||
|
|
||
| A domain identifies the authority responsible for defining and governing a set of content types. | ||
| By including the domain, receiving applications can locate the authoritative specification for a type, regardless of which application originally sent it. | ||
|
|
||
| **Requirements:** | ||
|
|
||
| - A domain MUST be a valid URL as defined in [RFC 3986](https://datatracker.ietf.org/doc/html/rfc3986) | ||
| - A domain MUST host or reference definitions for all content types within its namespace | ||
| - A domain SHOULD be a specification repository or index that developers can reference | ||
|
|
||
| **Specification Format:** | ||
|
|
||
| Domains are responsible for describing their types in whatever format is most appropriate. | ||
| The only requirement is that the information needed to parse and understand each type is accessible from the domain URL. | ||
|
|
||
|
|
||
| **Domain ID Mapping:** | ||
|
|
||
| To minimize payload size, domains are mapped to integer identifiers. | ||
| Each domain is assigned a unique `domain_id` which is used in the wire format instead of the full URL. | ||
|
|
||
| - A `domain_id` MUST be a positive integer value | ||
| - A `domain_id` MUST correspond to exactly one unique domain | ||
| - The canonical mapping of `domain_id` to domains can be found in [Appendix A: Domains](#appendix-a-domains) | ||
|
|
||
| ### Tag | ||
|
|
||
| A tag is a numeric identifier that uniquely specifies a content type within a domain's namespace. | ||
| After resolving the domain and tag, application developers have all the information needed to locate the definition and parse the payload. | ||
|
|
||
| **Requirements:** | ||
|
|
||
| - A tag MUST be a positive integer value | ||
| - A tag MUST uniquely identify a single type within its domain | ||
| - Two payloads with the same `(domain, tag)` tuple MUST conform to the same type specification | ||
| - A tag's meaning MUST NOT change after it has been assigned within a domain | ||
|
|
||
| **Domain Responsibility:** | ||
|
|
||
| Each domain is responsible for: | ||
| - Assigning and managing tag values within its namespace | ||
| - Documenting how each tag maps to a type specification | ||
| - Ensuring tag assignments remain stable and unambiguous | ||
|
|
||
| Tags are scoped to their domain, meaning the same tag value can represent different types in different domains without conflict. | ||
|
|
||
|
|
||
|
|
||
| ## Wire Format Specification / Syntax | ||
| ```protobuf | ||
| message ContentFrame { | ||
| uint32 domain_id = 1; | ||
| uint32 tag = 2; | ||
| bytes payload = 3; | ||
| } | ||
| ``` | ||
|
|
||
| **Field Descriptions:** | ||
|
|
||
| - **domain_id**: Identifies the domain that governs this content type | ||
| - **tag**: Identifies the specific content type within the domain's namespace | ||
| - **payload**: The encoded content data | ||
|
|
||
| All fields are required. | ||
|
|
||
|
|
||
| ## Implementation Suggestions | ||
|
|
||
| ### Tags to Specifications | ||
|
|
||
| Where possible, tag values should directly correspond to specification identifiers. | ||
| Using specification IDs as tags removes the need to maintain a separate mapping between tags and specifications. | ||
|
|
||
| ### Fragmentation | ||
|
|
||
| This protocol allows multiple competing definitions of similar content types. | ||
| Having multiple definitions of `Text` or `Image` increases fragmentation between applications. | ||
| Where possible, reusing existing types will reduce burden on developers and increase interoperability. | ||
|
|
||
| Domains should focus on providing types unique to their service or use case. | ||
|
|
||
|
|
||
| # Appendix A: Domains | ||
|
|
||
| ![TODO] Find appropriate home for this registry. | ||
|
fbarbu15 marked this conversation as resolved.
|
||
|
|
||
| Domain IDs are assigned sequentially on a first-come, first-served basis. | ||
| New domains are added via pull request. | ||
|
|
||
| **Registry Rules:** | ||
|
|
||
| - A domain MUST only appear once in the table | ||
| - A domain MAY be updated by the original submitter if the repository has been moved | ||
|
|
||
| **Registry:** | ||
|
|
||
| | domain_id | specification repository | | ||
| |-----------|--------------------------------------| | ||
| | 1 | https://github.com/logos-co/logos-lips | | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In the previous repo the folder structure helped to separate:
WIthout these its harder to navigate.
[Sand] Consider separating Delivery and Chat. The specifications are mutually exclusive; combining only adds confusion.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@fbarbu15 what do you think about this?
The current folder structure is per-team (e.g. there's anoncomms, messaging and research), but it doesn't make much sense for an external reader. Maybe indeed we should rather make dir-structure follow logical areas of application?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(I'll keep it as is in this PR anyway though, this is a separate discussion)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@igor-sirotin I'm ok with that, blockchain team asked for a similar folder structure like they had in notion and it looks like this now in LIPS

So if you guys need to categorize specs in any other way feel free to do it or lmk and I can look into it.