diff --git a/docs/rfc-v1-social-specs.md b/docs/rfc-v1-social-specs.md new file mode 100644 index 0000000..9771a9b --- /dev/null +++ b/docs/rfc-v1-social-specs.md @@ -0,0 +1,502 @@ +# RFC: pubky-social-specs v1 (first stable, first breaking release) + +> Status: DRAFT, for review. This document is the complete v1 design and rollout plan: the +> design model by model (v0 shape, v1 shape, why), then migration, then the rollout. Comment +> inline on the line you disagree with. The companion `v0-vs-v1.md` expands every model change +> with its full reasoning. +> +> Terms used throughout: +> - **epoch**: the `vN` path segment; each epoch is a disjoint subtree holding one generation of data (`social/v1` today). +> - **root**: the leading `pub` or `priv` path segment. The parser reports it as a resource's `visibility`; the `/priv/` root as a feature is the **privacy tier**. +> - **dual-root**: a resource that may live under either root (posts, files, feeds). +> - **now-or-never**: a rule only a breaking release can introduce; deferring it means waiting for `social/v2`. +> - **wire**: the stored/serialized JSON byte form. +> - **scheme tier**: a reference field's validation class: pubky-only, pubky+web, web-only, or universal (any scheme). +> - **nexus**: the shared indexer. +> - **substrate**: the homeserver's dumb blob store (no compare-and-swap, no server-side logic). +> - **tombstone**: the deletion marker readers act on; in v1, the absence of any public copy. +> - **pinned / frozen**: committed as a closed-form rule or data asset; never tracks library or Unicode updates. + +The first stable and first breaking release of the shared social-data layer: shared means many +applications read and write the same objects (posts, follows, tags), so the schema can belong to +no single app. Renames the crate +`pubky-app-specs` to `pubky-social-specs` (`1.0.0`) and moves all data from the single hard-coded +app path `/pub/pubky.app/` to a versioned, app-neutral epoch `/{pub|priv}/social/v1/` +(an epoch is the `vN` path segment; each epoch is a disjoint subtree holding one generation of +data). +The strategy has three parts. First, this one coordinated break makes every now-or-never +change, those that cannot be added additively later. Second, a permanent forward-compat contract +makes everything else additive, so v1.x grows without breaking. Third, a future epoch +(`social/v2`) stays reserved for the few changes no contract can make additive: re-pinning a +text or id function (which re-ids existing data), changing a resource's root (its leading `pub` +or `priv` segment) or per-kind content semantics, or breaking the path grammar. + +# Part A: Why break now + +- `pub/pubky.app/` hard-codes one app's domain as the home of shared data and the parser rejects + every other app. A shared spec must classify foreign data, not error on it. +- The path is the only version signal that survives the `/events/` feed, LIST, and anonymous GET. + v0 has none, so v1.x could not evolve without breaking old clients. +- No privacy tier, no file extensions, GET-per-file bookmarks (listing them costs one GET each), and `url::Url` validation that + normalizes junk into acceptance while rejecting valid short-form URIs. + +# Part B: Design, model by model + +## B0. Cross-cutting (applies to every model) + +- **Namespace + epoch: `pub/pubky.app/` becomes `{pub|priv}/social/v1/`.** App-neutral + (`pubky.app` wrongly signals one app owns shared data), versioned (the path is the only channel + that survives events + LIST + anonymous GET), and a bare word with no dot (a dotted directory + name like `pubky.app/` reads as an application bundle on macOS when a tree is exported to disk). +- **Folder ownership (the composition law).** The specification that defines an object + determines its storage namespace, never the application that writes it. An app writing social + objects writes them under `social/vN`; its own objects live under its own namespace. Apps + therefore compose spec packages, each bringing its capability scopes and path builders (request + `/pub/social/v1/:rw` alongside your app scopes; use each package's builders for its paths). + Consequence: tags are social objects, so the one canonical v1 write location is + `pub/social/v1/tags/`; indexers reading `tags/` directories inside other app namespaces is a + legacy read rule, not a v1 write model. +- **App namespaces SHOULD carry an epoch too** (`pub//v1/...`). Nothing can be mandated for + foreign namespaces (no enforcement point exists), but the recommendation is free and buys an + app the same migration mechanics this spec built for itself: old and new data coexist in + disjoint subtrees, and the path is the only version signal that survives the events feed, + LIST, and anonymous GET. The parser already anticipates this: `Foreign` classification surfaces the segment after the + namespace verbatim. For an app following the convention, that segment IS its version, so an + indexer can version-route conforming app data with a single match. +- **Namespace governance.** `social/vN` is owned by this repo: a resource type exists exactly + when the released crate parses it, and additions land as ordinary crate-minor PRs here (parser + arm + model + data assets + vectors in one change). Reserved: epoch segments `v[0-9]+`, the `_` + filename prefix, every current resource segment, and the `ext` member name; unknown segments + under `social/vN` parse as a handled `Resource::Unknown` (a valid classification readers + skip, never an error; distinct from the `Unknown` enum variant of the forward-compat contract + below), so additions never break deployed readers. A + new epoch (`social/v2`) is reserved for changes impossible additively (re-pinned text/id + functions, changed root or content semantics, grammar breaks). App namespaces are self-assigned + (reversed-domain form recommended: `app.locks`, `app.eventky`, not `locks.app`, for the same + reason the social segment is a bare word: a directory ending in `.app` is treated as an + application bundle by macOS when a tree is exported to disk); the parser classifies them + foreign, never invalid. +- **`.json` on every JSON leaf.** The homeserver derives the served type from magic bytes then the + path extension; extensionless JSON serves as octet-stream/plaintext. +- **A privacy tier `/priv/` (owner-only, excluded from `/events/`).** For state whose only + reader is the owner's own client; placement follows the ACTUAL reader set, not aspiration. + The leading path segment is the ROOT (`pub` or `priv`); the parser reports it as the + resource's visibility. The content family (posts, files, feeds) is dual-root: an object may + live under either root, and publish is a deterministic root migration. One rule here is + now-or-never, THE ROOT RULE: a public-rooted object must never reference a priv-root pubky + URI; private objects may reference both roots. (A public-to-private reference dangles for + every reader but the owner and leaks the path's existence through the 401-vs-404 oracle.) +- **Scheme tiers (reference field values), distinct from the path grammar.** Appendix A governs PATHS + (where objects live); reference FIELD VALUES (parent, embed, targets, images) are validated by + per-field scheme tiers: pubky-only, pubky+web, or the universal tier (any scheme-shaped URI + via a pinned opaque gate: lowercased scheme, rest verbatim). Per-field tiers (each model section gives the rationale): + + | Field | Tier | + |---|---| + | `post.parent` | pubky-only | + | `post.embed` | universal | + | `post.lock` | pubky-only | + | `collection.items[]` | pubky-only | + | tag target | universal | + | bookmark target | universal | + | `attachments[].uri` | pubky+web | + | `user.image`, `article.cover_image`, `collection.cover_image` | pubky+web | + | `user.links[].url` | web-only | +- **Forward-compat contract (permanent).** All wire enums (an enum as stored in JSON; "wire" + throughout means the stored byte form) are plain string enums (unit + variants, `rename_all` lowercase/snake_case), so `#[serde(other)] Unknown` plus + `#[non_exhaustive]` is well-defined; no model uses `deny_unknown_fields`; every future field is + optional + defaulted + skip-if-none. Degradation semantics are per-position: an `Unknown` in an + object's primary enum (`post.kind`) fails validation and readers skip the object; an `Unknown` + in a secondary enum (`feed.content`) degrades to "no constraint"; deserialization never crashes. + Unknown members are tolerated on read AND preserved on rewrite: every wire model carries an + opaque catch-all map (serde flatten: it absorbs every member the model does not declare), so a + client rewriting an object round-trips members it does not + understand instead of destroying another client's data (tolerating without preserving would let + any older client drop every field added after it shipped). Conformance vectors (committed input and expected-output files every implementation must + reproduce) cover both the unknown-value and the preservation behavior. Preservation fixes exactly one failure mode: a client deserializing into its own older types + and writing back, silently destroying fields it never modeled. It does not fix concurrency: + concurrent writers still clobber whole files under last-write-wins where that is the + documented rule. Two companion rules keep extensibility bounded. (1) TOTAL object size is + capped: 512 KiB for posts, 64 KiB for every other JSON resource, measured on stored bytes, + checked before parsing on read and after building on write. Unknown members made per-field + validation stop bounding size, and a total cap cannot be added later without a break, so it + ships now. (2) A conforming rewrite fetches the current object from the HOMESERVER, never + from an indexer view (views can be stale or partial). The caps cover JSON resources only + (media bytes keep their own media-size bound), and the wedge case is pinned: if a rewrite + cannot fit the cap while preserving unknown members, the writer fails the write and surfaces + it, never silently drops members; the user may explicitly discard extensions. + The indexer side: unknown members are carried verbatim into object views, so any client can read an extension + through the shared index before the indexer understands it, but they are never validated, + queried, or indexed; queryability requires an adopted projection (the indexer explicitly promoting the member into + its queryable schema). The extension ladder: + readable (carried) -> queryable (projection) -> validated (spec field). Deliberate extensions + SHOULD nest under the reserved `ext` member (`"ext": {"badge": {...}}`), one greppable home + whose meaning is pinned once: everything under `ext` is third-party data the base spec never + validates; treat it as hostile input (escape before rendering, validate against the + extension's own rules before interpreting). +- **Canonical-encoding id rule.** An id is valid if and only if re-encoding its decoded bytes reproduces the + input (closed-form final-char check). v0 accepted dozens of alias spellings per id (lowercase, + `O`->`0`, dangling bits), each a distinct homeserver key; that leniency is removed. +- **Engine-free validation.** `url::Url` (normalizes junk into acceptance), the `mime` crate, and + full-Unicode case/trim are replaced by pinned rules (a strict raw-string canonicalizer, a + frozen whitespace table (a committed code-point list that never tracks Unicode updates), + ASCII-only label folding, code-point lengths) that a Rust and a + hand-written JS implementation reproduce byte-for-byte. +- **No silent sanitize-rewrites.** v0 rewrote `[DELETED]` names to "anonymous" and + truncated-then-blanked over-long inputs; v1 makes invalid input a validation error. + +A migrated user's tree at a glance (`` a host key, `TS` a TimestampId, `H26` a HashId): + + pubky:/// + |-- pub/social/v1/ + | profile.json + | posts/TS1/TS1.json first version (editId reuses the post id) + | posts/TS1/TS2.json an edit; references still say posts/TS1 + | tags/H26.json + | follows/.json + | files/H26.jpg + | feeds/H26.json a published feed (copy of the private file) + |-- priv/social/v1/ + posts/TS3/TS3.json a draft + bookmarks/.json target readable from the filename + mutes/.json + feeds/H26.json + settings.json + last_read.json + +## B1. User (profile) +v0 `pub/pubky.app/profile.json` -> v1 `pub/social/v1/profile.json`. +- `image` accepts pubky/http/https via one shared image validator (cap 300). pubky-app avatars are + pubky file URIs; an http-only rule would reject every real avatar. +- The `[DELETED]` magic string dies entirely: v0's silent `[DELETED]` -> "anonymous" rewrite is + removed with NO replacement rule; `[DELETED]` is an ordinary legal name. **Required upgrade in nexus, the shared indexer (gates v1 indexing):** the indexer currently keys deletion on that literal; it must key + on a real flag (`deleted` on the indexed row / UserView) before indexing any v1 data. How a + deleted account is displayed then becomes pure client presentation (the indexer may + transitionally keep emitting the old literal at its view layer for old clients; storage and + query logic never key on it). +- Fields and caps otherwise unchanged; profile stays public (identity must be readable). + +## B2. Post +v0 `pub/pubky.app/posts/{id}` (one flat file, overwritten on edit) -> v1 +`{pub|priv}/social/v1/posts/{id}/{editId}.json`, referenced versionlessly as `posts/{id}`; +`{id}` and `{editId}` are both canonical TimestampIds (Appendix A3, A4). +- **Per-edit path versioning.** Each edit is a new file; the first version reuses the post id; + references use the versionless form so edits never orphan replies or re-key tags. A counter was + rejected (races across devices; the substrate, the homeserver's dumb blob store, has no + compare-and-swap). Now-or-never: a + `posts/{id}` file and a `posts/{id}/` directory are mutually exclusive on the homeserver. +- **Kinds renamed** `short`->`note`, `long`->`article` (nature, not length); all seven kinds kept. +- **`embed`** flattened from `{uri, kind}` to a plain URI string (kind is derivable from the + target), and it accepts ANY external URI, the same universal tier as tags: http/https through + the strict web gate (the pinned regex validator for web URLs) (an OpenStreetMap object URL is an ordinary https reference), and any other + scheme-shaped identifier (`nostr:`, `geo:`, `ipfs:`, `did:`) through a pinned opaque gate + (lowercased scheme + rest verbatim, no engine parsing). The indexer attaches the post to the same External Resource + nodes (its graph records for non-pubky targets) that it builds + for external tag targets. `parent` stays pubky-only (a reply is a social-graph edge with + thread semantics that exist only between posts). This also keeps migration total (every real v0 record has a valid v1 image; nothing fails to + migrate): v0's `Url::parse` accepted arbitrary schemes, so real v0 data can carry them. +- **`attachments`** become `Vec<{uri, alt?, name?}>`, always `[]` never null. Objects, not strings, + so per-item metadata (alt text now; hash/blurhash later) is additive; ships two committed fields. +- **`lock`** kept: the value is the lock-FILE URI (`pub/app.locks/.json`, illustrative), and presence + means "locked content" regardless of kind (matches the Locks feature's resolved design). +- **Dual-root:** posts may live under `/priv/` (drafts, private notes, private collections). +- **The post is a reusable envelope (adopted from review).** The crate exports the shared + mechanics, versioned storage, ids, parent/embed/attachments/lock, preservation, path helpers, + as a generic layer (`PostEnvelope`), with the social post as its first specialization + (closed kind set, wire bytes unchanged). App specs specialize it with their own kinds in their + own namespaces (a Mapky review, an Eventky event), where social readers classify them as + foreign data. The envelope fixes reference semantics uniformly (a reply edge means the same + thing everywhere); the specialization owns its kind vocabulary and content validation. This + makes the incubation path usable at launch: a schema proves itself in an app namespace before + being proposed for `social/vN`. +- The `[DELETED]` content sentinel dies with no replacement (same flag rule as B1); absence is + the tombstone (the deletion marker readers act on), synthesized by the indexer from real + deletion state, never from content strings. + +## B3. ArticleContent (new) +v0: pubky-app hand-rolls `{title, body}` JSON inside `long` posts, unspecified, cover smuggled as +`attachments[0]`. v1: a typed `{title, body, cover_image?}` content envelope in `content` when +`kind == article` (a per-kind content schema INSIDE the content string, distinct from the +`PostEnvelope` mechanics layer of B2). Per-kind content shape is now-or-never; the cover moves into the envelope. +Articles may carry parent/embed/attachments (an article can be a reply or carry media). + +## B4. CollectionContent +Shape unchanged (`{name, description?, items[], cover_image?}`). `items` now accept any +pubky URI (any resource, any app; the pubky-only scheme tier of B0), not just `posts/` under `pubky.app` (a curated +list may include foreign resources). Private collections come free from the dual-root post family. + +## B5. Tag +`tags/{id}.json`. Id = `HashId("{target}:{label}")`, the target canonicalized, the label +frozen-trimmed and ASCII-folded +label (v0 used engine `to_lowercase` and `url::Url` normalization, neither reproducible across +implementations; content-addressed ids must freeze their input functions). The `:` join is injective only because labels reject `:`; that restriction may never be +lifted while the id format stands. One write location, any target: every app writes tags at the +author's `pub/social/v1/tags/`, and the target may be any public resource: social objects, other +apps' objects, or ANY external URI (http/https via the strict web gate; other schemes, `nostr:`, +`geo:`, `ipfs:`, `did:`, via a pinned opaque gate that lowercases the scheme and keeps the rest +verbatim). v0 accepted these via `Url::parse`, so this also keeps migration total. A logical tag therefore has exactly one possible address: +re-tagging self-overwrites idempotently, apps on the same account converge on the same file, and +the indexer reads one namespace with no writing-app dimension (reading `tags/` directories in +other app namespaces survives only as a legacy rule; migrating those files is the owning app's +job). Because addresses converge, a tag writer SHOULD GET the address first and preserve unknown +members if a file exists: a blind PUT would destroy another app's enrichment of the same +statement (preservation protects read-modify-write, not write-without-read). + +## B6. Bookmark +v0 `pub/pubky.app/bookmarks/{HashId(uri)}` (public, one-way filename, GET-per-file) -> v1 +`priv/social/v1/bookmarks/{filename}.json`. +- **Private** (reader set is the owner). Targets take the universal tier: any public pubky + resource or any external URI, same domain as tags. Honest cost, for review: going private + retires the indexer's bookmark-derived public features, including collection-follows + (following a collection was modeled as a bookmark on it); an explicit follow/subscribe + resource is the deferred replacement candidate. +- **Target in the filename, reversibly.** Primary form: unpadded `base64url(canonical target)` + (no `=`; the parser's form check accepts alphabet characters only) for + targets up to 187 bytes. The math: the homeserver caps a path segment at 255 characters, and `.json` takes 5, leaving + 250. base64url is the densest standard encoding that is also `/`- and `%`-free and natively + JS-decodable, and 250 base64url characters carry 187 bytes, so 187 is the largest cap ANY + reversible encoding allows. Longer targets use the overflow form `~ + HashId(target)` + with the target kept in content. Listing costs zero GETs for primary-form entries (the + overwhelming majority); each overflow entry costs one GET to recover its target. +- Content shrinks to `{created_at}` (plus `target` only in overflow). + +## B7. Follow +`follows/{followeePk}.json`, `{created_at}`. Public (the social graph). Only the cross-cutting +changes apply; the filename-is-target pattern (one LIST answers "who do I follow") is kept. + +## B8. Mute +`priv/social/v1/mutes/{muteePk}.json`. Moves to `/priv/`; who you muted is sensitive and its only +reader is the owner (the indexer has zero mute consumers). Shape unchanged. + +## B9. LastRead +`priv/social/v1/last_read.json`, microseconds (was the lone milliseconds outlier). Private. + +## B10. File (media), the v0 File + Blob pair collapsed +v0: two objects, `files/{id}` metadata + `blobs/{hash}` bytes -> v1: ONE content-addressed media +object `files/{hash}.{ext}`, the raw bytes. +- `name` relocates to the attachment object (per-reference, so shared bytes can carry different + names). The declared MIME is consumed once at upload to pick the extension and is never stored; + size and served type come from the bytes and headers. Authoring time is the referencing post's id. +- Deletes the `src` indirection, the two-PUT dance, and the metadata-per-blob ambiguity. Dual-root. +- The extension comes from a frozen MIME-to-ext map (`.bin` fallback), path-only, never hashed. + +## B11. Feed +v0 `pub/pubky.app/feeds/{HashId(serde-json config)}` (public) -> v1 +`{priv|pub}/social/v1/feeds/{id}.json`. +- **Private by default, published by choice** (copy the same file to `/pub/`). Reader set is the + owner today; publishing is a deliberate act. +- **Content-addressed id over a pinned config string** (not serde output, which is field-order + fragile and not JS-reproducible). Six fixed segments: reach, layout, sort, content filter, + tags, and domain tags. Two users publishing the same config share an id, so future + cross-homeserver popularity ranking is additive indexer work. +- Tracks current v0 (#143): the `wot` and `me` reach values and the optional `domain_tags` + filter (same folding and cap rules as tags) are part of the v1 model, and `domain_tags` + participates in the id (two feeds differing only in domain filter are different feeds). +- All three enums gain `Unknown` (v0 hard-crashes old clients on any new reach/layout/sort value); + `name` gains a cap. + +## B12. Settings (new) +v0: pubky-app's `pub/pubky.app/settings.json` was the only unspec'd homeserver artifact, world-readable +while exposing the user's privacy posture (`require_pin`, `sign_out_inactive`) to a reader set of +one. v1: `PubkySocialSettings` at `priv/social/v1/settings.json`, all sections optional, whole-file +last-write-wins on a microsecond `updated_at`, the dead per-file `version` field dropped. Rewrites +preserve unknown members via the catch-all map (B0), so an older client editing one field cannot +destroy a section a newer client wrote; only the concurrent-edit race is lost, by last-write-wins design. + +## B13. Parser and `Resource` +v0 `url::Url`-based, hard-rejects any non-`pubky.app` app path, silently accepts userinfo/`..`/query +-> v1 one closed grammar: `Foreign` and `UnsupportedVersion` are first-class handled categories +(never errors), failed id validation yields `Unknown` (never a panic or error), wrong-root is +unrepresentable. A future `social/v2` reads as "upgrade me," not garbage. The normative grammar +is Appendix A; the reference crate and its committed conformance vectors are the executable form. + +## B14. IDs +TimestampId (post and edit ids), HashId (tag/media/feed files, 128-bit), PubkyId (host/follow/mute), all +under the canonical-encoding rule (B0). TimestampId gains a per-session monotonic mint guard (the +JS runtime mints at ms resolution, so same-ms writes would clobber on path). No id function +content-addresses a serialized struct, so the Rust/JS byte-identity surface is pure string/byte +functions. + +# Part C: Migration + +- **Client-side, opt-in, resumable from the homeserver tree alone, permanent multi-epoch.** A + dormant user may migrate years later in one pass; the indexer dual-reads every epoch forever + (the permanent v0 parser stays, since the v1 parser classifies `pubky.app` as foreign). +- **STRICTLY non-destructive.** Migration never deletes any legacy-epoch data. Privacy lost before + v1 is already lost; future activity is private. Private-tier legacy public copies are left inert + (the indexer stops surfacing them). +- **Deterministic and total** over real v0 data: each record sources from its highest present + epoch, transforms compose in memory writing only the latest, resume is by destination existence, + ids/hashes are re-derived not minted. +- **Deletion** is user-initiated only. Because migration copies and never deletes, a migrated + post lives at BOTH `pub/pubky.app/posts/{id}` and `pub/social/v1/posts/{id}/...`; deleting a + public object therefore removes every copy across epochs and both roots, or a from-scratch + reindex would resurrect it from the missed legacy copy; + private-tier deletes touch only the `/priv/` file. Absence is the tombstone: on a dumb blob + store the owner's files are the only durable state, so restoring an old backup republishes its + contents, accepted by design. Durable per-object tombstone files were considered and rejected + (the substrate cannot enforce them against the owner's own writes, and public tombstones would + leak deletion metadata forever). +- The indexer contract: dedup on `(author, resource_type, stable_id)`, tag id-sets (each tag edge keeps the set of ids asserting it across epochs, so un-tagging + removes all of them), intrinsic-time ranking (decode the post id), tombstone only when no publicly visible + copy survives. + +# Part D: Rollout and subtasks + +Spec crate on a long-lived `v1` branch, one PR per task, CI green on every commit, version +`1.0.0-alpha.N` until release. Each task is independently mergeable in this order. + +**Spine (serialized):** +- [ ] **S1** rename crate + types (wire-invariant). +- [ ] **S2** retire the wasm surface; native rlib; single-sourced DATA assets (limits, enum names). +- [ ] **S3** forward-compat contract (`Unknown` on every wire enum). +- [ ] **S4** validation core: limits table, canonical id validators, frozen text ops, mint guard. +- [ ] **S5** path epoch + canonicalizers + parser (atomic). +- [ ] **S6a** post wire shapes on the generic envelope (`PostEnvelope` + kind trait + social + alias; kinds, embed, attachments, Article envelope). +- [ ] **S6b** post storage, roots, lifecycle on the envelope, namespace-parameterized (versioned + builders, root rule, publish/unpublish/delete). +- [ ] **S7** tag + collection (canonical id inputs, reference-tier items). +- [ ] **S8** media collapse (single bytes object, MIME map, parser ext-strip). +- [ ] **S9** feed dual-root + the user profile field gates (image, links). +- [ ] **S10** private tier + bookmarks + settings. +- [ ] **S11** legacy_v0 module + cross-epoch normalization (`stable_id`/`resolve_deref`). + +**Pure-JS + gate:** +- [ ] **J1** Rust conformance-vector generator (byte-identity + verdict tiers, fuzzed). +- [ ] **J2** hand-written pure-JS package (ids, paths, canonicalizers, validation, builders). +- [ ] **J3** merge-blocking differential CI gate. + +**Migrator:** +- [ ] **M1** transform registry + v0 reader (Rust reference emits semantic vectors). +- [ ] **M2** engine (epoch discovery, resume, source re-check, abort-if-no-`/priv/`). +- [ ] **M3** pubky.app `/migrate` route (caps, upgrade flow, live counts, settings import). + +**Cross-repo:** +- [ ] **pubky-nexus:** per-epoch classify/adapt/normalize, permanent v0 parser, the deletion + flag (deletion keyed on real state, never name/content literals; gates v1 indexing), tag id-sets, + intrinsic-time ranking, multi-epoch tombstones, bookmark-feature retirement, mixed-epoch resync. +- [ ] **pubky-app:** v1 adoption (new caps, kind strings, own-tree legacy read union so + un-migrated users lose nothing, publish UI, media type threading, deletion engine). + - [ ] **moderation:** mixed epoch support by both nexus and homeserver synchronization services as well as by checkstep-request services + +**Release gates:** +- [ ] **IN-PRIV** verify the target homeserver runs the `/priv/` tier and permits the write + paths. Prerequisite for M2 onward and for any private-tier go-live, not a late release check. +- [ ] **REL** release `1.0.0` (crate + npm), merge `v1` to main after nexus dual-read is live. + +**Dependencies** (beyond the serialized spine order above): S11 needs S5. J1 needs S5 and +regenerates as later S tasks land; J2 needs J1; J3 needs J2 and is merge-blocking from then on. +M1 needs S11; M2 needs M1 and IN-PRIV; M3 needs M2 and J2. Nexus dual-read must be live before +any client writes v1 data; the pubky-app track needs J2 and that nexus gate. REL is last. + +**Acceptance, every task:** its listed artifact lands with the full CI bar green (fmt, clippy +with warnings denied, tests, doctests, feature checks, regenerate-and-diff on committed data +assets); from J3 on, additionally the Rust/JS differential gate. + +# Part E: Folds in and supersedes + +Supersedes the v1 roadmap #12. Resolves: #47 (json extensions), #48 (attachment type array), +#55 (content hash, deferred to v1.x as a flat optional sibling with the shape pinned here), #120 +(short-form URIs), #141 (reject non-canonical URIs). Mention-prefix cleanup (`pk:`) is handled +client-side and already shipped. + +No open design decisions remain; a carve-out allowing migration to delete some legacy data was +considered and rejected: migration is strictly non-destructive (Part C). + +# Appendix A: the v1 URI grammar (normative) + +The reference crate's parser and its committed conformance vectors are the executable +definition; this is the same grammar in human-readable form. Scope: this appendix governs PATHS +(where objects live and how path URIs classify); reference FIELD VALUES inside objects go +through the per-field scheme tiers described in Part B, not this grammar. + +## A1. URI forms and canonicalization + +Accepted input forms: `pubky://[/]` and the SDK short form `pubky[/]` +(scheme prefix case-sensitive; `Pubky://` is invalid). The canonical output form is always +`pubky://[/]`. + +Canonicalization is raw string work, byte-level, with no engine URL parser anywhere: + +- **Host:** exactly 52 z-base32 characters (lowercase alphabet `ybndrfg8ejkmcpqxot1uwisza345h769`), + final character `y` or `o` (the canonical-encoding rule: the trailing 4 pad bits must be zero). + A `@` or `:` anywhere in the host is invalid (no userinfo, no port). +- **Path segments** (split on `/`): reject an empty segment (this covers trailing slashes and + `//`), `.`, `..`, and any segment containing `%`, `?`, `#`, an ASCII control character, or a + frozen-whitespace code point. There is NO percent-decoding, NO case folding, and NO segment + normalization: what is stored is what was written. +- A bare host (`pubky://`) is valid and canonical. + +Failures here, plus the root check in A2 step 1, are the only HARD errors (in both cases no +resource with a visibility can be constructed); every other failure is a handled classification, +never an error and never a panic. + +## A2. Classification + +After canonicalization, split the path into segments and classify: + +1. An empty path (a bare host) classifies as **User** (public): the URI references the user + themselves. Otherwise segment 0 must be `pub` or `priv` (it becomes the resource's + visibility); anything else is a hard error. +2. Segment 1 not `social`: **Foreign** `{namespace, version?, rest}`. Foreign data is valid, + classified, and skipped by social readers; it is never an error. +3. Segment 1 `social`, segment 2 matching `v[0-9]+` but not the supported epoch: + **UnsupportedVersion** (a reader's "upgrade me" signal). +4. Segment 2 not an epoch segment: **Unknown**. +5. Otherwise dispatch on the remaining segments per the table below. Any non-match, any failed + id validation, any wrong-root spelling of a single-root resource, and any `_`-prefixed leaf + (reserved client-private names) is **Unknown**. + +## A3. Resource dispatch (owner-relative paths under `{root}/social/v1/`) + +| Resource | Path remainder | Roots | Leaf rule | +|---|---|---|---| +| User | `profile.json` | pub | none | +| Post (versionless reference) | `posts/{id}` | both | `{id}` = canonical TimestampId | +| Post (version) | `posts/{id}/{editId}.json` | both | both canonical TimestampIds | +| File (media) | `files/{hash}.{ext}` | both | strip exactly one extension, case-sensitively, ONLY if it is in the frozen extension set; remainder = canonical HashId; unknown or absent extension is Unknown | +| Tag | `tags/{id}.json` | pub | `{id}` = canonical HashId | +| Follow | `follows/{pk}.json` | pub | `{pk}` = canonical host key (as A1) | +| Mute | `mutes/{pk}.json` | priv | same | +| Bookmark | `bookmarks/{filename}.json` | priv | FORM check only: every character in the base64url alphabet, or `~` followed by 26 canonical Crockford characters; full round-trip validation is the reader's job | +| Feed | `feeds/{id}.json` | both | `{id}` = canonical HashId | +| Settings | `settings.json` | priv | none | +| LastRead | `last_read.json` | priv | none | + +`.json` stripping is exact and single: `follows/.json.json` leaves `.json`, which fails +key validation and classifies Unknown. + +## A4. Canonical id encodings + +An id is valid if and only if re-encoding its decoded bytes reproduces the input. Closed form: + +- **TimestampId** (post/edit ids): 13 chars of uppercase Crockford base32 + (`0123456789ABCDEFGHJKMNPQRSTVWXYZ`); final char in `{0,2,4,6,8,A,C,E,G,J,M,P,R,T,W,Y}` + (one trailing pad bit, must be zero). Lowercase and the alias letters `O/I/L/U` are invalid. +- **HashId** (tag/media/feed ids): 26 chars, same alphabet; final char in `{0,4,8,C,G,M,R,W}` + (two pad bits). +- **Host key**: as A1 (52 z-base32, final `y`/`o`). + +Time bounds are never checked at parse time; only canonicality is. + +## A5. Representative vectors + +`` is any valid host key, `TS` a canonical TimestampId, `H26` a canonical HashId. + +| Input | Result | +|---|---| +| `pubky/pub/social/v1/posts/TS` | Public Post reference (short form accepted) | +| `pubky:///priv/social/v1/posts/TS/TS.json` | Private Post version | +| `pubky://` | User (bare host) | +| `pubky:///pub/social/v1/files/H26.svg` | Public File, id `H26` | +| `pubky:///pub/social/v1/files/H26.JPG` | Unknown (extension set is case-sensitive) | +| `pubky:///pub/social/v1/posts/ts-lowercase` | Unknown (non-canonical id) | +| `pubky:///pub/social/v2/posts/TS` | UnsupportedVersion | +| `pubky:///pub/pubky.app/posts/TS` | Foreign | +| `pubky:///pub/social/v1/mutes/.json` | Unknown (mutes are priv-rooted) | +| `Pubky:///pub/social/v1/profile.json` | hard error (scheme case) | +| `pubky://user@/pub/social/v1/profile.json` | hard error (userinfo) | +| `pubky:///pub/social/v1/posts/../profile.json` | hard error (dot-dot) | +| `pubky:///dav/social/v1/profile.json` | hard error (unknown root) | diff --git a/docs/v0-vs-v1.md b/docs/v0-vs-v1.md new file mode 100644 index 0000000..1072307 --- /dev/null +++ b/docs/v0-vs-v1.md @@ -0,0 +1,457 @@ +# v0 vs v1, model by model + +> Companion to `rfc-v1-social-specs.md`: the same design, expanded. This document walks every +> model and explains each change: what it was in v0 (`pubky-app-specs` 0.6.0, verified against +> the July 2026 main), what it becomes in v1 (`pubky-social-specs` 1.0.0, epoch `social/v1`), +> and why the change is an improvement: which problem or requirement it tackles. + +--- + +## 0. Cross-cutting changes (apply to every model) + +These are listed once here; the per-model sections below only add what is specific to them. + +- **Namespace and epoch: `pub/pubky.app/` becomes `{pub|priv}/social/v1/`.** + Why: v0 hard-codes one app's domain as the location of SHARED social data and the parser + rejects every other app's path, contradicting multi-app interop, the project's first-class + goal. And v0 has no version signal at all, while the path is the only channel that survives + the events feed, LIST, and anonymous GET. Epochs in the path also make old and new data + physically coexist in disjoint subtrees, which is what makes non-destructive client-side + migration possible at all. A third, practical reason the new segment is a bare word: dotted + directory names are a filesystem hazard. Back up or export a homeserver tree to disk (pubky-app's + data-export ZIP preserves paths) and you get a folder literally named `pubky.app`, which macOS + treats as an application bundle: contents hidden behind an app icon, double-click tries to + launch it. `social` carries no extension, and no directory in the social tree ever will; only + leaf files carry extensions. +- **`.json` on every JSON leaf.** Why: the homeserver derives the served Content-Type from + magic bytes and then the path extension; extensionless JSON (everything in v0 except + `profile.json`) serves as `application/octet-stream`/plaintext. The extension fixes serving + and downloads for free, using machinery the server already has. +- **A privacy tier.** Why: v0 stores mutes, bookmarks, saved feeds, and the client's + settings world-readable and on the public events feed although their verified reader set is + exactly one, the owner's own client. `/priv/` (owner-only, excluded from public events, + shipped on pubky-core main) is their correct home. The rule that decides placement is the + ACTUAL reader set, not aspiration. +- **Type rename `PubkyApp*` to `PubkySocial*`, wire-invariant.** Why: the crate name and type + prefixes teach every integrator that one app owns shared data. Serde field names and enum + strings stay byte-identical: this break costs a compile, not a migration. +- **Forward-compat contract.** Every wire enum gains `#[serde(other)] Unknown`; no model + may ever use `deny_unknown_fields`; every future field must be optional, defaulted, and + skip-if-none. Unknown members are tolerated on read AND preserved on rewrite: every wire model + carries an opaque flattened catch-all map, so a rewriting client round-trips members it does + not understand (tolerating without preserving would let any older client silently destroy + every field added after it shipped). Why: v0 has exactly one `Unknown` catch-all (`PostKind`); adding a value to any + feed enum hard-crashes every old client on deserialize. This contract is what turns "break + once, then grow additively" from an aspiration into a property. The bound extensibility + needs: TOTAL object size is capped (posts 512 KiB, every other JSON resource 64 KiB; media + bytes keep their own media-size bound), checked before parsing on read and after building on + write; rewrites fetch from the homeserver rather than indexer views; a rewrite that cannot fit + the cap while preserving unknown members fails and surfaces rather than silently dropping + them; and the indexer carries unknown members verbatim in its views (readable by any client) + while indexing only adopted projections. Deliberate extensions SHOULD nest under the reserved `ext` member, defined once + as unvalidated third-party data: hostile input until an extension's own rules validate it. +- **Canonical-encoding id validation.** An encoded id is valid if and only if re-encoding its decoded + bytes reproduces the input, with closed-form regexes and final-char sets. Why: v0's validators + decode Crockford aliases (`O` as `0`, lowercase, a dangling final bit) and z-base32 dangling + bits, so one logical id has dozens of accepted spellings, each a DISTINCT homeserver key: + identity forks, dedup splits, and broken bytewise ordering. Verified empirically. The ed25519 + point check also leaves PubkyId validation: v0 ran it native-only, which made Rust and JS + disagree by construction. +- **Pinned text operations.** Trim uses a frozen whitespace table (a DATA asset); tag-label + lowercasing is ASCII-only; lengths are code points; all wire i64s must fit in 53 bits. Why: + engine trim/lowercase/Unicode tables differ between Rust and JS and across browser versions + (verified divergences on U+FEFF, U+0085, full-Unicode case mapping, i64 rounding), and + content-addressed ids freeze whatever functions feed them. Changing these later re-ids data, + a break fixable only by opening a new `social/vN` epoch, so they are pinned now, engine-free. +- **One URI grammar.** All `pubky://` validation goes through one raw-string + canonicalizer (accepts the SDK short form, #120; rejects userinfo, `..`, `%`, query, fragment, + whitespace, control, #141); web references are gated by a pinned regex. Reference field values are validated by per-field + scheme tiers: pubky-only, pubky+web, web-only, or universal (any scheme-shaped URI via the + pinned opaque gate); `url::Url` and WHATWG + `new URL()` leave the normative surface entirely. Why: v0's `url::Url` normalizes junk into + acceptance and rejects valid short forms, and two independent validation surfaces (parser vs + field validators) had already drifted; engine URL parsers cannot be version-pinned across + browsers. +- **The root rule.** A public-rooted object must never reference a priv-root pubky URI; + private objects may reference both roots. Why: such a reference dangles for every reader but + the owner and leaks the existence of a private path via the 401-vs-404 oracle. This is the one + now-or-never piece of the private tier: tightening validation later is epoch-class. +- **Folder ownership (the composition law).** The specification that defines an object + determines its storage namespace, never the application that writes it; apps compose spec + packages (each bringing capability scopes and path builders). App namespaces SHOULD carry an + epoch (`pub//v1/...`): unenforceable for foreign namespaces, but the parser already + surfaces the post-namespace segment from Foreign paths, so for conforming apps that segment IS + the version and the convention buys version-routing for free. `social/vN` itself is governed by the spec repo: a resource type exists exactly when the + released crate parses it. +- **No silent sanitize-rewrites.** v0 silently rewrote `[DELETED]` names to "anonymous", + truncated-then-blanked over-long `file.src`, and passed unparseable URLs through + `sanitize_url`. v1 makes invalid input a validation error. Why: silent rewrites hide bugs and + make two implementations disagree about what was stored. + +--- + +## 1. User (profile) + +v0: `pub/pubky.app/profile.json`, `{name, bio?, image?, links?, status?}`. +v1: `pub/social/v1/profile.json`, same fields. + +- **`image` explicitly allows `pubky`, `http`, `https` and uses the one shared image validator, + cap 300.** Why: pubky-app avatars ARE pubky URIs today (`pubky:///pub/.../files/` is what + the client writes into `image`, verified end-to-end); an http-only rule, which one design + draft proposed, would have invalidated every real avatar. One validator now covers + `user.image` and both `cover_image` fields, one rule for one kind of value. +- **The `[DELETED]` magic string dies entirely.** v0 silently rewrote a user named `[DELETED]` + to "anonymous" because the indexer keys its deletion handling on that literal (verified), so a + user carrying the name would be treated as deleted. v1 removes the rewrite with NO replacement + rule: `[DELETED]` is an ordinary legal name. The load-bearing requirement moves to the indexer + contract instead: **nexus (the shared indexer) must key deletion on a real flag** (`deleted` + on the indexed row) + before indexing v1 data, and display of deleted accounts becomes pure client presentation. A + magic string survives nowhere in the v1 wire rules. +- **`links[].url` validated by the pinned web gate instead of `Url::parse` + `sanitize_url`.** + Why: cross-implementation determinism (cross-cutting rationale above); v0's `sanitize_url` + passed invalid URLs through unchanged. +- Unchanged: field set, caps (name 3..50, bio 160, links 5 x {100, 300}, status 50), and wire + field names. The profile stays public: identity is the one thing that must be readable. + +## 2. Post + +v0: `pub/pubky.app/posts/{id}`, one flat file, overwritten on edit; +`{content, kind, parent?, embed?: {kind, uri}, attachments?: [String], lock?}`. +v1: `{pub|priv}/social/v1/posts/{id}/{editId}.json`, referenced versionlessly as +`posts/{id}`. + +- **The post becomes a reusable envelope.** The shared mechanics (versioned storage, ids, + references, attachments, lock, preservation) ship as a generic crate layer that app specs + specialize with their own kinds in their own namespaces; the social post is the first + specialization, wire-identical to the shape described here. Why: the consumers are real + (Mapky, Eventky), the layer costs no wire change, and it turns the governance incubation path + into something usable at launch. +- **Per-edit path versioning.** Storage is one file per edit; the first version reuses the + post id; every reference (reply, embed, tag, bookmark, collection item) uses the versionless + form. Why: v0 edits overwrite in place, so nothing distinguishes an edit from a new post on + the events feed, and there is no history. A counter-based scheme was rejected because the + substrate has no compare-and-swap: counters race across the owner's devices. TimestampId + editIds mint with no prior read, sort chronologically under bytewise LIST order, and + `decode(editId)` doubles as an approximate edit timestamp. Versionless references mean edits + never orphan replies or re-key tags. This is now-or-never: a `posts/{id}` file and a + `posts/{id}/` directory are mutually exclusive on the homeserver, so the layout cannot be + retrofitted. +- **Dual-root: private posts.** Drafts, personal notes, and private collections live under + `/priv/` with identical shapes and ids; publish is a deterministic root migration; unpublish + deletes the public copies. Why: drafts and private collections are committed + product needs, and the Locks flow independently demonstrates the value of post-shaped private + content. Public stays the default root, so this adds capability without changing anyone's + existing mental model. +- **Kinds renamed: `short` to `note`, `long` to `article`.** Why: the old names describe length, + not nature; `article` is what the thing actually is, and it now has a typed envelope (model 3) + instead of a hand-rolled convention. All seven concrete kinds survive (each has a live + creation path in pubky-app); `Link` deliberately stays untyped because the URL lives inside the + content text, so there is no per-kind shape decision being missed. +- **`embed` collapses from `{kind, uri}` to a plain URI string, and accepts ANY external + target (the universal tier: http/https strictly gated, any other scheme via a pinned opaque + gate).** Why: the embedded target's kind is derivable by resolving the target; + storing it duplicates state that can go stale. External embeds make quoting a web resource + first-class (the indexer reuses the External Resource nodes it already builds for external tag + targets), and they keep migration total: v0's `Url::parse` gate accepted arbitrary embed URLs, + so real v0 posts can carry them. `parent` stays pubky-only: reply threads are social-graph + edges between posts. +- **`attachments` becomes `Vec<{uri, alt?, name?}>`, always present, default `[]` (#48).** Why: v0's + `Option>` made every consumer branch on null-vs-empty; and a bare string array can + never grow per-item metadata without a breaking change. The object form makes alt text (a + committed accessibility need) and `name` (the display filename, relocated from the deleted v0 + File object by the media collapse) ship now, and every future per-attachment field (hash, + blurhash, dimensions, `content_type`/`size`) additive. Exactly two optional fields ship; + nothing speculative. +- **`lock` is kept with corrected semantics.** The value is the lock FILE URI + (`pubky:///pub/app.locks/.json`, illustrative), pubky-only; presence means "locked + content" regardless of kind. Why: this matches the resolved Locks design (pubky-app #2029); + earlier drafts (including v0 doc comments) mis-described it as a lock-server URI. The client-owned + teaser object inside a locked post's `content` stays deliberately client-owned, per the Locks + team's own recorded decision. +- **Reference fields get one shared cap (1024) and canonicalization.** Why: v0 was a mix + (attachment URI 200, src 1024, parent/embed/lock effectively uncapped or 200); one + `reference_uri_max_length` replaces four inconsistent rules. +- **The `[DELETED]` content sentinel is removed; absence is the tombstone.** Why: a + magic content string must not survive the one clean break; deletion is now defined honestly as + "delete every version in every epoch and both roots, retry to completion", with nexus + synthesizing its own tombstones from DELETE events and real deletion state, never from content + strings (the same required flag upgrade as the user model). + +## 3. ArticleContent (new model) + +v0: none. pubky-app hand-rolls `JSON.stringify({title, body})` into `long` posts, unspecified, +with the cover image smuggled as `attachments[0]` (both verified in the client). +v1: `PubkySocialArticleContent {title, body, cover_image?}`, a typed content envelope in +`content` when `kind == article` (a per-kind schema inside the content string, distinct from +the `PostEnvelope` mechanics layer). + +- **The envelope exists at all.** Why: an unspec'd JSON convention inside a spec'd field is + interop debt; any other client rendering articles must reverse-engineer pubky-app. Per-kind + content shapes are now-or-never (changing what `content` means for a kind is a break), so this + is exactly what the one break budget is for. +- **`cover_image` moves INTO the envelope.** Why: `attachments[0]`-as-cover is positional + convention, invisible in the type system; the field is explicit, validated by the shared image + validator, and composes with real attachments. Migration maps the old convention in. +- **Articles may carry `parent`, `embed`, and `attachments`.** Why: an article can legitimately + be a reply or a quote and carry media; and a forbid rule would have made any v0 long post + holding more than the single mapped cover attachment FAIL migration. Composability plus + migration totality. +- **Caps: title 100 code points, body 50000 (renamed from `post_long_content_max_length`), raw + envelope 52000.** Why: the title cap formalizes what pubky-app's UI already enforces; the raw cap + is a cheap pre-parse bound. pubky-app's brittle "body budget = long cap minus 100 minus 22 bytes + of JSON skeleton" arithmetic dies because validation is field-level now. + +## 4. CollectionContent + +v0: `PubkyAppCollectionContent {name, description?, items[], cover_image?}` in `content` when +`kind == collection` (shipped shortly before v1). +v1: same shape, three rule changes. + +- **`items` accept any reference-tier pubky URI (any resource, any app).** Why: v0's item check + hard-restricted items to `posts/` under `pubky.app`, contradicting the interop goal; a curated + list may legitimately include files, profiles, or another app's resources. Items stay + pubky-only (a web link belongs in a post) and stay plain strings (per-item annotation is + speculative, recorded as an accepted low-probability future break). +- **`cover_image` uses the shared image validator (cap 300).** Why: one rule for one kind of + value, aligned with Article and `user.image`. +- **Private collections come free.** A collection post under `/priv/` is a private + curation list, a requested future feature that costs zero extra spec surface because + collections are posts. +- Unchanged: the v0 guards (collections carry no parent, embed, or attachments) because they are + live in v0 and cost migration nothing. + +## 5. Tag + +v0: `pub/pubky.app/tags/{id}`, `{uri, label, created_at}`, id = `HashId("{uri}:{label}")` where +the uri was normalized by `Url::parse(...).to_string()` and the label by full-Unicode lowercase. +v1: `pub/social/v1/tags/{id}.json`, same fields. + +- **The hash input is pinned: canonicalized target + frozen-trim + ASCII-folded label.** Why: + v0's normalization was engine-dependent twice over (`url` crate re-serialization and ICU case + tables), and content-addressed ids freeze their input functions forever; an engine-skew fork + in a tag id never heals. The ASCII-only fold trades non-Latin case-insensitive dedup (bounded + regression, documented) for permanent cross-implementation determinism. +- **The injectivity invariant is stated:** `"{uri}:{label}"` is unambiguous only because labels + reject `:`; that restriction may never be lifted while the id format stands. Why: implicit + invariants get broken by well-meaning future edits. +- **`uri` gains the shared 1024 cap** (was uncapped) **and hash inputs always use the + canonicalizer's output** (v0's bookmark hashed the RAW string while tag hashed a normalized + one, two different identity rules for the same kind of value). +- **Migration non-invariance is documented:** a tag id embedding a social target changes + across epochs by construction; nexus dedups on `(author, normalized target, label)` with an + id-SET per edge so cross-epoch un-tagging works. Why: without this, dual-read double-counts + every tag and a "like" placed before migration can never be removed after it. +- **One write location, any target.** Every app writes tags at the author's + `pub/social/v1/tags/` (folder ownership above); the target may be any public pubky resource or + ANY external URI (the universal tier: http/https via the strict web gate, other schemes like + `nostr:`/`geo:`/`ipfs:` via the pinned opaque gate). One logical tag has exactly one address, + so re-tags self-overwrite and the indexer drops the writing-app dimension for v1 data; tag + files under other app namespaces survive as a legacy READ rule only. Because addresses + converge, a tag writer SHOULD GET the address first and preserve unknown members if a file + exists: a blind PUT would destroy another app's enrichment (e.g. `ext.badge`) of the same + statement. +- Tags stay public-only in v1; tagging private objects is deferred (dual-rooting a resource + later is additive). + +## 6. Bookmark + +v0: `pub/pubky.app/bookmarks/{HashId(raw uri)}`, content `{uri, created_at}`: world-readable, +and the filename is one-way, so listing your bookmarks costs one GET per bookmark. +v1: `priv/social/v1/bookmarks/{filename}.json`. + +- **Targets take the universal tier** (any public pubky resource or any external URI), same + domain as tags; over-cap and exotic targets all representable (overflow form below). +- **Moves to `/priv/`.** Why: what you saved is personal state with zero cross-user readers + (verified: even pubky-app reads bookmark state via nexus, which only surfaces it to the owner); + world-readable bookmarks are a privacy leak. +- **The target moves into the filename, reversibly (the primary form):** `base64url_nopad(canonical target)` for + targets up to 187 bytes. Why: LIST returns keys only, so a reversible filename makes "list all + my bookmarks" ZERO GETs (v0's defining defect, #47's sibling). 187 bytes is the exact + substrate maximum (250 chars + `.json` = 255), and at that cap base64url is the ONLY standard + encoding that fits at all. base64url is also `/`-free, `%`-free, JS-decodable natively, and + already in the SDK dependency stack. +- **An overflow form for long targets:** `~ + HashId(target)` with the target kept in content, + from 188 bytes up to the shared 1024 code-point reference cap. Why: real bookmarks exceed 187 bytes (maps and shop URLs); without an + overflow they could not be represented at all under the reversible form. `~` is outside the + base64url alphabet, so the two forms are unambiguous. +- **Content shrinks to `{created_at}`** (plus `target` only in overflow). Why: the target lives + in the filename; duplicating it invites mismatch. Stated honestly: recency SORT still costs + GETs (created_at is in content), and each OVERFLOW entry costs one GET to recover its target; + primary-form listing is zero GETs. +- **The hash/encoding input is the canonicalizer's output, never the raw spelling.** Why: v0 + hashed the raw string, so two spellings of one URL made two bookmarks. +- **Read-side rules are pinned:** decode-then-re-encode must reproduce the filename + byte-for-byte, UTF-8 decode is fatal, failures are skipped entries. Why: JS base64 decoding is + catastrophically lenient (verified: it accepts padding, aliases, and garbage the Rust crate + rejects), and a hostile filename must not fork implementations. + +## 7. Follow + +v0: `pub/pubky.app/follows/{followeePk}`, `{created_at}`. +v1: `pub/social/v1/follows/{followeePk}.json`. Shape unchanged. + +- Only the cross-cutting changes apply (path, `.json`, canonical PubkyId spelling). Follows stay + public: they are the social graph, nexus's core input. The filename-is-the-target pattern + (one LIST answers "who do I follow") was already right in v0 and is kept. + +## 8. Mute + +v0: `pub/pubky.app/mutes/{muteePk}`, `{created_at}`, world-readable. +v1: `priv/social/v1/mutes/{muteePk}.json`. Shape unchanged. + +- **Moves to `/priv/`.** Why: who you muted is among the most sensitive social data there is, + and its verified reader set is the owner alone: nexus main has ZERO mute consumers (the + watcher no-ops mute events), and pubky-app already reads mutes by LISTing its own directory. The + move costs nexus nothing and the client a path change. + +## 9. LastRead + +v0: `pub/pubky.app/last_read` (no extension), `{timestamp}` in MILLISECONDS, world-readable. +v1: `priv/social/v1/last_read.json`, microseconds. + +- **Moves to `/priv/`.** Why: pure reading-activity metadata, no cross-user reader. +- **Milliseconds become microseconds.** Why: it was the lone unit outlier in a spec where every + other timestamp is microseconds; a unit change is only fixable at a break, so this is the + window. Migration multiplies by 1000, the only unit change in the shared transform table + (the pubky-app-owned settings import performs the same ms-to-µs conversion on its side). + +## 10. File (media), the v0 File + Blob pair collapsed + +v0: TWO objects per upload: `files/{id}` metadata (`{name, created_at, src, content_type, +size}`, with a HARD 21-entry MIME whitelist gate and a silent truncate-then-blank sanitize on +`src`) pointing at `blobs/{HashId(bytes)}`, extensionless raw bytes. +v1: ONE object: `{pub|priv}/social/v1/files/{hash}.{ext}`, the raw bytes, content-addressed. +The metadata sidecar is deleted. + +- **The collapse itself.** Why: both premises of the v0 split died. `name` now has a better home, + the attachment object's optional `name` field (per-reference, so two posts can attach the same + bytes under different names, which one-name-per-File could not express); authoring time is + carried by the referencing post's own id; `size` and the served type come free from the bytes + and headers (nexus downloads the bytes anyway to build CDN variants); and the v1 nexus adapter + is new code regardless, so "nexus is event-driven off the File PUT" stopped being an argument. + Deleted with it: the `src` indirection, the two-PUT upload dance, the several-metadata-objects- + per-blob extension ambiguity, and the word "blob" (a backup now shows a `files/` folder holding + openable `{hash}.jpg` files). Honest costs, accepted: renaming an upload means editing the + referencing post; the same bytes declared under two MIMEs duplicate storage instead of sharing + one blob (rare, documented fork); an uploaded-but-never-referenced file carries no metadata. +- **A canonical, path-only extension from a frozen MIME-to-ext map.** Why: the homeserver ignores + the PUT Content-Type and derives the stored type from magic bytes, then the path extension + (verified, `file_metadata.rs`); sniff-miss text types (svg, csv, txt, json, html, xml) served + wrong in v0. The declared type is consumed exactly once, at upload, to derive `{ext}` via a + pinned essence regex (the essence is the bare type/subtype, parameters stripped) + + single-valued map; it is never stored. The ext is NEVER part of the + hash, so the content address and dedup are untouched; `.bin` is the total fallback. +- **The MIME whitelist gate is removed.** Why: a closed list on world-readable content is a + forward-compat trap (it already rejected avif, heic, webm audio, opus, wasm); and the Rust + `mime` crate could not stay the judge because its verdicts are not reproducible in JS and it + accepts the malformed `"image/"` (verified). The old list survives as an advisory hint. +- **The parser strips exactly one known-map extension before id validation.** Why: without the + strip, recompute-and-compare id validation would reject every single media file. +- **Dual-root.** Why: a private draft whose images sat in public `files/` would leak (media + directories are anonymously LISTable). Publish copies bytes across roots identically (the hash, + and therefore the id, is root-independent). + +## 11. Feed + +v0: `pub/pubky.app/feeds/{HashId(serde_json(config))}`, `{feed: config, name, created_at}`, +public, enums crash on unknown values. (Current v0 also carries the `wot`/`me` reaches and an +optional `domain_tags` filter, added mid-2026 in #143; v1 includes all three, and `domain_tags` +joins the id input as its own trailing segment.) +v1: `{priv|pub}/social/v1/feeds/{id}.json`. + +- **Private by default, published by choice (dual-root).** Why: a saved feed is a personal + config whose verified reader set is the owner (pubky-app only ever LISTs its own feeds dir; no + share or subscribe feature exists); public-by-default was aspiration, not fact. Publishing is + a deliberate act: copy the same bytes to `/pub/`, unpublish deletes the copy. +- **The id stays content-addressed but the hash input is a pinned canonical string, never serde + output.** Why the content-addressing: identical configs self-overwrite + (natural dedup), migration re-derives ids purely, and two users publishing the same config + share an id, so public-feed identity is readable from `/events/` paths alone, which makes a + future cross-homeserver popularity ranking purely additive indexer work. Why the pinned + string: v0 hashed `serde_json::to_string(config)`, which silently re-ids every feed on any + struct reshuffle and is not byte-reproducible in JS. `name` and `created_at` stay outside the + hash: personal labels on a shared identity. Tags sort inside the input so `[a,b]` and `[b,a]` + are one filter; the format is injective because `:` and `,` are invalid in labels. +- **All three enums gain `Unknown`; an unknown `content` filter degrades to "no filter".** Why: + the highest value-per-byte fix in v1. In v0, the day a new reach/layout/sort value ships, + every old client hard-crashes on deserialize; this is the one guarantee only a version + boundary can make. +- **`name` gains a cap (100).** Why: it was the only unbounded display name. +- **Edit semantics stated honestly:** a config change derives a new id (write new, delete old, + re-publish if desired); pubky-app's v0 flow already works exactly this way, including the + known orphan-file behavior, now documented instead of accidental. + +## 12. Settings (new model) + +v0: none in the spec. pubky-app hand-rolls `pub/pubky.app/settings.json`: WORLD-READABLE, exposing +`require_pin`, `sign_out_inactive`, and the rest of the user's privacy posture to anyone, read +by nobody but the owner's client (verified: the only unspec'd homeserver artifact in the app). +v1: `PubkySocialSettings` at `priv/social/v1/settings.json`. + +- **It exists, and it is private.** Why: the strongest reader-set case in the audit; a security + posture file must not be public. Spec'd because its content (notification, content-filter, + and language preferences) is client-portable social state any client benefits from sharing. +- **Every section is optional.** Why: a client writes only what it uses; other clients' unknown + sections and fields survive a rewrite via the preservation rule (cross-cutting above), not + merely deserialization tolerance, which alone would drop them on the next whole-file write. +- **Whole-file last-write-wins on `updated_at` (now microseconds).** Why: it formalizes exactly + what pubky-app already does at bootstrap; anything cleverer (field-wise merge) is machinery + without a demonstrated need. +- **The per-file `version` field is dropped.** Why: verified dead, pubky-app checks it but never + bumps it; schema evolution is governed by the path epoch and crate semver like every other + model, so a second, parallel versioning channel is a contradiction waiting to happen. + +## 13. Parser and `Resource` (the read side of every model) + +v0: `url::Url`-based, hard-rejects any app path that is not `pubky.app`, silently accepts +userinfo/`..`/query/fragment/extra segments, never validates ids, never panics, but errors on +all foreign data. +v1: one closed grammar (normative form: Appendix A of `rfc-v1-social-specs.md`). + +- **`Foreign` and `UnsupportedVersion` are first-class handled categories, never errors.** Why: + v0's defining interop failure was erroring on other apps' data; an indexer iterating the + events feed must classify and skip, not crash or log-spam. A future `social/v2` object reads + as "upgrade me", not garbage. +- **Failed id or format validation yields `Unknown`, never an error; every access is + bounds-safe.** Why: the parser's consumers run unattended over hostile input forever; the two + hard errors that remain (an uncanonicalizable URI: bad scheme case, bad host, userinfo, + dot-dot, or an unknown root) exist only because no `ParsedUri` can be + constructed, and callers treat them as skips. +- **Wrong-root parses to `Unknown` for single-root resources; `visibility` carries the root for + the dual-root three.** Why: "private object at a public path" becomes unrepresentable at the + type level, the one structural guard the homeserver cannot give. +- **Reserved names: `^v[0-9]+$` epoch segments, `_`-prefixed private filenames, the `ext` + field.** Why: reserving names is free; colliding with them later is not. + +## 14. IDs (the identity layer under every model) + +- **TimestampId: unchanged format, monotonic mint guard in BOTH implementations.** Why: + the JS runtime mints at millisecond resolution, so same-ms writes collide on path, which + under path-versioning is silent data loss; the guard is three lines. The JS codec uses BigInt, + byte-for-byte equal to Rust (verified including `i64::MAX`). +- **HashId: unchanged format (blake3, first 16 bytes, Crockford, 26 chars), kept at 128 bits.** + Why: under owner-only-write, pubkey-namespaced paths, a collision cannot substitute content at + someone else's path, and per-user counts are nowhere near the birthday bound; widening buys + nothing and costs every path 13 chars. +- **Canonical spellings enforced everywhere** (cross-cutting above), and the stale "z-base32" + doc comments on Crockford code die. + +--- + +## The deltas at a glance + +| Model | v0 path | v1 path | Headline change | +|---|---|---|---| +| User | `pub/pubky.app/profile.json` | `pub/social/v1/profile.json` | pubky avatars legal; `[DELETED]` sanitize gone | +| Post | `posts/{id}` flat file | `{root}/.../posts/{id}/{editId}.json` | edit versioning; dual-root drafts; kinds renamed; attachments become objects | +| Article | (hand-rolled JSON) | typed envelope | formalized; cover in envelope | +| Collection | envelope, posts-only items | envelope, reference-tier items | interop items; private collections free | +| Tag | `tags/{id}` | `tags/{id}.json` | engine-free pinned hash input | +| Bookmark | `bookmarks/{HashId}` public, GET-per-file | `priv/.../bookmarks/{b64u\|~hash}.json` | private, reversible filename, overflow form | +| Follow | `follows/{pk}` | `follows/{pk}.json` | unchanged shape | +| Mute | `mutes/{pk}` public | `priv/.../mutes/{pk}.json` | private | +| LastRead | `last_read` ms, public | `priv/.../last_read.json` µs | private; unit fixed | +| File (media) | `files/{id}` meta + `blobs/{hash}` bytes | `{root}/.../files/{hash}.{ext}` | ONE object (collapse); canonical extension; dual-root | +| Feed | `feeds/{HashId(serde_json)}` public | `{priv\|pub}/.../feeds/{id}.json` | private default + publish; pinned hash string; enums get Unknown | +| Settings | (unspec'd, public) | `priv/.../settings.json` | spec'd, private, version field dropped |