Skip to content

docs(book): add DisTrO explainer chapter - #651

Open
salch-cred wants to merge 1 commit into
PsycheFoundation:mainfrom
salch-cred:docs/distro-explainer
Open

docs(book): add DisTrO explainer chapter#651
salch-cred wants to merge 1 commit into
PsycheFoundation:mainfrom
salch-cred:docs/distro-explainer

Conversation

@salch-cred

Copy link
Copy Markdown

Summary

Adds a new "Psyche in depth" chapter: DisTrO (explain/distro.md).

DisTrO was previously only referenced in passing — a box inside the decentralized training flow diagram in general-workflow.md and an optimizer stanza in the run-config docs — with no explanation of what it actually is or does. This chapter explains, from the Rust implementation in shared/modeling/src/distro.rs:

  • The problem: gradient synchronization bandwidth makes conventional data-parallel training impossible between arbitrary internet nodes.
  • The two ideas that fix it: transform-based compression (DCT + top-k per chunk) and residual accumulation, so small components dropped by top-k are deferred into later steps instead of being lost.
  • A per-step walkthrough (mermaid diagram + numbered steps): decode peers' updates, apply aggregate, compute gradient, update the momentum residual (subtracting what was already sent to avoid double counting), encode, top-k, optional 1-bit sign quantization, broadcast over P2P.
  • Where it fits: the optimizer handles all weight synchronization client-to-client; the Solana coordinator only handles the state machine, data assignments, and witness proofs — weight data never goes on-chain.
  • Configuration: a table mapping compression_decay, compression_chunk, compression_topk, and quantize_1bit to their effect and their interaction with max_round_train_time.

Notes for reviewers

  • All technical claims were verified against the code: Distro::generate / Distro::apply / error_correction in shared/modeling/src/distro.rs, the TransformDCT/CompressDCT encode/compress path, the Distro optimizer definition in shared/core/src/definitions.rs, and the quantization path in shared/modeling/src/trainer.rs (quantize_1bitquantize_nozeros_tensor_to_boolean_sign).
  • Only additions: one new page + one SUMMARY.md line.
  • Happy to adjust tone, depth, or placement.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant