Skip to content

sccache-dist for OpenEmbedded/Yocto: upstreaming tracker #1

Description

@jetm

Tracks upstreaming the sccache-dist changes that make distributed compilation
work for OpenEmbedded/Yocto host-mode builds. The original single PR
(mozilla#2750, closed) grew to ~40 commits; this splits it into small,
independently reviewable PRs against mozilla/sccache, landed in waves.

All branches are rebased on current upstream main and build/test green
(cargo check --all-features --all-targets, full test suite, fmt, clippy).

Plan

Legend: fill the PR link as each opens. Lane A is independent; the scheduler work
is a base-first stack (9 -> 8a -> 8b -> 8c -> 8d -> 6 -> 7).

Wave 1 - credibility

Wave 2 - fix surface

  • PR 3 dist/pkg: toolchain packaging correctness (dist-pkg-toolchain-packaging, depends PR 5)
  • PR 4 compiler/gcc: distributed-compile correctness and force-local policy (compiler-gcc-dist-correctness)
  • PR 5 compiler/rust: make distributed compiles resolve std and --target (compiler-rust-dist-std-target)

Wave 3 - scheduler stack (base-first)

  • PR 9 dist: report a per-server breakdown in scheduler status (dist-scheduler-per-server-status)
  • PR 8a dist/scheduler: drop a stopped server promptly (dist-scheduler-drop-stopped-server, on 9)
  • PR 8b dist/scheduler: topology-aware, deterministic routing (dist-scheduler-topology-routing, on 8a)
  • PR 8c dist/scheduler: idempotent updates and lost-job reaping (dist-scheduler-idempotent-reaping, on 8b)
  • PR 8d dist/scheduler: liveness-lease reaping and restart-safe accounting (dist-scheduler-liveness-lease, on 8c) - wire-format change
  • PR 6 dist/http: reuse connections and offload large deserialize (dist-http-connection-reuse, on 8d) - new config keys + arc-swap
  • PR 7 dist/server: shrink toolchain locks so compiles stop convoying (dist-server-toolchain-locks, on 6)

Wave 4 - optional / chore

  • PR 10 dist: load and dispatch instrumentation (dist-load-instrumentation, depends PR 2, ask-first)
  • compiler: concatenate strings without relying on deref coercion (compiler-string-concat-fix)
  • ci: mirror the CI validation gate in pre-commit (ci-precommit-gate)
  • chore: fix clippy lints flagged by a newer toolchain (chore-clippy-newer-toolchain)

Discussion anchor

Upstream RFC issue (scheduler arc + instrumentation ask): mozilla#2775.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions