diff --git a/AGENTS.md b/AGENTS.md index 6e5a58e76777..d2b45616892d 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -133,6 +133,55 @@ test/lint/lint-circular-dependencies.py Functional-test prerequisites and usage details live in `test/README.md`. Several Dash-specific tests need the `dash_hash` Python package. +### Running Linters Exactly Like CI + +Running `test/lint/*` scripts directly on the host is fine when the locally +installed tools behave. If host results disagree with CI (version drift in +codespell/flake8/mypy/shellcheck, or a lint passing locally but failing in +CI), reproduce the CI lint job (`ci/dash/lint.sh` with CI-pinned tool +versions) with the lint container: + +```bash +# One-time setup; rebuild only when ci/lint/ changes. +docker build --platform=linux/amd64 -t dash-linter ci/lint +``` + +**Commit or stash tracked changes before running.** With `PULL_REQUEST=true`, +the range is fed to `test/lint/commit-script-check.sh`, which checks out +commits and runs `git reset --hard` while replaying `scripted-diff:` commits. +The command below refuses to start with tracked changes present. Only run it +against commits you trust because scripted-diff verification commands are +executed inside the container. + +```bash +( git diff --quiet && git diff --cached --quiet || { echo "Commit or stash tracked changes before running the lint container." >&2; exit 1; }; G="$(git rev-parse --path-format=absolute --git-common-dir)"; docker run --rm --platform=linux/amd64 --user "$(id -u):$(id -g)" -e HOME=/tmp -v "$PWD":"$PWD" -v "$G":"$G" -w "$PWD" -e BUILD_TARGET=linux64 -e CHECK_DOC=1 -e PULL_REQUEST=true -e COMMIT_RANGE="$(git merge-base develop HEAD)..HEAD" dash-linter bash -c 'git config --global --add safe.directory "$PWD" && ./ci/dash/lint.sh' ) +``` + +Notes: + +- The dual mount (`$PWD` plus the git common dir) makes this work from git + worktrees as well as normal checkouts; run it from the repo/worktree root. +- The image selects the x86_64 or aarch64 ShellCheck payload for the target + architecture, matching the multi-architecture support in CI's `ci-slim` + container. +- `--user` plus `HOME=/tmp` keeps generated caches and temporary git config + writable without leaving root-owned files in the bind-mounted worktree. +- `COMMIT_RANGE` is computed against your local `develop`; make sure it exists + and is current with `dashpay/dash` (CI computes the range against a freshly + fetched `origin/develop`). +- Do not use the bare `docker run ... dash-linter` flow from + `test/lint/README.md` to reproduce CI: its default entrypoint merge-bases + against `master` instead of `develop` and runs `check-doc.py` plus + git-subtree checks that the CI lint job does not run. +- The image builds cppcheck from source at the same version as CI's `ci-slim` + container, so the cppcheck lint runs locally too; expect the first + `docker build` to take a few minutes. +- Codespell warnings are non-fatal in CI; pre-existing hits on `develop` are + expected. +- The run creates a `ci-cache-/` directory in the repo root + (same CI env default as the real job). Keep it for faster cppcheck reruns + or delete it freely; it is untracked. + ## Backport Work Dash Core regularly backports Bitcoin Core changes. Treat backports as diff --git a/CLAUDE.md b/CLAUDE.md index 6e5a58e76777..d2b45616892d 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -133,6 +133,55 @@ test/lint/lint-circular-dependencies.py Functional-test prerequisites and usage details live in `test/README.md`. Several Dash-specific tests need the `dash_hash` Python package. +### Running Linters Exactly Like CI + +Running `test/lint/*` scripts directly on the host is fine when the locally +installed tools behave. If host results disagree with CI (version drift in +codespell/flake8/mypy/shellcheck, or a lint passing locally but failing in +CI), reproduce the CI lint job (`ci/dash/lint.sh` with CI-pinned tool +versions) with the lint container: + +```bash +# One-time setup; rebuild only when ci/lint/ changes. +docker build --platform=linux/amd64 -t dash-linter ci/lint +``` + +**Commit or stash tracked changes before running.** With `PULL_REQUEST=true`, +the range is fed to `test/lint/commit-script-check.sh`, which checks out +commits and runs `git reset --hard` while replaying `scripted-diff:` commits. +The command below refuses to start with tracked changes present. Only run it +against commits you trust because scripted-diff verification commands are +executed inside the container. + +```bash +( git diff --quiet && git diff --cached --quiet || { echo "Commit or stash tracked changes before running the lint container." >&2; exit 1; }; G="$(git rev-parse --path-format=absolute --git-common-dir)"; docker run --rm --platform=linux/amd64 --user "$(id -u):$(id -g)" -e HOME=/tmp -v "$PWD":"$PWD" -v "$G":"$G" -w "$PWD" -e BUILD_TARGET=linux64 -e CHECK_DOC=1 -e PULL_REQUEST=true -e COMMIT_RANGE="$(git merge-base develop HEAD)..HEAD" dash-linter bash -c 'git config --global --add safe.directory "$PWD" && ./ci/dash/lint.sh' ) +``` + +Notes: + +- The dual mount (`$PWD` plus the git common dir) makes this work from git + worktrees as well as normal checkouts; run it from the repo/worktree root. +- The image selects the x86_64 or aarch64 ShellCheck payload for the target + architecture, matching the multi-architecture support in CI's `ci-slim` + container. +- `--user` plus `HOME=/tmp` keeps generated caches and temporary git config + writable without leaving root-owned files in the bind-mounted worktree. +- `COMMIT_RANGE` is computed against your local `develop`; make sure it exists + and is current with `dashpay/dash` (CI computes the range against a freshly + fetched `origin/develop`). +- Do not use the bare `docker run ... dash-linter` flow from + `test/lint/README.md` to reproduce CI: its default entrypoint merge-bases + against `master` instead of `develop` and runs `check-doc.py` plus + git-subtree checks that the CI lint job does not run. +- The image builds cppcheck from source at the same version as CI's `ci-slim` + container, so the cppcheck lint runs locally too; expect the first + `docker build` to take a few minutes. +- Codespell warnings are non-fatal in CI; pre-existing hits on `develop` are + expected. +- The run creates a `ci-cache-/` directory in the repo root + (same CI env default as the real job). Keep it for faster cppcheck reruns + or delete it freely; it is untracked. + ## Backport Work Dash Core regularly backports Bitcoin Core changes. Treat backports as diff --git a/ci/lint/04_install.sh b/ci/lint/04_install.sh index 9ca5ebe37216..ed369bce6138 100755 --- a/ci/lint/04_install.sh +++ b/ci/lint/04_install.sh @@ -42,6 +42,15 @@ ${CI_RETRY_EXE} pip3 install pyzmq==24.0.1 ${CI_RETRY_EXE} pip3 install vulture==2.6 SHELLCHECK_VERSION=v0.8.0 -curl -sL "https://github.com/koalaman/shellcheck/releases/download/${SHELLCHECK_VERSION}/shellcheck-${SHELLCHECK_VERSION}.linux.x86_64.tar.xz" | \ +ARCH_INFERRED="${TARGETARCH}" +if [ -z "${ARCH_INFERRED}" ]; then + ARCH_INFERRED="$(dpkg --print-architecture || true)" +fi +case "${ARCH_INFERRED}" in + amd64|x86_64) SC_ARCH="x86_64" ;; + arm64|aarch64) SC_ARCH="aarch64" ;; + *) echo "Unsupported architecture for ShellCheck: ${ARCH_INFERRED}"; exit 1 ;; +esac +curl -sL "https://github.com/koalaman/shellcheck/releases/download/${SHELLCHECK_VERSION}/shellcheck-${SHELLCHECK_VERSION}.linux.${SC_ARCH}.tar.xz" | \ tar --xz -xf - --directory /tmp/ mv "/tmp/shellcheck-${SHELLCHECK_VERSION}/shellcheck" /usr/bin/ diff --git a/ci/lint/Dockerfile b/ci/lint/Dockerfile index 76f9c27b06ca..4e4ac23f8da8 100644 --- a/ci/lint/Dockerfile +++ b/ci/lint/Dockerfile @@ -5,11 +5,37 @@ # entire repo as docker context during build; if it lived elsewhere, it wouldn't be # able to make back-references to pull in the install scripts. So here it lives. +# Builder for cppcheck; keep the version in sync with +# contrib/containers/ci/ci-slim.Dockerfile so results match the CI lint job. +FROM debian:bookworm-slim AS cppcheck-builder +ARG CPPCHECK_VERSION=2.21.0 +ARG CPPCHECK_ARCHIVE_SHA256=f028ff75ca5372738f3737c8b3e8611426a6526b6aea2ef01301ab0f5902f044 +RUN set -ex; \ + apt-get update && apt-get install -y --no-install-recommends \ + curl \ + ca-certificates \ + cmake \ + make \ + g++ \ + && rm -rf /var/lib/apt/lists/*; \ + echo "Downloading Cppcheck version: ${CPPCHECK_VERSION}"; \ + curl -fL "https://github.com/danmar/cppcheck/archive/${CPPCHECK_VERSION}.tar.gz" -o /tmp/cppcheck.tar.gz; \ + echo "${CPPCHECK_ARCHIVE_SHA256} /tmp/cppcheck.tar.gz" | sha256sum -c -; \ + mkdir -p /src/cppcheck && tar -xzf /tmp/cppcheck.tar.gz -C /src/cppcheck --strip-components=1; \ + rm /tmp/cppcheck.tar.gz; \ + cd /src/cppcheck; \ + mkdir build && cd build && cmake .. && cmake --build . -j"$(nproc)"; \ + strip bin/cppcheck + FROM python:3.10.19-bookworm +ARG TARGETARCH ENV DEBIAN_FRONTEND=noninteractive ENV LC_ALL=C.UTF-8 +COPY --from=cppcheck-builder /src/cppcheck/build/bin/cppcheck /usr/local/bin/cppcheck +COPY --from=cppcheck-builder /src/cppcheck/cfg /usr/local/share/Cppcheck/cfg + # This is used by the 04_install.sh script; we can't read the Python version from # .python-version for the same reasons as above, and it's more efficient to pull a # preexisting Python image than it is to build from source. diff --git a/contrib/containers/ci/ci-slim.Dockerfile b/contrib/containers/ci/ci-slim.Dockerfile index 73c864983939..53e70dc0d46c 100644 --- a/contrib/containers/ci/ci-slim.Dockerfile +++ b/contrib/containers/ci/ci-slim.Dockerfile @@ -1,6 +1,7 @@ # Builder for cppcheck FROM debian:bookworm-slim AS cppcheck-builder ARG CPPCHECK_VERSION=2.21.0 +ARG CPPCHECK_ARCHIVE_SHA256=f028ff75ca5372738f3737c8b3e8611426a6526b6aea2ef01301ab0f5902f044 RUN set -ex; \ apt-get update && apt-get install -y --no-install-recommends \ curl \ @@ -11,6 +12,7 @@ RUN set -ex; \ && rm -rf /var/lib/apt/lists/*; \ echo "Downloading Cppcheck version: ${CPPCHECK_VERSION}"; \ curl -fL "https://github.com/danmar/cppcheck/archive/${CPPCHECK_VERSION}.tar.gz" -o /tmp/cppcheck.tar.gz; \ + echo "${CPPCHECK_ARCHIVE_SHA256} /tmp/cppcheck.tar.gz" | sha256sum -c -; \ mkdir -p /src/cppcheck && tar -xzf /tmp/cppcheck.tar.gz -C /src/cppcheck --strip-components=1; \ rm /tmp/cppcheck.tar.gz; \ cd /src/cppcheck; \