Skip to content

chore(lint): enable buf lint test, expand buildifier lint, and fix docs - #1673

Open
VLukiantsev wants to merge 2 commits into
chore/harden-ui-eslint-and-typecheckfrom
chore/enable-buf-lint-and-fix-bzl-docs
Open

VLukiantsev wants to merge 2 commits into
chore/harden-ui-eslint-and-typecheckfrom
chore/enable-buf-lint-and-fix-bzl-docs

Conversation

@VLukiantsev

@VLukiantsev VLukiantsev commented Nov 12, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Enable //api:protos_lint_test via buf_lint_test (api/buf.yaml), add docstrings and $(execpath ...) in oci.bzl, expand buildifier from -warnings=unused-variable to -lint=warn in tools/lint/precommit_runner.py and CI (probing -warnings=-LANG_proto_library-single-deps,-inline-deps so both upstream OSS buildifier v8.0.3 and Google internal buildifier pass cleanly), fix all make-location and constant-glob warnings across active BUILD.bazel files + oci.bzl, replace the removed Blueprint v1 link in docs/design/topologies.md, and sync docs/developer.md.

Motivation & Context

Enabling Buf protobuf lint (while leaving frozen .proto sources untouched), full Buildifier lint checks across Bazel files, and clean doc links prevents schema style drift and deprecated $(location)/constant-glob BUILD bugs; docs/design/topologies.md is updated alongside this check-enabling slice to purge an obsolete Blueprint v1 reference.

Key Changes

Modified 22 file(s):

  • api/BUILD.bazel & api/buf.yaml: Add //api:protos_lint_test (buf_lint_test, error_format = "msvs"), update buf.yaml header comment, and add lint: config (STANDARD minus 4 multi-directory PACKAGE_* rules, descriptor-path ignore_only legacy baseline, and an inline TODO tracking unused structured_message.proto import cleanup in api/protos/experiment/api.proto).
  • tools/lint/precommit_gates.py, tools/lint/precommit_runner.py, tools/lint/precommit_runner_test.py, & .github/workflows/lint.yml: Add stdin_devnull to gates.exec_cmd, add include_untracked: bool = False to gates.git_ls_files for Bazel file discovery in check_bazel (with gates.chunked(..., 500) batching), probe Google-only -LANG_proto_library-single-deps,-inline-deps exclusions for compatibility with CI OSS buildifier v8.0.3, run buildifier -lint=warn after fix, and add unit tests covering both probe paths in test_check_bazel_invokes_buildifier.
  • oci.bzl: Add module docstring and oci_image_for_service Args: docstring, and switch $(location //tools/common:build_oci.py) to $(execpath //tools/common:build_oci.py) in the generated *_tarball genrule (tarball_target).
  • cmd/ui/BUILD.bazel, deploy/helm/BUILD.bazel, deploy/profiles/stable-2026a/components/{clickhouse,ingestion-gateway,redpanda}/BUILD.bazel, deploy/profiles/stable-2026b/components/{clickhouse,hpo,ingestion-gateway,redpanda,vizier}/BUILD.bazel, internal/storage/{postgres,spanner}/migrations/version/BUILD.bazel: Replace $(location ...) with $(execpath ...) and move explicit files (Chart.yaml, values.yaml, values-local.yaml, nginx.conf, generate_config.sh, karma.conf.js) out of glob(...), dropping redundant "package-lock.json" already matched by "*.json".
  • docs/design/topologies.md & docs/developer.md: Mark Blueprint v1 removed in topologies.md, and document //api:protos_lint_test, _PROTOS coverage (including //api/filter:filter_proto), and buildifier -lint=warn -mode=check in docs/developer.md.

Assumptions

  • Using descriptor-set paths (api/... and virtual-import xmanager_cloud/...) in api/buf.yaml ignore_only matches what rules_buf's buf_lint_test feeds to protoc-gen-buf-lint — buf_lint_test passes compiled descriptor names rather than api/-relative paths — running standalone buf lint CLI directly against api/ outside Bazel would mismatch ignore_only paths without adjusting config.
  • Probing -warnings=-LANG_proto_library-single-deps,-inline-deps before passing -warnings= keeps CI (buildifier v8.0.3) and Google-internal buildifier compatible while retaining multi-dep go_proto_library rules (grpc-gateway deps) and DRY _KIND_3P_DEPS in test/e2e/BUILD.bazel — upstream OSS buildtools rejects unrecognized -warnings= tokens whereas Google buildifier needs them excluded — future default warnings added in newer buildifier releases could fail until addressed or suppressed, and unintended multi-dep go_proto_library / deps-variable patterns are not flagged when excluded.
  • Listing //api:protos_lint_test targets explicitly in _PROTOS (api/BUILD.bazel) covers active repository .proto definitions except cloud_xmanager_v1_proto (excluded by design as a source-duplicate wrapper) — _PROTOS feeds both protos_breaking_test and CI uncovered-proto verification — any new proto_library added under //api skips lint until added to _PROTOS.
  • Moving explicit filenames (Chart.yaml, values.yaml, values-local.yaml, nginx.conf, generate_config.sh, karma.conf.js) out of glob(...) requires checked-in inputs to remain present — glob(...) would silently omit accidentally deleted files — deleting or renaming any of those files will now fail Bazel loading loudly until the corresponding BUILD.bazel list is updated.

Verification & Testing

  • Commands Executed:
    • ./.githooks/pre-commit --check & ./.githooks/pre-commit on #1673 commit (Buildifier -lint=warn, Ruff, Mypy, helm lint, test/helm 88 tests, Trivy, and Bazel tests), bazel build //deploy/profiles/stable-2026b/..., python3 tools/analyze_diff_size.py --base-ref chore/harden-ui-eslint-and-typecheck, and bazel test --test_output=errors //tools/lint:precommit_gates_test //tools/lint:precommit_runner_test //api:protos_lint_test.
  • Coverage & Test Status:
    • //api:protos_lint_test, //api:protos_breaking_test, //tools/lint:all, //deploy/profiles/stable-2026b/... (8 targets), Buildifier -lint=warn, test/helm (88 tests), test/helm/profiles/stable2026b:stable2026b_test, and 321/321 backend Bazel tests on #1673 (149 prod lines / 41 test lines, threshold_exceeded: false) passed locally. Upstream OSS buildifier v8.0.3 verified in CI Code Quality & Linting.

Related Issues / Bugs

N/A (tracked pre-existing unused structured_message.proto import in api/protos/experiment/api.proto via inline TODO in api/buf.yaml).

TAG=agy
CONV=ba8411ef-c093-4f45-950a-b3bc1899268c

@VLukiantsev
VLukiantsev added this pull request to stack #1675 November 12, 2026 19:46

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request integrates Protobuf linting by adding a buf_lint_test target to api/BUILD.bazel and configuring lint rules in api/buf.yaml. Additionally, it removes unused-variable warnings from the pre-commit buildifier check, updates documentation to note the deprecation of Cluster Toolkit Blueprint v1, and refactors oci.bzl to use $(execpath) and include docstrings. The review feedback correctly identifies that the paths specified in api/buf.yaml under ignore_only must be relative to the api/ directory (the Buf module root) rather than the workspace root, and that paths starting with xmanager_cloud/ cannot be resolved from this location.

Comment thread api/buf.yaml Resolved
@VLukiantsev VLukiantsev changed the title chore: enable buf lint test, full buildifier lint, and fix topologies link chore(lint): enable buf lint test, expand buildifier lint, and fix docs Nov 13, 2026
@VLukiantsev
VLukiantsev force-pushed the chore/enable-buf-lint-and-fix-bzl-docs branch from 3bf7271 to 4f09559 Compare November 13, 2026 00:48
@VLukiantsev

Copy link
Copy Markdown
Collaborator Author

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request integrates Buf linting for protobuf files, updates Bazel build files to use $(execpath) instead of $(location), refactors Helm chart filegroups, and removes legacy Cluster Toolkit Blueprint v1 documentation. The pre-commit hook is also updated to use git ls-files for finding Bazel files. A review comment suggests ensuring that files gathered in the pre-commit hook actually exist on disk using a file existence check rather than just a non-empty string check, preventing potential failures with deleted but unstaged files.

Comment thread .githooks/pre-commit Resolved
@VLukiantsev
VLukiantsev force-pushed the chore/enable-buf-lint-and-fix-bzl-docs branch 2 times, most recently from 08f58d0 to c345143 Compare November 13, 2026 09:19
@VLukiantsev
VLukiantsev force-pushed the chore/enable-buf-lint-and-fix-bzl-docs branch from c345143 to 26329ad Compare November 13, 2026 10:14
@VLukiantsev
VLukiantsev force-pushed the chore/enable-buf-lint-and-fix-bzl-docs branch from 26329ad to ff7de7a Compare November 13, 2026 10:56
@VLukiantsev
VLukiantsev force-pushed the chore/enable-buf-lint-and-fix-bzl-docs branch from ff7de7a to b9e3435 Compare November 13, 2026 11:58
@VLukiantsev
VLukiantsev force-pushed the chore/enable-buf-lint-and-fix-bzl-docs branch 2 times, most recently from 8b8fbc7 to e4b9803 Compare November 13, 2026 13:16
@VLukiantsev
VLukiantsev force-pushed the chore/enable-buf-lint-and-fix-bzl-docs branch 2 times, most recently from b704614 to b147bd1 Compare November 13, 2026 14:58
@VLukiantsev
VLukiantsev force-pushed the chore/enable-buf-lint-and-fix-bzl-docs branch from b147bd1 to b8d1571 Compare November 16, 2026 08:37
@VLukiantsev
VLukiantsev force-pushed the chore/enable-buf-lint-and-fix-bzl-docs branch from b8d1571 to 1585f04 Compare November 16, 2026 09:17
@VLukiantsev
VLukiantsev force-pushed the chore/enable-buf-lint-and-fix-bzl-docs branch from 1585f04 to 5a1b95d Compare November 16, 2026 11:14
@VLukiantsev
VLukiantsev removed this pull request from stack #1675 November 16, 2026 11:16
@VLukiantsev
VLukiantsev added this pull request to stack #1702 November 16, 2026 11:16
@VLukiantsev
VLukiantsev force-pushed the chore/enable-buf-lint-and-fix-bzl-docs branch from 5a1b95d to f710acb Compare November 16, 2026 11:44
@VLukiantsev
VLukiantsev force-pushed the chore/enable-buf-lint-and-fix-bzl-docs branch from f710acb to e8af856 Compare November 16, 2026 12:36
@VLukiantsev
VLukiantsev marked this pull request as ready for review November 16, 2026 19:25
@github-actions
github-actions Bot requested a review from AntonTyb November 16, 2026 19:25
@VLukiantsev
VLukiantsev force-pushed the chore/enable-buf-lint-and-fix-bzl-docs branch from e8af856 to 3abde37 Compare November 16, 2026 19:57
@VLukiantsev

Copy link
Copy Markdown
Collaborator Author

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces Buf linting (buf_lint_test) for protobuf APIs, updates pre-commit linting tools (specifically buildifier integration to support Google-specific warnings and untracked files), refactors several Bazel BUILD files to use $(execpath) instead of $(location) and explicitly list certain files instead of globbing them, and updates documentation regarding deployment topologies and developer checks. I have no feedback to provide as there are no review comments.

@VLukiantsev
VLukiantsev force-pushed the chore/enable-buf-lint-and-fix-bzl-docs branch from 3abde37 to 9d24eac Compare November 16, 2026 20:08
@VLukiantsev
VLukiantsev force-pushed the chore/enable-buf-lint-and-fix-bzl-docs branch from 9d24eac to 1e93c68 Compare November 16, 2026 20:13
@VLukiantsev
VLukiantsev force-pushed the chore/enable-buf-lint-and-fix-bzl-docs branch 2 times, most recently from 1b72a67 to 6c754aa Compare November 17, 2026 11:58
@VLukiantsev
VLukiantsev force-pushed the chore/enable-buf-lint-and-fix-bzl-docs branch from 6c754aa to 4b75d34 Compare November 17, 2026 13:17
- Add `//api:protos_lint_test` (`buf_lint_test`) and `lint:` config (`STANDARD` with multi-directory package exceptions and descriptor-path `ignore_only` baseline) in `api/buf.yaml`.
- Expand `buildifier` to `-lint=warn` in `tools/lint/precommit_runner.py` and `.github/workflows/lint.yml` with a runtime probe excluding Google-only `-LANG_proto_library-single-deps,-inline-deps` when supported, use `git ls-files` cleanly in `check_bazel`, fix `make-location` (`$(execpath)`) and `constant-glob` warnings across 9 `BUILD.bazel` files + `oci.bzl`, and add `oci.bzl` docstrings.
- Replace dead `deploy/ctk_v1/README.md` link in `docs/design/topologies.md` and document `//api:protos_lint_test` and `buildifier -lint=warn` in `docs/developer.md`.

TAG=agy
CONV=ba8411ef-c093-4f45-950a-b3bc1899268c
@VLukiantsev
VLukiantsev force-pushed the chore/enable-buf-lint-and-fix-bzl-docs branch from 4b75d34 to 944f1d9 Compare November 17, 2026 14:09

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant