Repository navigation
chore(lint): add doc link checker, OSS hygiene ratchets, and CI tool pins - #1674
VLukiantsev wants to merge 1 commit into
Conversation
There was a problem hiding this comment.
Code Review
This pull request introduces a new Markdown link linter (check_doc_links.py) to validate relative links and restrict internal Google URLs, integrated via pre-commit hooks and Bazel. It also updates check_legacy_imports.py to flag generic helper filenames and OSS TODO(b/...) references. The review feedback identifies critical issues in resolve_repo_target concerning case-sensitive external scheme checks and incorrect resolution of root-relative links, and suggests adding corresponding unit tests to cover these scenarios.
95c3e06 to
007bea5
Compare
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request introduces a new Markdown link linter (check_doc_links.py) with associated tests and Bazel/pre-commit integrations to validate relative links and block forbidden internal URLs. It also updates check_legacy_imports.py to track generic helper filenames and internal TODO(b/...) references, and refines the fence-parsing logic in check_agent_doc_paths.py. The review feedback recommends enhancing the new link linter by decoding percent-encoded characters (e.g., %20) using urllib.parse.unquote to prevent false positives, adding tests for this behavior, and extending the forbidden URL pattern to catch protocol-relative corporate shortlinks.
007bea5 to
35c2b17
Compare
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request introduces a new documentation link and OSS URL hygiene linter (check_doc_links.py) along with its corresponding unit tests, Bazel build targets, and integration into the pre-commit githook. Additionally, it updates the check_legacy_imports.py linter to track and prevent the introduction of new generic helper filenames (e.g., containing 'util', 'helper', 'misc', or 'common') and internal Google bug tracker references (TODO(b/...)) in public directories, updating the baseline configuration accordingly. Refactorings were also made to the Markdown parsing logic in check_agent_doc_paths.py to share inline code and fence block detection. There are no review comments, and I have no feedback to provide.
07a15d7 to
e4500eb
Compare
e4500eb to
2138a44
Compare
2138a44 to
5429e88
Compare
5429e88 to
0279c5f
Compare
377bdbf to
e825c5c
Compare
e825c5c to
3083e0c
Compare
3083e0c to
41029bf
Compare
b77012e to
1e01195
Compare
1e01195 to
6054b14
Compare
6054b14 to
abff99c
Compare
abff99c to
08cff84
Compare
08cff84 to
1a45f99
Compare
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request introduces a new linter, check_doc_links.py, along with its corresponding unit tests, Bazel build targets, and pre-commit gate integration. This linter validates relative Markdown links and blocks forbidden internal URLs (such as corp.google.com or go/ shortlinks) in non-agent Markdown documentation. Additionally, the existing check_legacy_imports.py linter is updated to track and restrict generic helper filenames and internal TODO(b/...) references, with corresponding updates to its test suite and baseline metrics. The check_agent_doc_paths.py utility was also refactored to expose code and fence parsing logic for reuse. There are no review comments to address, and we have no additional feedback to provide.
4ba475e to
8b78ea4
Compare
…pins - Add `tools/lint/check_doc_links.py`, `check_doc_links_test.py`, and Bazel targets wired into `tools/lint/precommit_gates.py`, `.github/workflows/lint.yml`, and `tools/README.md`, with shared CommonMark `opens_fence`/`INLINE_CODE_RE` in `check_agent_doc_paths.py`, balanced/angle/reference-link parsing, case-insensitive external/file scheme checks, protocol-relative handling, and `forbidden_url` deduplication. - Add `oss_b_todo_references` and `generic_helper_filenames` ratchets to `check_legacy_imports.py` and `legacy_imports_baseline.json` (plus broadened `HandledError` guidance and unit tests). - Pin `ruff==0.16.4`, `mypy==2.4.1`, `helm` `v3.20.1` (`helm version --short` check), and `Check PR Diff Size` warning telemetry in `.github/workflows/lint.yml`. TAG=agy CONV=ba8411ef-c093-4f45-950a-b3bc1899268c
8b78ea4 to
e367865
Compare
Summary
Add
tools/lint/check_doc_links.py(with unit tests andcheck_doc_links_lib/check_doc_links_testBazel targets) wired intotools/lint/precommit_gates.py,.github/workflows/lint.yml, andtools/README.mdto validate relative Markdown links (including balanced parentheses,<...>angle destinations, linked images, root-relative paths, percent-encoded segments, and reference definitions) and reject non-codefile://, corporate hostnames, and shortlink URLs outside.agents/. Share CommonMark backtick-fence info-stringopens_fence/_FENCE_REand exact-backtick-runINLINE_CODE_REhelpers intools/lint/check_agent_doc_paths.py, extendtools/lint/check_legacy_imports.pyandtools/lint/legacy_imports_baseline.jsonwithoss_b_todo_references(api/,cmd/,pkg/,sdks/at baseline0) andgeneric_helper_filenames(baseline27, exemptingdeploy/**/templates/*.tpl), pinruff==0.16.4,mypy==2.4.1, andhelm(v3.20.1), and add non-blocking PR diff-size::warning::telemetry in.github/workflows/lint.yml.Motivation & Context
Automated link checking, generic-filename and internal issue-ID
TODOratchets, internal-URL blocks, pinned CI tool versions, and PR size warnings prevent documentation rot and OSS hygiene regressions across concurrent contributions.Key Changes
Modified/added 11 file(s):
tools/lint/precommit_gates.py&tools/lint/precommit_gates_test.py: Registercheck_doc_linksinPYTHON_GATES(gated infixmode on stagedAM*.md,check_doc_links.py/check_agent_doc_paths.py, orDpaths; unconditional in--checkmode), expandcheck_agent_doc_pathstrigger forcheck_agent_doc_paths.py, and add unit tests asserting both triggers intest_python_gates_include_doc_link_and_agent_doc_triggers..github/workflows/lint.yml: Pinruff==0.16.4,mypy==2.4.1, and Helmv3.20.1(replacing Helmv3.15.2and checkinghelm version --short | grep -q '^v3\.'forubuntu-latestwhich pre-installs Helmv4.3.1), addCheck Documentation Links & OSS URLs, and add PR-onlyCheck PR Diff Size(tools/analyze_diff_size.py) emitting::warning::when thresholds are exceeded.tools/README.md: Documentoss_b_todo_referencesand generic filename ratchets undercheck_legacy_imports.pyand add thecheck_doc_links.pyentry.tools/lint/BUILD.bazel: Addpy_librarycheck_doc_links_libandpy_testcheck_doc_links_test.tools/lint/check_agent_doc_paths.py: Extract sharedopens_fence(_FENCE_REenforcing CommonMark backtick fence info-string rules without extra backticks) and hardenedINLINE_CODE_REenforcing exact inline backtick-run delimiter matching across both doc linters.tools/lint/check_doc_links.py&tools/lint/check_doc_links_test.py: New linter scanning tracked*.mdfiles outside.agents/and unit tests covering percent-decoding, root-relative links, nested/tilde/unclosed fences, empty/malformed/multibyte inputs, linked images, parenthesized/angle/reference links, forbidden internal URLs, working-tree deletions, and CLI JSON output.tools/lint/check_legacy_imports.py,check_legacy_imports_test.py, &legacy_imports_baseline.json: Addoss_b_todo_references(^(api|cmd|pkg|sdks)/, baseline0) andgeneric_helper_filenames(27, exemptingdeploy/**/templates/**/*.tpl), broadenHandledErrorguidance, and test boundary cases.Assumptions
generic_helper_filenamesis ratcheted at the existing27-file repository count while exemptingdeploy/**/templates/**/*.tplfiles — ratcheting at27avoids mass cross-repo renames inside this linter PR while automatically ratcheting down viapre-commitas files are refactored — swapping one generic filename for another keeps net count27, non-code generic names outsidedeploy/**/templates/**/*.tplfail pre-commit/CI until renamed, while generic.tplpartials underdeploy/**/templates/, generic directory names, andtools/sharedfilenames remain unchecked by this metric.oss_b_todo_referencesscans only^(api|cmd|pkg|sdks)/for parenthesized internal issue-IDTODOreferences at baseline0— those four roots contain public OSS APIs, binaries, packages, and SDKs per the Open-Source TODO standard — internal issue IDs underinternal/,deploy/,examples/,test/, ortools/(or unparenthesizedTODO:forms) are not flagged by this metric.check_doc_links.pyscans tracked*.mdfiles outside.agents/(includingAGENTS.mdand*/AGENTS.md), treats/pathlinks as repo-root-relative, skips non-file://URI schemes and non-corporate protocol-relative//...external links inresolve_repo_target, and triggers inpre-commitfixmode whenAM.md,check_doc_links.py,check_agent_doc_paths.py, orDpaths are staged —.agents/owns internal skill/rule conventions checked bycheck_agent_doc_paths.py— internal URLs or broken links inside.agents/, 4-space indented code blocks, inline code spans, or trailing lines after unclosed fences are not flagged bycheck_doc_links.py..github/workflows/lint.ymlpinsruff==0.16.4,mypy==2.4.1, and Helmv3.20.1inside availability/version guards (if ! command -v <tool>for Ruff/Mypy;if ! helm version --short | grep -q '^v3\.'for Helm) —ubuntu-latestpre-installs Helm v4 while pre-installed tools save setup latency when matching — ifubuntu-latestpre-installs a different Ruff/Mypy version onPATHthe guard skips the pin, or if a pinned version is unavailable or stricter than local tools, the CICode Quality & Lintingjob fails until aligned.Check PR Diff Sizeruns only whengithub.event_name == 'pull_request', callstools/analyze_diff_size.py --base-ref "origin/${BASE_REF}...", and emits a non-blocking::warning::annotation — diff size serves as an advisory split prompt rather than a hard merge blocker — threshold-exceeding PRs still pass CI unless gated in review.Verification & Testing
./.githooks/pre-commit --check(Buildifier-lint=warn,gofmt,golangci-lint,check_go_determinism,go mod tidy,check_legacy_imports,check_examples_coverage,check_matrix_contract,check_agent_doc_paths,check_doc_links,check_file_naming,check_ui_selector_contract, Prettier, ESLint,tsc,addlicense, Ruff, Mypy,terraform fmt, TFLint,helm lint,test/helm88 tests, Trivy,actionlint,shellcheck, Non-UI JS ESLint, and Bazel tests)bazel test --test_output=errors //tools/lint:all //cmd/ui:ui_unit_testsnode --test .github/scripts/*.test.js./.githooks/pre-commit --checkpassed (320/320backend Bazel tests pass, Ruff355files, Mypy345source files,check_go_determinism504files,check_doc_links78docs /0violations,88/88test/helmtests),//tools/lint:all(9/9) and//cmd/ui:ui_unit_testspassed, andnode --test .github/scripts/*.test.jspassed (229/229). CI-onlyruff==0.16.4,mypy==2.4.1,helm v3.20.1, andCheck PR Diff Sizesteps run on GitHubpull_requestevents after push.Related Issues / Bugs
N/A (extends #1066 legacy-hygiene ratchets; does not close #1066).
TAG=agy
CONV=ba8411ef-c093-4f45-950a-b3bc1899268c