A CI-oriented tool that compares accessibility scans between two builds. It blocks only newly introduced regressions with evidence from the scan.
Developers add accessibility issues by accident. Running an audit on every commit generates too much noise. Teams need a tool that compares the current build against a known baseline.
The tool must flag only regressions. It must not fail on pre-existing issues. It must provide structured output for CI systems.
The tool stores scan results as JSON baseline files. Each page scan records violations, passes, and incomplete checks from axe-core.
The diff engine compares nodes by URL, rule ID, and CSS selector. A node present only in the current scan is a regression. A node present only in the baseline is a fixed issue. An entirely new violation rule marks all its nodes as new violations.
GitHub Actions workflow commands annotate pull requests. Each regression produces a warning or error annotation. A clean diff produces a notice.
The scanner uses Playwright to load pages and axe-core to audit them. It supports custom viewports, timeouts, WCAG tag filters, and rule exclusion lists.
git clone https://github.com/DanielCuevas1208/web-a11y-regression-guard.git
cd web-a11y-regression-guard
npm install
npm run buildInstall Playwright browsers once if you plan to run scans:
npx playwright install chromiumWrite a script that uses the scanner API. The method requires a
Playwright Page object.
import { chromium } from "playwright";
import { scan } from "web-a11y-regression-guard";
const browser = await chromium.launch();
const page = await browser.newPage();
const report = await scan(page, {
urls: ["https://example.com"],
tags: ["wcag2a", "wcag2aa"],
});
await browser.close();Save the report as your baseline.
npx a11y-guard diff --baseline fixtures/scans/baseline.json --current fixtures/scans/current-with-regressions.jsonOutput:
FAIL: 3 regression(s) found across 2 violation rule(s).
[SERIOUS] https://example.com/page-a - color-contrast: Elements must meet...
Selector: .intro-text
[CRITICAL] https://example.com/page-a - label: Form elements must have labels
Selector: #signup-email
[CRITICAL] https://example.com/page-a - button-name: Buttons must have...
Selector: .close-btn
npx a11y-guard diff \
--baseline baseline.json \
--current current.json \
--annotate \
--error-on-regressionThe --annotate flag prints workflow commands to stdout. Pipe the
output to the CI log for inline PR annotations.
| Option | Description |
|---|---|
--baseline |
Path to the baseline scan JSON file |
--current |
Path to the current scan JSON file |
--annotate |
Print GitHub Actions workflow commands |
--error-on-regression |
Exit with code 1 when regressions exist |
--title |
Title for the annotation group |
--file |
Source file path for annotation context |
import { loadBaseline, diff, annotate } from "web-a11y-regression-guard";
const baseline = loadBaseline("baseline.json");
const current = loadBaseline("current.json");
const result = diff(baseline, current);
console.log(annotate(result, { mode: "error" }));src/
types.ts Shared type definitions
baseline.ts Load, save, and validate scan reports
diff.ts Compare two reports and find regressions
annotations.ts Format results as GitHub workflow commands
scanner.ts Playwright + axe-core integration
cli.ts Commander-based CLI entry point
index.ts Public API barrel export
fixtures/
pages/ Sample HTML pages for testing
scans/ Pre-recorded scan reports for testing
tests/
baseline.test.ts Baseline I/O and validation tests
diff.test.ts Regression detection tests
annotations.test.ts GitHub command formatting tests
The project uses fixture-driven tests. Each test loads a scan report
from fixtures/scans/ and verifies expected behavior.
npm testThe test suite includes 38 deterministic tests. It covers baseline loading, diff computation, and annotation formatting.
| Fixture | Purpose |
|---|---|
baseline.json |
Initial scan with two violations on two pages |
current-clean.json |
Same as baseline, no new issues |
current-with-regressions.json |
Three extra violation nodes added |
current-shifted.json |
Violations moved between pages |
pages/accessible.html |
WCAG-conformant HTML form |
pages/inaccessible.html |
Page with common violations |
The tool compares exact scan reports. It does not tolerate URL changes or selector drift between builds. Use stable URLs and stable selectors.
It depends on axe-core for detection. Axe-core finds approximately 30% to 50% of WCAG issues. Manual review is still necessary.
The tool does not moderate between two concurrent branches. It requires a single baseline and a single current report.
It does not scan SPAs automatically. Write a script that navigates
through each route before calling scan().
MIT