Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,8 @@ description: Maintain reviewer-first standalone test projects that use concise c

Use two sources of truth:

- `reviews/` contains behavior a human can judge.
- Executable tests contain a nearby `TEST-MAP: <CASE-ID>` comment.
- Root `reviews/` centrally contains behavior a human can judge across every module.
- Executable tests live under `suites/<suite>/` and contain a nearby `TEST-MAP: <CASE-ID>` comment.

Do not create coverage ledgers, approval statuses, or internal ID chains.

Expand All @@ -22,7 +22,7 @@ Keep each invocation bounded:

Keep context small:

- Read the nearest `AGENTS.md`, affected review document, feedback file, mapped tests, and only the source or diff needed for the current behavior.
- Read the nearest `AGENTS.md`, affected root review document, root feedback file, mapped suite tests, and only the source or diff needed for the current behavior.
- Locate with `rg` or equivalent before reading targeted ranges. Avoid repository-wide file dumps and repeated reads of unchanged instructions.
- Batch related reads and checks.
- After context compaction, recover from current files, `git diff`, and the affected document rather than replaying the whole repository.
Expand Down
2 changes: 1 addition & 1 deletion assets/repo-tests/root/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ Split larger scopes coherently instead of omitting behavior. Group related field

## Feedback and mapping

Read the nearest `reviews/review-feedback.md` before revising cases. On corrective feedback, append:
Read root `reviews/review-feedback.md` before revising cases. On corrective feedback, append:

```text
- model: <model-id-or-unavailable> | cases: <case IDs or review scope> | feedback: <human feedback verbatim>
Expand Down
3 changes: 2 additions & 1 deletion assets/repo-tests/root/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,8 @@ This repository stores human-reviewed test intent and automation mapped by `TEST
{{SUITE_LIST}}

- `source/{{PROJECT_SLUG}}/`: ignored product checkout.
- `reviews/`: concise behavior tables.
- `reviews/`: centralized behavior tables for every suite.
- `suites/<suite>/`: module-owned executable automation, commands, and fixtures.
- `scripts/check_test_map.py`: computed mapping coverage.

Work on one review document at a time: present changed rows, stop for confirmation, implement confirmed cases, then run a focused test and the mapping checker. Stable commands follow.
Expand Down
6 changes: 3 additions & 3 deletions assets/repo-tests/suite/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,9 @@
Read root `../../AGENTS.md`; this file only routes suite-specific work.

- Focus: {{SUITE_FOCUS}}.
- Reviews: `reviews/`; executable areas: {{CODE_DIRS}}; reusable inputs: `fixtures/`.
- Read `reviews/review-feedback.md` when present.
- Reviews: root `../../reviews/`; executable areas: {{CODE_DIRS}}; reusable inputs: `fixtures/`.
- Read root `../../reviews/review-feedback.md` when present.
- Keep case IDs globally unique, map tests with `TEST-MAP: <CASE-ID>`, and mirror that mapping with `- [x]`/`- [ ]` at the start of the scenario cell.
- After confirmation, implement direct, readable tests; add an abstraction only when it improves readability without hiding behavior or assertions that prove the expected behavior.
- Verify with the focused suite command and `python3 ../../scripts/check_test_map.py --root ../..`.
- Verify with the focused suite command and `python3 ../../scripts/check_test_map.py`.
- Do not advance to another review document automatically or poll CI repeatedly.
4 changes: 2 additions & 2 deletions assets/repo-tests/suite/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ Focus: {{SUITE_FOCUS}}.

## Layout

- `reviews/`: concise reviewer-facing case tables.
- `../../reviews/`: centralized reviewer-facing case tables for every suite.
- {{CODE_DIRS}}: executable tests or targets.
- `fixtures/`: suite-owned reusable inputs.

Expand All @@ -21,5 +21,5 @@ Focused run: pending target integration
Mapping check from this directory:

```bash
python3 ../../scripts/check_test_map.py --root ../..
python3 ../../scripts/check_test_map.py
```
3 changes: 0 additions & 3 deletions assets/repo-tests/suite/reviews/README.md

This file was deleted.

11 changes: 6 additions & 5 deletions references/initialize.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,21 +28,22 @@ After map confirmation, select one coherent review document and follow `review-c

After row confirmation, follow `automation-maintenance.md`. Implement only confirmed IDs.

## Layout choice
## Layout

Use the flat layout for one runner or assurance approach:
Keep reviewer-facing intent centralized at the root and group all executable automation by module or runner under `suites/`, even when the project starts with one suite:

```text
<project>-tests/
├── AGENTS.md
├── README.md
├── source/<project>/
├── reviews/<area>/<behavior>.md
├── tests/ or benchmarks/ or targets/
├── fixtures/
├── suites/<suite>/
│ ├── tests/ or case/ or benchmarks/ or targets/
│ └── fixtures/
└── scripts/check_test_map.py
```

Create `suites/<suite>/` only for genuinely independent runners such as API, P2P, performance, or fuzz. Keep case IDs unique across the project.
Do not create suite-local `reviews/` copies. Add another `suites/<suite>/` directory when a new module or runner needs separate commands, fixtures, or executable ownership. Keep case IDs unique across the project.

Reuse a matching checkout under `source/<project>/`; clone only when absent and never overwrite a conflicting path.
4 changes: 2 additions & 2 deletions references/review-cases.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ Prefix every scenario cell with `- [ ]` for a case without mapped automation or

## Corrective feedback

Before revising cases, read the nearest `reviews/review-feedback.md` when it exists.
Before revising cases, read root `reviews/review-feedback.md` when it exists.

When a human corrects AI-authored cases—missing or unnecessary cases, scenario or expectation errors, priority changes, merges, splits, renames, or deletions—append one physical line:

Expand All @@ -26,4 +26,4 @@ When a human corrects AI-authored cases—missing or unnecessary cases, scenario
- Do not record approval without a correction.
- Keep the file as reusable analysis feedback, not a case status, approval ledger, resolution log, or run history.

In a single-suite project use root `reviews/review-feedback.md`; in a multi-suite project use the affected suite's file.
Use one root `reviews/review-feedback.md` across all suites so corrections remain centralized with the review documents.
7 changes: 4 additions & 3 deletions scripts/check_test_map.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
)
TASK_CHECKBOX = re.compile(r"^-\s+\[(?P<state>[ xX])\]\s+")
TEST_MAP = re.compile(rf"\bTEST-MAP:\s*(?P<case>{CASE_TOKEN})\b")
CODE_DIR_NAMES = {"tests", "benchmarks", "targets"}
CODE_DIR_NAMES = {"case", "tests", "benchmarks", "targets"}
CODE_SUFFIXES = {
".c",
".cc",
Expand All @@ -40,6 +40,7 @@
".tsx",
}
SKIP_DIRS = {".git", ".idea", ".pytest_cache", ".venv", "node_modules", "source", "__pycache__"}
PROJECT_ROOT = Path(__file__).resolve().parent.parent


def parse_args() -> argparse.Namespace:
Expand All @@ -49,8 +50,8 @@ def parse_args() -> argparse.Namespace:
parser.add_argument(
"--root",
type=Path,
default=Path.cwd(),
help="Test-project root (default: current directory)",
default=PROJECT_ROOT,
help="Test-project root (default: repository containing this script)",
)
parser.add_argument(
"--require-complete",
Expand Down
104 changes: 33 additions & 71 deletions scripts/init_repo_tests.py
Original file line number Diff line number Diff line change
Expand Up @@ -198,34 +198,16 @@ def main() -> int:
if missing:
raise SystemExit(f"required skill assets are missing: {', '.join(missing)}")

single_suite = len(suite_specs) == 1
if single_suite:
suite, metadata = suite_specs[0]
code_dirs = metadata["code_dirs"]
layout = (
f"A single {metadata['label']} approach uses root `reviews/`, "
f"{', '.join(f'`{name}/`' for name in code_dirs)}, and `fixtures/`."
)
suite_list = (
f"- Root review and execution scope: {metadata['label']} — {metadata['focus']}."
)
review_locations = "- `reviews/<area>/<interface-or-behavior>.md`"
commands = (
"Replace after target integration:\n\n"
"```text\nSetup: pending target integration\nRun: pending target integration\n"
"Focused run: pending target integration\n```"
)
else:
layout = "Independent runners and assurance approaches live under `suites/<suite>/`."
suite_list = "\n".join(
f"- `suites/{suite}/`: {metadata['label']} — {metadata['focus']}."
for suite, metadata in suite_specs
)
review_locations = "\n".join(
f"- `suites/{suite}/reviews/<area>/<interface-or-behavior>.md`"
for suite, _ in suite_specs
)
commands = "Setup and run commands live in each suite README."
layout = (
"Reviewer-facing case documents are centralized under root `reviews/`; "
"executable automation and fixtures are grouped by module under `suites/<suite>/`."
)
suite_list = "\n".join(
f"- `suites/{suite}/`: {metadata['label']} — {metadata['focus']}."
for suite, metadata in suite_specs
)
review_locations = "- `reviews/<area>/<interface-or-behavior>.md`"
commands = "Setup and run commands live in each suite README."

common = {
"PROJECT_NAME": args.project.strip(),
Expand Down Expand Up @@ -269,65 +251,45 @@ def main() -> int:
)
checker_output.chmod(0o755)

if single_suite:
_, metadata = suite_specs[0]
for suite, metadata in suite_specs:
code_dirs = metadata["code_dirs"]
replacements = {
**common,
"SUITE_TYPE": suite,
"SUITE_LABEL": metadata["label"],
"SUITE_FOCUS": metadata["focus"],
"CODE_DIRS": ", ".join(f"`{name}/`" for name in code_dirs),
}
suite_output = output / "suites" / suite
render_tree(
suite_template,
suite_output,
replacements,
force=args.force,
created=created,
replaced=replaced,
skipped=skipped,
)
write_file(
output / "fixtures" / "README.md",
suite_output / "fixtures" / "README.md",
fixtures_readme(metadata["label"]),
force=args.force,
created=created,
replaced=replaced,
skipped=skipped,
)
for directory, purpose in metadata["code_dirs"].items():
for directory, purpose in code_dirs.items():
write_file(
output / directory / "README.md",
suite_output / directory / "README.md",
readme_for_directory(metadata["label"], directory, purpose),
force=args.force,
created=created,
replaced=replaced,
skipped=skipped,
)
else:
for suite, metadata in suite_specs:
code_dirs = metadata["code_dirs"]
replacements = {
**common,
"SUITE_TYPE": suite,
"SUITE_LABEL": metadata["label"],
"SUITE_FOCUS": metadata["focus"],
"CODE_DIRS": ", ".join(f"`{name}/`" for name in code_dirs),
}
suite_output = output / "suites" / suite
render_tree(
suite_template,
suite_output,
replacements,
force=args.force,
created=created,
replaced=replaced,
skipped=skipped,
)
write_file(
suite_output / "fixtures" / "README.md",
fixtures_readme(metadata["label"]),
force=args.force,
created=created,
replaced=replaced,
skipped=skipped,
)
for directory, purpose in code_dirs.items():
write_file(
suite_output / directory / "README.md",
readme_for_directory(metadata["label"], directory, purpose),
force=args.force,
created=created,
replaced=replaced,
skipped=skipped,
)

print(f"test-project: {output.resolve()}")
print(f"layout: {'single-suite' if single_suite else 'multi-suite'}")
print("layout: centralized-reviews-with-suites")
print(f"created: {len(created)}")
print(f"replaced: {len(replaced)}")
print(f"preserved: {len(skipped)}")
Expand Down
Loading