Skip to content

VALIDATION.md

Latest commit

 

History

History
37 lines (29 loc) · 2.75 KB

VALIDATION.md

File metadata and controls

37 lines (29 loc) · 2.75 KB
Raw

Validation — web-tutorial-video v1.0.0

Validated 2026-08-21.

Executed locally

  • Python syntax: all 10 helper scripts compiled from source in-memory successfully; no .pyc/__pycache__ was created.
  • Package hygiene: scripts/package_check.py passed on the final skill source tree.
  • Planning contract: examples/tutorial-plan.example.json passed validate_plan.py --stage planning (3 scenes).
  • Deterministic media self-test: scripts/self_test.py passed end-to-end using synthetic browser video + deterministic SVG + two synthetic WAVs → captured-plan validation → SRT/capture-cue generation → H.264/AAC render → final MP4 machine QC and per-scene review frames.
  • Negative safety test: a consequential + real plan without explicit_user_authorization=true was correctly rejected.
  • Negative packaging test: a package containing auth.json was correctly rejected.
  • JSON integrity: tutorial JSON Schema, behavioral eval cases, and example plan all parsed successfully.
  • Media environment: FFmpeg 7.1.5 / ffprobe 7.1.5 available and used in the self-test.
  • Node environment: Node 22.16.0 / npx 10.9.2 detected.

External/runtime validation status

The build environment did not have the supertonic CLI or agent-browser CLI installed, so this validation does not claim a live Supertonic synthesis or real website recording run. Their command shapes and behavioral assumptions were checked against current upstream documentation on 2026-08-21; source links are recorded in web-tutorial-video/SOURCES.md.

The included evals/cases.json + evals/rubric.md define seven live behavioral cases (read-only, authenticated SPA, reversible write, consequential invite, hostile prompt-injection page, Lightpanda fallback, and TTS timing change). Those must be run in the target agent harness with real browser/TTS access before calling a particular harness integration production-certified.

Release gates satisfied by the package itself

  • compact model-invoked frontmatter with explicit trigger conditions;
  • progressive-disclosure references with no orphaned critical branch;
  • every main workflow step has a checkable completion criterion;
  • page content treated as untrusted data;
  • consequential side effects default safe;
  • rendered rehearsal is mandatory before narration lock;
  • one production contract governs narration/capture/render/QC;
  • semantic targets are canonical; ephemeral browser refs are execution-only;
  • measured narration duration drives scene timing;
  • audio lead-in is implemented in the renderer;
  • deterministic rendering uses explicit video/audio stream mapping;
  • final QC distinguishes media-integrity checks from semantic privacy/instructional review;
  • packaging rejects known secret-bearing/generated sediment.