Skip to content

Use every frozen self-play record in training and validation - #10

Merged
arhancanli merged 1 commit into
engine/macbook-continuous-20261005from
engine/training-row-coverage-20261005
Oct 5, 2026
Merged

arhancanli merged 1 commit into
engine/macbook-continuous-20261005from
engine/training-row-coverage-20261005

Conversation

@arhancanli

@arhancanli arhancanli commented Oct 5, 2026 •

Copy link
Copy Markdown
Owner

The trainer dropped the end of every shuffled record buffer unless it filled a complete batch. On a read-only 73,465,643-position snapshot, the existing iterator omitted 2,753,535 training rows (3.79%) and 15,148 validation rows (2.06%). This carries tails into the next buffer and emits one final partial batch, so every frozen row participates exactly once per pass.

The optimizer schedule now uses the actual ceiling batch count. Training loss and throughput are weighted by records, validation checks its full held-out count, and both passes reject incomplete coverage or nonfinite losses. Frozen inputs may grow beyond their captured length, but a shortened captured prefix fails instead of silently shrinking the dataset. Model architecture, outcome weighting, optimizer and promotion rules retain their existing settings; any resulting network still requires the prescribed strength gate.

Validation: all 51 controller/stream tests pass with ResourceWarning treated as an error, including five new coverage/prefix regressions. Syntax and diff checks pass. The real 1024-hidden Metal smoke passed with the full 16,384 batch size on a deliberately unaligned 102,401-record fixture. Both epochs covered all 77,825 training rows in five optimizer steps; every validation pass covered all 24,576 held-out rows. Appended records/partial bytes stayed outside the captured prefix, and the actual Dataset rejected a shortened prefix. Checkpoint re-export was byte-identical, 96 native integer evaluations matched the independent emulator, and 16 native searches returned legal moves. Exact-commit CI passed at 068cdec with release Rust tests, full perft/SPRT calibration, WASM and all 51 Python tests. No model or strength gain is claimed. The playing binary, accepted network and active search checkpoint are unchanged.

@arhancanli
arhancanli merged commit 068cdec into engine/macbook-continuous-20261005 Oct 5, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant