Skip to content

fix: DCEL corruption crashes in trim/split and shape registration #101

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
1egoman wants to merge 6 commits into
base: main
Choose a base branch
from
Open

fix: DCEL corruption crashes in trim/split and shape registration #101

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
3be97aa
fix: address DCELShapeIndex duplicate edge causing thrown error
1egoman Sep 19, 2026
dce010a
fix: avoid empty undo transactions for no-op selection actions
1egoman Sep 19, 2026
d11d8c7
fix: guard main-polygon dedup against vertex count mismatch in trim
1egoman Sep 19, 2026
4185f6a
fix: skip trims whose boundary retraces an untouched shape
1egoman Sep 19, 2026
377235f
fix: skip degenerate self-loop edges when registering shapes
1egoman Sep 19, 2026
800c0d4
chore: add fuzzer harness and adapt it to the edit multi-tool
1egoman Sep 21, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
185 changes: 185 additions & 0 deletions docs/fuzzer.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,185 @@
# Fuzz Harness

The fuzzer is a headless, browser-free way to shake bugs out of the cad2d core. It
constructs the real app classes (`Sheet`, `GeometryStore`, `HistoryManager`,
`ToolManager`, `ActionsManager`, `SerializationManager`) in a plain Node process,
simulates user operations through the actual tool handler methods, and uses a
three-tier oracle to decide whether the resulting document is broken.

Because everything it touches is pure TypeScript with no DOM, it runs fast, fully
deterministically, and can be scaled to many parallel worker processes.

## Layout

```
fuzz/
README.md quick-start (same commands as this doc)
cli.ts argument parsing; spawns one worker process per "thread"
worker-entry.ts worker process entrypoint (config passed via env)
worker.ts the fuzz loop: walks, bug classification, shrinking, artifacts
app.ts FuzzApp: the headless core harness + document snapshot
ops.ts the op vocabulary: executeOp / describeOp / pickRandomOp
invariants.ts Tier-1 oracle (cheap structural checks)
judge.ts Tier-3 oracle (LLM judge + targeted symptom judge)
generator.ts LLM seed-sequence generation (the --prompt feature)
mutator.ts seed mutation: delete/insert/swap/nudge/duplicate
shrink.ts ddmin shrinker
repro.ts replay an artifact and check whether the bug reproduces
corpus.ts corpus read/write helpers
llm.ts OpenAI-compatible chat client (opencode go by default)
prng.ts seeded PRNG
types.ts OpRecord / CorpusEntry / config shared types
artifacts/ bug reports (written at runtime; gitignored if configured)
corpus/ interesting sequences (written at runtime)
```

## How it works

### Ops

A single user action is an `OpRecord` - a fully concrete, replayable operation.
Draws go through the real tools (`handleMouseDown` / `handleKeyDown`), edits
through `GeometryStore.updateById`, booleans/arrange through
`ActionsManager.execute`, trim/split through the `TrimSplitTool` handler methods.
Every op records its targets explicitly by id and stores absolute coordinates,
so replaying the same list of ops against a fresh app deterministically
reproduces the same document. Geometry ids are generated by a per-app counter
(`..._fuzzN`) instead of random UUIDs precisely to make this true.

`pickRandomOp(app, rng)` produces a random feasible op given the current state.
The op library supports: polygons/rectangles/ellipses (open+closed), translate,
vertex + arc-control-point moves, insert-point-on-edge, delete, union / difference /
intersection, open-close, convert-to-polygon, flips, raise/lower, link-dimensions,
linear/horizontal/vertical constraints, trim/split, undo/redo, and sheet settings.

### The three-tier oracle

| tier | when | cost | detects |
| --------- | ------------------------- | -------- | --------------------------------------------------------------------------------------------------------------------------------------------- |
| crash | every op | free | any op that throws |
| invariant | every op | free | NaN geometry, stale selection, leftover working shapes, `save()` failing or emitting NaN, and `undo()`+`redo()` not restoring identical state |
| LLM judge | every `--check-every` ops | LLM call | documents that "look wrong" (missing/spurious shapes, degeneracies, violated expectations) |

The invariant tier is what makes this useful without any LLM: the `undo-redo`
round-trip check alone has caught dozens of DCEL corruption bugs.

### Finding, shrinking, and logging a bug

When a walk triggers a bug:

1. **Classify** - crash, invariant, or (with the LLM enabled) semantic.
2. **Shrink** - ddmin deletion against a replay oracle. Crash/invariant bugs
shrink by replaying candidate sequences on a fresh app and checking for the
same failure; semantic bugs shrink by re-asking the LLM whether the specific
symptom still manifests. The result is a minimal reproduction sequence.
3. **Write artifacts** to `fuzz/artifacts/<case-id>/`:
- `meta.json` - kind, seed, error/violation text, op counts
- `ops.json` - the minimized reproduction sequence
- `ops.full.json` - the complete pre-shrink run
- `state.json` - structured document snapshot at the time of the bug
- `export.svg` - SVG export of the buggy document
- `notes.md` - human-readable report + repro command
4. **Append to the corpus** so other workers explore mutations in that region.

Artifacts are deduplicated by (kind + minimized ops), so one DCEL failure
doesn't fill the artifacts dir with hundreds of identical reports.

### Corpus and LLM-guided seeds

`fuzz/corpus/` holds "interesting" op sequences (JSON). Workers pick a corpus
entry ~70% of the time and run a mutated variant (`mutator.ts`: delete a step,
insert a random op, swap adjacent steps, duplicate, or nudge a coordinate by a
tiny epsilon - the nudges are what explore near-degenerate geometry like an
edge landing exactly on a vertex).

With `--prompt "..."`, the LLM generates the initial seed set for a scenario
(e.g. "an edge that passes exactly through a vertex of another shape"), then
random mutation explores around it.

## Running

Requirements: Node 18+ (global `fetch`), `tsx` (`npm install -D tsx` already done).

```bash
# Crash + invariant fuzzing, no LLM required (recommended first run)
npx tsx fuzz/cli.ts run --threads 4 --duration 300 --llm off

# With the LLM judge + seed generation (needs an API key)
FUZZ_LLM_API_KEY=sk-... npx tsx fuzz/cli.ts run --threads 4 --check-every 20 --duration 300

# Just generate prompt-driven seeds into the corpus
npx tsx fuzz/cli.ts gen --prompt "an edge that passes exactly through a vertex of another shape"

# Reproduce a logged bug (exit code 1 = reproduces)
npx tsx fuzz/cli.ts repro fuzz/artifacts/bug-1789831568711-555269

# List the current corpus
npx tsx fuzz/cli.ts corpus
```

### `run` options

| option | default | meaning |
| --------------- | ---------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `--threads` | 1 | parallel worker processes |
| `--check-every` | 20 | run the LLM judge every N ops |
| `--max-ops` | 200 | max ops per walk before the worker starts a fresh walk |
| `--duration` | 120 | wall-clock budget in seconds per worker; workers self-stop at this, and the CLI respawns any worker that dies early (e.g. an OOM) until the budget elapses |
| `--prompt` | - | LLM seed prompt; populates the corpus before workers start |
| `--llm` | auto | `on` / `off`; `auto` = on when an API key is available |
| `--corpus` | `fuzz/corpus` | interesting-sequences directory |
| `--artifacts` | `fuzz/artifacts` | bug report output directory |
| `--seed` | time | base seed for the first worker |
| `--exit-on-bug` | - | exit each worker after its first bug (useful for small/smoke runs) |

### LLM configuration (env)

| env | default | meaning |
| -------------------------- | ------------------------------- | ---------------------------- |
| `FUZZ_LLM_BASE_URL` | `https://opencode.ai/zen/go/v1` | OpenAI-compatible base URL |
| `FUZZ_LLM_API_KEY` | read from opencode `auth.json` | API key |
| `FUZZ_LLM_MODEL` | `deepseek-v4-flash` | model id |
| `FUZZ_LLM_RATE_PER_MINUTE` | 30 | client-side request throttle |

## Determinism

Workers use a seeded PRNG (`PRNG` in `fuzz/prng.ts`). Geometry ids come from a
per-app monotonic counter instead of `uuid()` so identical op sequences produce
identical ids and therefore identical documents. This is what makes:

- **shrinking** trustworthy (a candidate either reproduces or it doesn't),
- **repro** a real check (`fuzz/cli.ts repro <case>` replays `ops.json` and
re-runs the oracles against a fresh app),
- **corpus sharing** safe between workers.

One consequence: ids in artifacts are `ply_fuzz3`-style, not real UUIDs. `state.json`
and `export.svg` are the ground truth of the broken document for investigation.

## Investigating a reported bug

1. Read `notes.md` for the summary and the minimized op sequence.
2. Run `npx tsx fuzz/cli.ts repro fuzz/artifacts/<case>` - it replays `ops.json`
deterministically and prints the reproduced failure (or "NOT REPRODUCED").
3. Open `export.svg` to see the document; `state.json` has the full structured
state (geometries, constraints, datums, filters, selection, undo depths).
4. If the minimized sequence references geometries by id, those ids will exist
because replay regenerates them deterministically - there is no "which shape
did step 3 mean?" guessing.

## Practical notes

- The DCEL trim/split + undo/redo paths are the current hotbed of bugs; expect
`linkNext` / `splitEdge` / `mergeEdges` failures and undo/redo consistency
violations.
- The DCEL bug can also manifest as a runaway allocation (JS heap OOM). Workers
run with a 1GB heap cap and the CLI respawns crashed slots, so a single OOM
doesn't kill the run.
- `--llm off` runs are the cheap way to leave it going for hours on a CI box:
no network, no API key, just crash + invariant detection.

## Tests

Fuzzer unit tests live in `fuzz/__tests__/fuzz.test.ts` and run with the normal
suite (`npm test`). They cover PRNG determinism, deterministic app replay,
invariant health on a valid document, op picking/description, ddmin shrinking,
and the repro runner.
96 changes: 96 additions & 0 deletions fuzz/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
# Cad2d fuzz harness

A headless fuzzer for the cad2d core. It drives the real app classes
(`Sheet`, `GeometryStore`, `HistoryManager`, `ToolManager`, `ActionsManager`,
`SerializationManager`) with simulated user operations - no browser, no Pixi -
and uses a three-tier oracle to detect bugs:

1. **Crash** - any op that throws is an immediate bug (per-op, free).
2. **Invariant** - cheap structural checks after every op (per-op, free):
no NaN geometry, valid selection, no leftover working shapes, serialization
health, and an `undo()` + `redo()` round-trip consistency check.
3. **LLM judge** - every `--check-every` ops the LLM inspects the operation log
- SVG export + structured geometry dump and flags sessions that "look wrong".

When a bug is found the sequence is **shrunk** (ddmin chunk deletion against a
replay oracle), then logged to `fuzz/artifacts/<case>/` with the minimized op
sequence, the full sequence, the doc state, and the SVG export.

The fuzzer also mixes in **LLM-guided seeds**: with `--prompt`, the LLM produces
initial op sequences targeting the requested scenario (e.g. "an edge that passes
exactly through a vertex of another shape"). Those seeds populate `fuzz/corpus/`,
and workers mutate known-interesting corpus entries (delete/insert/swap/nudge
epsilon) to explore nearby regions. Found-bug sequences are appended to the
corpus so all workers converge on interesting geometry.

## Layout

```
fuzz/
cli.ts argument parsing + spawns one worker process per thread
worker-entry.ts worker process entrypoint (reads config from env)
worker.ts the fuzz loop: runs walks, classifies + shrinks + logs bugs
app.ts headless core harness (FuzzApp) + document snapshot
ops.ts the op vocabulary: executeOp / describeOp / pickRandomOp
invariants.ts Tier-1 oracle
judge.ts Tier-3 LLM oracle (generic judge + targeted symptom judge)
generator.ts LLM seed-sequence generation (the --prompt feature)
mutator.ts seed mutation (delete/insert/swap/nudge/duplicate)
shrink.ts ddmin shrinker
repro.ts replay an artifact and check if the bug reproduces
corpus.ts corpus read/write helpers
llm.ts OpenAI-compatible chat client (opencode go by default)
prng.ts seeded PRNG
types.ts OpRecord / CorpusEntry / config types
artifacts/ bug reports (written at runtime)
corpus/ interesting sequences (written at runtime)
```

## Running

```bash
# LLM tested locally first; judge/seed features need an API key
FUZZ_LLM_API_KEY=sk-... npx tsx fuzz/cli.ts run --threads 4 --check-every 20 --duration 300

# Crash/invariant-only fuzzing (no LLM)
npx tsx fuzz/cli.ts run --threads 4 --duration 300 --llm off

# Seed the corpus with an LLM for a scenario
npx tsx fuzz/cli.ts gen --prompt "an edge that passes exactly through a vertex of another shape"

# Replay a bug report to check it reproduces
npx tsx fuzz/cli.ts repro fuzz/artifacts/bug-12345
```

Options for `run`:

| option | default | meaning |
| --------------- | -------------- | ----------------------------------------- |
| `--threads` | 1 | parallel worker processes |
| `--check-every` | 20 | run the LLM judge every N ops |
| `--max-ops` | 200 | max ops per walk before restart |
| `--duration` | 120 | wall-clock budget per worker (seconds) |
| `--prompt` | - | LLM seed prompt (populates the corpus) |
| `--llm` | auto | `on` / `off`; auto = on when a key exists |
| `--corpus` | fuzz/corpus | directory for interesting sequences |
| `--artifacts` | fuzz/artifacts | bug report output directory |
| `--seed` | time | base seed for the first worker |
| `--exit-on-bug` | - | stop a worker after its first bug |

LLM config via env: `FUZZ_LLM_BASE_URL` (default opencode go), `FUZZ_LLM_API_KEY`
(default: read from the opencode auth.json), `FUZZ_LLM_MODEL`
(default `deepseek-v4-flash`), `FUZZ_LLM_RATE_PER_MINUTE` (default 30).

## Determinism

Workers use a seeded PRNG. Geometry ids are generated deterministically
(`..._fuzzN` counters) instead of random UUIDs so replaying an op sequence
produces the exact same document. This is what makes shrinking, repro and
corpus-sharing trustworthy.

## Notes

- Verified bugs found so far center on DCEL corruption during trim/split and
undo/redo of trims (`linkNext: ... does not exist`, `splitEdge: ... not
found`) - a known-fragile subsystem. Deduplicated artifacts will concentrate
on distinct op signatures so investigation isn't swamped with one giant bug.
Loading
Loading