Skip to content

fix: DCEL corruption crashes in trim/split and shape registration - #101

Open
1egoman wants to merge 6 commits into
mainfrom
fix-dcel-corruption
Open

1egoman wants to merge 6 commits into
mainfrom
fix-dcel-corruption

Conversation

@1egoman

@1egoman 1egoman commented Sep 21, 2026

Copy link
Copy Markdown
Contributor

Summary

This branch fixes a family of DCEL (Doubly-Connected Edge List) corruption bugs discovered by the fuzzer. The crashes all surface the same way — linkNext: half-edge "he_..." does not exist, splitEdge/mergeEdges "edge not found", or TypeError — during undo/redo of trim / split operations on overlapping shapes.

The root cause is a single theme: re-registering polygon boundaries into the DCEL can mint degenerate edges (zero-length self-loops, duplicate half-edge pairs under one edge key), which orphans half-edge references and corrupts the ref-counted edge cache. Later undo/redo replay then trips over the dangling entries.

A fuzzer (added in the last commit) replays randomized sequences and checks invariants after each op. End-to-end result across the 253-artifact corpus:

State Artifacts passing
Before this branch 0
After first 3 fixes 127 / 253
After all 5 fixes 182 / 253

Full Jest suite: 1052 passing tests across 37 suites (10 skipped).


Fix 1 — 3be97aa — Address DCELShapeIndex duplicate edge causing thrown error

Problem. When a new polygon edge passes within floating-point noise of an existing vertex, addVertex(split.point) minted a near-duplicate vertex, and two adjacent existing edges were both "split" at that phantom point — producing two half-edge pairs for the same edge key. The edge cache can only hold one pair per key, so the second overwrite orphaned the first while shapes still referenced it. On undo/redo, releaseEdge/mergeEdges culled a live pair → linkNext: half-edge "he_..." does not exist.

Fix. In _registerShape Phase 2, snap each computed intersection point onto the closest of the four relevant vertex positions (existing origin/dest, candidate origin/dest) when within SPLIT_ENDPOINT_EPSILON (1e-9). addVertex then returns the genuinely-shared vertex id, so the existing endpoint-touch guard fires and no phantom vertex / duplicate edge key is created.

Before

[undo-redo-symmetry] invariant 'undo-redo-symmetry' threw:
  linkNext: next half-edge "he_121" does not exist.

After — the minimized repro (ellipse + open polygon + trim-split) reports:

NOT REPRODUCED - the minimized sequence no longer triggers the bug.

Fix 2 — dce010a — Avoid empty undo transactions for no-op selection actions

Problem. ToggleLinkDimensionsAction and OpenClosePolygonAction iterate the selection and silently skip geometries outside their target type. With a selection containing only such shapes, applyTransaction captured zero entries but still pushed an empty undo frame — so undo() became a silent no-op, tripping undo() did not change the document at all.

Fix. Pass { omitIfEmpty: true } to both transactions (a pre-existing HistoryManager option already used by the selection inspector).

Before

[undo-redo-symmetry] undo() did not change the document at all (no-op undo on non-empty stack)

After — all 35 no-op-undo artifacts report NOT REPRODUCED.


Fix 3 — d11d8c7 — Guard main-polygon dedup against vertex count mismatch in trim

Problem. In TrimSplitTool.processCurrentTrim, the "does the main polygon already exist?" dedup compared points.every((p, i) => equals(p, mainPoints[i])). When an existing polygon had more vertices than the reconstructed boundary, mainPoints[i] was undefined and PolygonSegment.equals crashed reading .type.

Before

TypeError: Cannot read properties of undefined (reading 'type')
    at Object.equals (polygon.ts:563)
    at TrimSplitTool.processCurrentTrim (TrimSplitTool.tsx:430)

After — vertex counts are compared first; mismatched polygons are skipped. All 6 TypeError artifacts now report NOT REPRODUCED.


Fix 4 — 4185f6a — Skip trims whose boundary retraces an untouched shape

Problem. walkCombinedBoundary is a heuristic (see #63): it can produce a boundary that hugs an overlapping, untouched shape instead of the union of the shapes being trimmed. Re-registering that boundary as a new polygon duplicated the untouched shape's edges → shared-edge ref-count corruption → undo/redo crash.

Example (rectangle (5.497,25.771)-(7.409,36.34) + closed polygon [(3.126,54.906),(5.093,40.035),(6.522,9.195),(13.767,10.635)], trim at (8.44, 24.126)):

Before — the trim produced a corrupt merged region whose shape contained interleaved zero-length self-loop edges:

pairs: v_2->v_3, v_3->v_3, v_3->v_4, v_4->v_4, v_4->v_23, v_23->v_23, ...

and crashed on undo with linkNext: next half-edge "he_97" does not exist.

After — the trim is detected as ambiguous (none of the boundary vertices belong to an affected shape) and skipped with an explanatory console.warn; geometry is left untouched:

[trim-split] skipped: combined boundary ( 6 edges) does not touch any of the affected shapes ( ply_fuzz2 ).
             Likely trimming an overlapping shape by mistake — leaving geometry unchanged.
NOT REPRODUCED - the minimized sequence no longer triggers the bug.

Fix 5 — 377235f — Skip degenerate self-loop edges when registering shapes

Problem. In _registerShape Phase 5, when an intersection split point resolves to the candidate edge's own destination, the split loop already adds the real prevVId → destId edge, but the trailing final segment closed into a zero-length self-loop edge (destId → destId). Self-loop pairs orphan half-edges from the edge cache and corrupt ref counts → linkNext crashes on undo/redo.

Fix. When prevVId === candidate.destId and the last split was a colinear-duplicate contact (uOnExisting < 0.5), skip the redundant final segment. Shared-edge endpoint touches (u ≈ 1 — used by colinear-merges in the shared-edge removal tests) are preserved.

After — the guard plus the trim-skip above leaves 182 / 253 fuzzer artifacts passing and the shared-edge tests still green.


Last commit — 800c0d4 — Add the fuzzer harness, adapted to the current tool registry

The regression tests live in fuzz/__tests__/fuzz.test.ts. They need the harness (fuzz/app.ts, fuzz/ops.ts, fuzz/repro.ts, fuzz/invariants.ts, ...), which is added here. The harness is adapted to the current architecture where trim-split lives inside the edit multi-tool (instead of a flat top-level tool): setActiveTool('edit') + changeToolSubTool('edit', 'trim-split'). Also adds the tsx devDependency used to run fuzz/cli.ts.

Testing

  • npm test — 37 suites, 1052 passing, 10 skipped.
  • Fuzzer corpus rerun after all fixes via fuzz/cli.ts repro <artifact> → NOT REPRODUCED for the 182 fixed cases.
  • Regression tests added for each fix in fuzz/__tests__/fuzz.test.ts and existing unit suites.

Root cause
The fuzzer's trimmed geometry was a degenerate network where the new main-polygon edge passes essentially exactly through an existing vertex of the open polygon (the intersection point lands ~3e-14 sheet-units from the existing vertex v_20, well within float noise of it).
In DCELShapeIndex._registerShape, Phase 2 collects segment intersections, then Phase 4 splits existing DCEL edges at those points. Because the intersection point was almost but not exactly on v_20:
1. addVertex(split.point) minted a near-duplicate vertex v_45 instead of reusing v_20 (the exact-id check splitVId === currentOriginId || currentDestId at DCELShapeIndex.ts:1891 never fired).
2. Two adjacent existing edges (v_15→v_20 and v_20→v_25) both got split at the same phantom point, producing the same edge key v_20|v_45 twice, each time with a different half-edge pair. The second splitEdge overwrote the _edgeCache entry, orphaning the first pair while shapes still referenced it.
3. Ref counts went inconsistent, so releaseEdge/mergeEdges later culled a half-edge pair that a shape's halfEdgeIds still referenced → linkNext: half-edge does not exist on undo/redo replay.
Fix
In _registerShape Phase 2, snap each computed intersection point onto the closest of the four relevant vertex positions (existing edge origin/dest, candidate edge origin/dest) when within SPLIT_ENDPOINT_EPSILON = 1e-9 (snapPointToNearbyVertex, DCELShapeIndex.ts). This makes addVertex return the genuinely-shared vertex ID, so the existing endpoint-touch guard fires and no phantom vertex / duplicate edge-key is ever created.
I initially also tried a Phase-5 guard, but it regressed an existing test, so I dropped it — the Phase-2 snap alone fixes the corruption at the source.
ToggleLinkDimensionsAction and OpenClosePolygonAction skipped geometries
outside their target type (polygons for link-dimensions, non-polygons for
open/close). When the selection contained only such shapes, the wrapped
applyTransaction captured zero entries yet still pushed an empty undo
frame, making undo() a silent no-op and breaking undo/redo symmetry.

Use omitIfEmpty so nothing is pushed when no shape was modified.
TrimSplitTool.processCurrentTrim deduplicates the trimmed main polygon by
comparing candidate polygons against the computed boundary with
points.every((p, i) => equals(p, mainPoints[i])). When an existing polygon
has more vertices than the boundary (e.g. an unmodified shape whose edges
only partly coincide), mainPoints[i] was undefined and PolygonSegment.equals
crashed on `b.type`. Compare vertex counts first so mismatched polygons are
skipped instead of throwing.
walkCombinedBoundary can produce a boundary that hugs an overlapping, untouched
shape instead of the union of the shapes being trimmed. Re-registering that
boundary as a new polygon duplicates the untouched shape's edges, corrupting
DCEL shared-edge ref counts and crashing undo/redo with linkNext: half-edge
does not exist.

Detect this when none of the boundary vertices belong to an affected shape and
skip the trim with a console.warn instead of corrupting the document.
When a new shape's boundary edge lands a colinear u≈0 contact on an existing
edge, Phase 5's final segment used to close into a zero-length self-loop edge
(prevVId === destId). Those self-loop pairs orphan half-edges from the edge
cache and corrupt DCEL ref counts, crashing undo/redo with
linkNext: half-edge does not exist.

The real prevVId -> destId edge was already added by the split loop, so the
trailing self-loop is redundant. Skip it unless the split is a load-bearing
shared-edge endpoint (u≈1) that colinear merges rely on.
Adds the fuzzer harness (app, ops, repro, invariants, shrink, corpus ...)
referenced by the regression tests, and adapts the trim-split op to the
current tool registration (trim-split now lives inside the 'edit'
multi-tool rather than as a flat top-level tool).

Also adds the tsx devDependency used to run fuzz/cli.ts.
@vercel

vercel Bot commented Sep 21, 2026 •

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
cad2d Ready Ready Preview Sep 21, 2026 1:12am UTC

This branch was successfully deployed

1 active deployment
Preview — 800c0d42 Deployed Sep 21, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant