diff --git a/.changeset/safe-merges-route.md b/.changeset/safe-merges-route.md new file mode 100644 index 00000000..cf5abf2e --- /dev/null +++ b/.changeset/safe-merges-route.md @@ -0,0 +1,24 @@ +--- +"@osmix/change": patch +"@osmix/router": patch +"osmix": patch +--- + +Preserve input topology during merges, conservatively reconcile compatible patch entities with the base, +validate routing-sensitive references, and insert multiple intersections in way order. Within-file duplicate +scans in the Merge app are now diagnostic only; regenerate older merged PBFs from their source inputs. Correct +the router priority queue so shortest-path searches visit lower-cost states first, and honor the one-way +direction implied by OSM roundabouts plus reverse one-way (`oneway=-1`) tags. + +Restore the original 1-meter matching behavior as explicit, cross-dataset fuzzy conflation for imported data. +Callers select transferable properties independently from patch-network attachment; exact merge behavior +remains the default. Unique, high-confidence pedestrian and one-to-one-way matches can apply automatically, +while routing properties, motor roads, ambiguity, relation involvement, and uncertain geometry require review. +Grade conflicts, restrictions, protected tags, dangling references, way collapse, and base-topology rewrites +remain blocked. Add public candidate/evidence/decision APIs, restart-safe worker review sessions, CAR/WALK +topology diagnostics, and a dedicated Merge-app review step. + +Add atomic, filter-wide conflation decisions with worker-computed previews. The Merge app can transfer +properties, attach networks, or reject every candidate matching the current filters across all pages, while +showing skipped ambiguity and overwritten decisions before confirmation. Accepted candidates now have a stable +summary and filter status, and complete bulk decision snapshots remain restart-safe. diff --git a/README.md b/README.md index c5039814..1a2310a9 100644 --- a/README.md +++ b/README.md @@ -68,6 +68,16 @@ const mergedOsm = await merge(osm, patchOsm); console.log(mergedOsm.id); ``` +The high-level merge preserves each source dataset and only reconciles compatible entities across the base +and patch. If a PBF was produced by an older release that automatically deduplicated within each input, +regenerate it from the original source files rather than trying to repair rewritten routing topology. + +Imported datasets with sub-meter coordinate offsets can opt into safe fuzzy conflation. The explicit +`conflation` configuration restores the historical 1-meter search while separating selected-property +transfer from patch-network attachment. High-confidence matches may apply automatically; ambiguous, +routing-affecting, relation-involved, or structurally uncertain candidates are reviewable or blocked. Exact +merge behavior and empty-merge identity remain unchanged when conflation is not configured. + ### Use in a Web Worker ```ts check-docs monaco-pbf diff --git a/apps/merge/DESIGN.md b/apps/merge/DESIGN.md index cee35473..703fc189 100644 --- a/apps/merge/DESIGN.md +++ b/apps/merge/DESIGN.md @@ -93,6 +93,13 @@ App-level helpers (`src/components/`): long worker tasks. - `Details`/`DetailsSummary`/`DetailsContent` — collapsible section; the standard way to make a titled, togglable region. +- `InfoTooltip` — moves optional explanatory prose behind a compact, + keyboard-accessible information trigger. Keep essential labels and current + values visible. +- `MergeStepGuide` — the standard layered explanation at the top of each + numbered merge stage. +- `StepActions` — the full-width vertical action footer for Merge workflow + stages. It keeps long decision labels contained in the narrow sidebar. - `ActionButton` — async button with spinner/transition handling. When to use what: @@ -105,6 +112,98 @@ When to use what: wizard options). - **Card** for titled sections in the sidebar blocks. +### Merge step actions + +Use `StepActions` for navigation and processing choices at the bottom of a +Merge workflow stage. Step footers remain vertical at every sidebar width: +buttons fill the available width, labels may wrap, and long OSM terminology +must not force horizontal scrolling. + +Place secondary actions first and the primary forward action last. Back, skip, +and "without" alternatives use the outline variant; preview, continue, apply, +and download actions use the default variant. Keep compact header actions, +candidate toolbars, and other non-footer controls in their existing horizontal +groups. Do not relax the global button primitive's single-line behavior to fix +a workflow-footer layout. + +## Merge workflow guidance + +Every merge stage must explain itself where the user makes the decision. Keep +the explanation layered so that experienced users can scan the workflow while +new users can inspect the consequences before applying anything: + +1. Show one plain-language summary at the top of the numbered step card, before + controls or results. +2. Follow it with a collapsed **How this step works** disclosure using + `MergeStepGuide`. Do not duplicate these disclosures at individual call + sites; add or revise the app-private guide registry instead. +3. In the expanded content, identify the inputs being read, changes that may + occur, invariants the step preserves, and its output. Include a warning only + when the user can make an irreversible or topology-affecting choice. +4. Reset the disclosure when moving between steps. Opening help must never + change a form value, review decision, workflow state, or worker operation. + +Use the merge terms consistently: + +- **Base OSM** is the authoritative existing dataset whose identity and + untouched geometry are preserved unless a same-ID patch update explicitly + replaces them. +- **Patch OSM** contains imported additions and updates. +- **Direct merge** adds patch-only entities and applies same-ID updates. +- **Exact reconciliation** combines different IDs only when their serialized + coordinates or ordered geometry and routing context agree. +- **Imported-data matching** is the optional proximity workflow. **Property + transfer** copies only selected tag values; **network attachment** rewrites + only accepted references in patch-created ways. +- **Intersection creation** connects compatible same-grade crossings while + leaving ambiguous and grade-separated crossings disconnected. +- **Review each merge stage** exposes previews and checkpoints. **Run automatic + merge** skips those checkpoints and uses only behavior explicitly configured + for the automatic path. + +Labels must state what a control changes instead of relying on a placeholder. +Put concise supporting text next to unfamiliar controls and connect it with +`aria-describedby`. Humanize internal status and reason-code values in visible +copy, but do not change the stable values used by workers or saved decisions. + +`Details` is the shared disclosure primitive. Its open-state styles target Base +UI's `data-panel-open` attribute. Disclosure triggers remain keyboard +accessible, and decorative chevrons are hidden from assistive technology. + +### Explanatory diagrams + +Use a compact SVG only when topology or data flow is materially clearer as a +picture. Merge diagrams follow these constraints: + +- Provide a fixed `viewBox` and responsive `width: 100%`; never give the SVG a + fixed rendered width that can overflow the sidebar. +- Give each diagram an accessible name and description with React + `useId()`-backed `` and `<desc>` elements. +- Use semantic foreground, muted, info, success, warning, destructive, and + border tokens. Never encode meaning by color alone: pair colors with labels, + shapes, or solid/dashed line styles. +- Set connector strokes to `vector-effect="non-scaling-stroke"` so they remain + legible at narrow widths. +- Avoid animation and `<foreignObject>`. SVG text must remain understandable at + both 320 px and 512 px sidebar widths. + +### Browser test boundaries + +Keep the real Monaco Merge journey focused on integration behavior that needs +an actual parsed OSM and worker-backed merge. Load each input once, verify its +real metadata, and advance through the workflow without repeating presentation +checks that can run against production components in the lightweight guidance +harness. + +Use that harness for responsive geometry, long-label and long-filename +containment, accessible control names, and controlled action-state transitions. +Run the real Merge journey, guidance harness, and worker-runtime coverage as +ordered Playwright projects. This prevents additional Chromium contexts or +intensive Web Worker activity from competing with MapLibre rendering and PBF +parsing on a small CI runner. The real journey uses one app worker; the +dedicated worker-runtime project retains single-worker, multi-worker, +replication, recovery, and disposal coverage. + ## Loading, progress & status - Quick/inline waits: `Spinner`. diff --git a/apps/merge/README.md b/apps/merge/README.md index 1328254a..08dbe99b 100644 --- a/apps/merge/README.md +++ b/apps/merge/README.md @@ -4,10 +4,12 @@ Osmix Merge is a Vite + React app for comparing and reconciling OpenStreetMap PB ## Highlights -- Load “base” and “patch” `.osm.pbf` files, preview differences, and step through merge tasks (direct merge, node/way deduplication, intersection creation). +- Load authoritative base and imported patch `.osm.pbf` files, preview differences, and step through direct + merge, exact reconciliation, optional imported-data matching, and intersection creation. - Select Auto, Full, or View loading according to the dataset and available browser memory. - Visualize both datasets with raster previews produced on the worker thread plus interactive vector overlays for selected entities. -- Inspect individual OSM files, find duplicate entities, and apply the generated changes back into the in-memory index. +- Inspect individual OSM files for possible duplicate entities without mutating the source data. +- Opt in to reviewed, one-meter proximity matching for importing selected properties or attaching compatible imported networks without rewriting base geometry. - Built-in Nominatim search, entity lookups, and task logging keep large merges manageable. ## Prerequisites @@ -47,7 +49,8 @@ pnpm exec playwright install --with-deps # first run pnpm run --filter @osmix/merge test:e2e ``` -Tests load sample fixtures from `fixtures/monaco.pbf` and exercise both Merge and Inspect flows. +The worker harness loads `fixtures/monaco.pbf`; the guidance harness renders the real merge-step disclosure +components and checks keyboard interaction, state isolation, and narrow-sidebar layout. ## Core workflow @@ -67,24 +70,86 @@ projections, storage estimate, budgets, and selection reasons. Check System dist memory class from separately tested `ArrayBuffer` and `SharedArrayBuffer` ceilings; typed-array element counts are derived from those tested byte ceilings. -When View omits the all-node index, merge, node/way deduplication, complete/smart extraction, routing, and +When View omits the all-node index, merge, exact node/way reconciliation, complete/smart extraction, routing, and other all-node-dependent controls are disabled with an explanation and a **Reload using Full** action. Simple in-stream extraction remains available. The app does not build the large index synchronously on first use. ### Merge view (default route) -1. **Select OSM PBF files** – Upload base + patch files and review metadata. The files stay local thanks to the File System Access API. -2. **Review changeset** – Each step runs an operation on the worker (`osm.worker.ts`) that uses `@osmix/core` and `@osmix/change` to generate or update an `OsmixChangeset`. Logs stream into the sidebar while progress indicators update the UI. -3. **Inspect intermediary results** – Toggle MapLibre vector overlays to compare base/patch rasters, click features to see details, and jump the map to selected entities. -4. **Apply actions** – Deduplicate nodes or ways, generate direct changes, create intersections, and download the resulting change list as JSON. Applying the final changes replaces the in-memory base dataset. - -The stepper resets selection state between actions, and you can jump backward or forward if you need to rerun a task. +1. **Select merge inputs** – Choose `Base OSM — authoritative existing dataset` and + `Patch OSM — imported additions and updates`. Both files stay local. Select **Review each merge stage** for + previews and checkpoints or **Run automatic merge** to use the configured automatic path. +2. **Review diagnostics** – Optional base and patch scans report possible within-file duplicates without + mutating either input. Nearby roads can be intentionally separate because of topology, access, or grade + separation. +3. **Preview the direct merge** – Patch-only entities are added, same-ID patch updates take precedence, and + base-only entities remain. In the reviewed workflow this is a preview until the cumulative merge is + accepted. +4. **Match imported data (optional)** – Discover nearby cross-dataset candidates. Property transfer copies + selected tags onto preserved base entities; network attachment rewrites only accepted references in + patch-created ways. Ambiguous and routing-affecting candidates remain reviewable. +5. **Reconcile exact matches** – Combine compatible entities with different IDs only when coordinates or + ordered geometry agree at OSM precision. Base IDs are preserved and patch references are rewritten. +6. **Create intersections** – Connect compatible same-grade crossings. Unsafe endpoint reuse, ambiguous + crossings, and grade-separated roads remain separate. +7. **Inspect and download** – Compare the result on the map and download the merged PBF or change summary. + The result stays in memory until downloaded, and the original input files are never modified. + +Each numbered stage includes a concise summary and a collapsed **How this step works** explanation of its +inputs, possible changes, safety guarantees, and output. The stepper resets selection state between actions, +and you can jump backward or forward if you need to rerun a task. +In verified mode, the direct merge is first shown as a preview. The app then regenerates and applies one +cumulative direct-merge plus optional reconciliation changeset from the untouched source inputs. Intersection +changes are generated only after that merged base has been rebuilt and indexed, so newly added patch ways are +included in the crossing scan. + +The automatic workflow skips diagnostic scans and intermediate checkpoints. Imported-data matching remains +off unless configured explicitly; when enabled, automatic mode applies only high-confidence automatic +candidates and reports unresolved candidates without accepting them. Once the first generated changeset is +applied, cancellation cannot restore the prior in-memory workflow state, though the source files remain +untouched and can be loaded again. + +### Safe imported-data matching + +The original Merge tool used a one-meter proximity search to combine datasets whose independently created +entities do not have identical OSM coordinates. That remains useful for GeoJSON, Shapefile, and other +non-OSM sources, but proximity alone is unsafe for road topology: nearby surface and tunnel roads, parallel +paths, school boundaries, and ambiguous intersections must remain separate. + +**Match imported data** restores that workflow as an explicit opt-in conflation stage: + +- **Property transfer** preserves the base entity ID, coordinates, references, and relation membership while + copying only the tag keys entered in the form. Patch values win for those selected keys; missing patch + values never delete base values. Structural keys are blocked, and routing-affecting keys require review. +- **Network attachment** preserves the base node and rewrites only accepted references in imported patch + ways. Automatic matches must be unique and agree on routing family, grade context, and local bearing. + Restrictions, relation-member rewrites, way collapse, and other integrity hazards remain blocked. + +The default radius is one meter. High-confidence matches are automatic; accepted, review, blocked, unmatched, +and rejected candidates remain visible through paged status, entity, and reason filters. Selecting a candidate +draws the imported source and proposed base target together on the map and shows its geometry evidence and +property diff. Review decisions are stable candidate-ID records and are restored with the worker session. + +The **Filtered matches** toolbar applies property transfer, network attachment, or rejection to every candidate +matching the current filters across all pages. Automatic matches already apply unless rejected. Before changing +decisions, the app shows how many automatic and review candidates are eligible, how many blocked or ambiguous +matches will be skipped, and how many prior decisions will be replaced. Accepted and rejected rows may leave the +active status filter, so the list returns to its first page after a successful action. + +In verified mode, discovery happens against the untouched base and patch before either dataset is changed. +The app then generates one cumulative direct, exact-reconciliation, and accepted-conflation changeset, +reports CAR and WALK graph-count/component deltas, and applies it atomically. Intersection creation remains a +separate final stage. **Run automatic merge** stays exact-only unless matching was explicitly enabled; when enabled, +it accepts only automatic candidates and reports unresolved counts without silently approving them. The +enabled fast path uses the same session generation and CAR safety gate, applies the cumulative result, then +creates intersections against that indexed base. The patch is cleared only after both stages and merged-file +metadata refresh complete. ### Inspect view (`/inspect`) -- Load a single PBF, run duplicate detection, and page through the resulting change list. +- Load a single PBF, run diagnostic duplicate detection, and page through the resulting candidate list. - Fit to the file’s bounding box, search for entities, and drill into their tags and relations. -- Apply deduplications directly to the dataset and immediately preview the updated geometry. +- Investigate candidates against the source data; the Inspect view does not apply proximity-based changes. ## Map & rendering stack @@ -97,6 +162,7 @@ The stepper resets selection state between actions, and you can jump backward or - Stateful loading, IndexedDB writes, and changesets stay on the control worker. Read-only tiles and queries use available compute workers, with queued MapLibre tile requests cancelled when the map no longer needs them. - Workers cache `Osmix` instances keyed by dataset id, share their backing buffers, expose change pagination, and return transferable typed arrays whenever possible. - If a single non-shared worker restarts, datasets previously loaded from IndexedDB are reconstructed with a read-only replay before the slot accepts more work. One-shot mutations and IndexedDB writes are never retried. +- Active imported-data discovery options, filters, and review decisions are replayed after a recoverable control-worker restart while both untouched inputs still exist. Applying the cumulative merge invalidates that review session. - Local PBF files are hashed incrementally from `File.stream()` in a worker, avoiding a second whole-file input buffer. PBF URLs are hashed while the parser consumes a single response, then re-keyed to the final lowercase SHA-256 without copying the dataset buffers. @@ -128,6 +194,15 @@ See [Australia-scale manual verification](./AUSTRALIA-PBF-CHECKLIST.md) for the - **A core typed-array allocation failed** – The panel identifies the mandatory entity column and compares its single-buffer requirement with the current browser's tested ceiling. Auto, Full, and View retain core entity columns, so use a smaller regional extract when the panel says changing profiles cannot help. +- **A file was merged with an older Osmix release** – Older merges may have normalized each input before + combining them, which can change routing topology. Regenerate the output from the original base and patch + PBFs; the resulting file cannot be repaired reliably after references have been rewritten. +- **A merge reports new routing-integrity problems** – The result was rejected before replacing the base. + Inspect the reported entity IDs for missing references, degenerate highways, or detached turn restrictions, + then correct the source data rather than discarding the affected restriction. +- **A proximity candidate is blocked** – Review its reason code and map comparison. Grade conflicts, + restrictions, relation membership, and changes that would collapse a way cannot be overridden. Multiple + targets and other uncertain candidates require an explicit accepted target or can be left unchanged. ## Related packages diff --git a/apps/merge/e2e/guidance-harness.html b/apps/merge/e2e/guidance-harness.html new file mode 100644 index 00000000..c2375b95 --- /dev/null +++ b/apps/merge/e2e/guidance-harness.html @@ -0,0 +1,12 @@ +<!doctype html> +<html lang="en"> + <head> + <meta charset="UTF-8" /> + <meta name="viewport" content="width=device-width, initial-scale=1.0" /> + <title>Osmix merge guidance harness + + +
+ + + diff --git a/apps/merge/e2e/guidance-harness.tsx b/apps/merge/e2e/guidance-harness.tsx new file mode 100644 index 00000000..46e2a043 --- /dev/null +++ b/apps/merge/e2e/guidance-harness.tsx @@ -0,0 +1,202 @@ +import { useState } from "react"; +import { createRoot } from "react-dom/client"; + +import "../src/main.css"; +import { + AutomaticMergeProgress, + CONFLATION_AUTOMATIC_MERGE_STEPS, +} from "../src/components/automatic-merge-progress"; +import { InfoTooltip } from "../src/components/info-tooltip"; +import { MergeStepGuide } from "../src/components/merge-step-guide"; +import { OsmInputCardHeader } from "../src/components/osm-input-card-header"; +import { StepActions } from "../src/components/step-actions"; +import { Button } from "../src/components/ui/button"; +import { Card, CardContent } from "../src/components/ui/card"; + +interface InputHarnessState { + baseDownloads: number; + baseLoaded: boolean; + patchDownloads: number; + patchLoaded: boolean; +} + +interface HarnessState { + decision: string; + inputs: InputHarnessState; + propertyKeys: string; + workerCalls: number; + workflowStep: string; +} + +const harnessState: HarnessState = { + decision: "pending", + inputs: { + baseDownloads: 0, + baseLoaded: true, + patchDownloads: 0, + patchLoaded: true, + }, + propertyKeys: "name surface kerb", + workerCalls: 0, + workflowStep: "direct", +}; + +function GuidanceHarness() { + const [propertyKeys, setPropertyKeys] = useState(harnessState.propertyKeys); + const [workerCalls, setWorkerCalls] = useState(harnessState.workerCalls); + const [automaticStepIndex, setAutomaticStepIndex] = useState(0); + const [inputs, setInputs] = useState(harnessState.inputs); + const updateInputs = (update: (current: InputHarnessState) => InputHarnessState) => { + setInputs((current) => { + const next = update(current); + harnessState.inputs = next; + return next; + }); + }; + + return ( +
+
+ + { + updateInputs((current) => ({ ...current, baseLoaded: false })); + }} + onDownload={async () => { + updateInputs((current) => ({ + ...current, + baseDownloads: current.baseDownloads + 1, + })); + }} + title="Base OSM — authoritative existing dataset" + /> + {inputs.baseLoaded ? null : ( + + + + )} + + + + { + updateInputs((current) => ({ ...current, patchLoaded: false })); + }} + onDownload={async () => { + updateInputs((current) => ({ + ...current, + patchDownloads: current.patchDownloads + 1, + })); + }} + title="Patch OSM — imported additions and updates" + /> + {inputs.patchLoaded ? null : ( + + + + )} + +
+ +
+ +
+ Candidate statuses + + Automatic matches apply unless rejected. Review matches need a decision. + +
+ +
+ + +
+ + + +
+
Workflow step
+
{harnessState.workflowStep}
+
Decision
+
{harnessState.decision}
+
+ +
+ + + + + + + + + + + + +
+ + +
+
+ ); +} + +window.guidanceHarness = { + readState: () => ({ ...harnessState, inputs: { ...harnessState.inputs } }), +}; + +createRoot(document.getElementById("root")!).render(); + +declare global { + interface Window { + guidanceHarness: { + readState: () => HarnessState; + }; + } +} diff --git a/apps/merge/e2e/guidance.spec.ts b/apps/merge/e2e/guidance.spec.ts new file mode 100644 index 00000000..39159292 --- /dev/null +++ b/apps/merge/e2e/guidance.spec.ts @@ -0,0 +1,243 @@ +import { expect, test } from "@playwright/test"; + +interface HarnessState { + decision: string; + inputs: { + baseDownloads: number; + baseLoaded: boolean; + patchDownloads: number; + patchLoaded: boolean; + }; + propertyKeys: string; + workerCalls: number; + workflowStep: string; +} + +test.beforeEach(async ({ page }) => { + await page.goto("/e2e/guidance-harness.html"); +}); + +test("guidance starts collapsed and supports mouse and keyboard disclosure", async ({ page }) => { + const trigger = page.getByRole("button", { name: "How this step works" }); + const details = page.locator('[data-slot="merge-step-guide-details"]'); + + await expect(trigger).toHaveAttribute("aria-expanded", "false"); + await expect(details).toBeHidden(); + + await trigger.click(); + await expect(trigger).toHaveAttribute("aria-expanded", "true"); + await expect(details).toBeVisible(); + await expect(details.getByRole("heading", { level: 3 })).toHaveCount(4); + + await trigger.click(); + await expect(trigger).toHaveAttribute("aria-expanded", "false"); + + await trigger.focus(); + await trigger.press("Enter"); + await expect(trigger).toHaveAttribute("aria-expanded", "true"); + + await trigger.press("Space"); + await expect(trigger).toHaveAttribute("aria-expanded", "false"); +}); + +test("opening help leaves merge inputs, decisions, workflow state, and worker calls unchanged", async ({ + page, +}) => { + const readState = () => page.evaluate(() => window.guidanceHarness.readState()); + const before = await readState(); + const trigger = page.getByRole("button", { name: "How this step works" }); + + await trigger.click(); + await trigger.press("Enter"); + await trigger.press("Space"); + + await expect(page.getByLabel("OSM tag keys to transfer")).toHaveValue(before.propertyKeys); + await expect(page.getByTestId("workflow-state")).toContainText(before.workflowStep); + await expect(page.getByTestId("workflow-state")).toContainText(before.decision); + expect(await readState()).toEqual(before); +}); + +test("info tooltips reveal long guidance on hover and keyboard activation", async ({ page }) => { + const trigger = page.getByRole("button", { name: "About candidate statuses" }); + const tooltip = page.locator('[data-slot="info-tooltip-content"]'); + + await expect(tooltip).toBeHidden(); + await trigger.hover(); + await expect(tooltip).toContainText("Automatic matches apply unless rejected"); + + await page.mouse.move(0, 0); + await expect(tooltip).toBeHidden(); + + await trigger.focus(); + await trigger.press("Enter"); + await expect(tooltip).toBeVisible(); + await expect(trigger).toHaveAttribute("aria-expanded", "true"); + + await page.keyboard.press("Escape"); + await expect(tooltip).toBeHidden(); + await expect(trigger).toHaveAttribute("aria-expanded", "false"); +}); + +test("automatic merge progress advances completed, running, and remaining steps", async ({ + page, +}) => { + const progress = page.getByRole("list", { name: "Automatic merge progress" }); + + await expect(page.locator('[data-slot="automatic-merge-elapsed"]')).toHaveText("9:42"); + await expect(page.getByRole("progressbar")).toHaveCount(0); + await expect(progress.locator('[data-status="completed"]')).toHaveCount(0); + await expect(progress.locator('[data-status="running"]')).toContainText( + "Discover imported-data matches", + ); + await expect( + progress.locator('[data-status="running"] [data-slot="automatic-merge-latest-message"]'), + ).toContainText("Worker message for Discover imported-data matches"); + await expect(progress.locator('[data-status="remaining"]')).toHaveCount(4); + + await page.getByRole("button", { name: "Advance automatic merge" }).click(); + + await expect(progress.locator('[data-status="completed"]')).toHaveCount(1); + await expect(progress.locator('[data-status="running"]')).toContainText( + "Generate and validate merge changes", + ); + await expect( + progress.locator('[data-status="running"] [data-slot="automatic-merge-latest-message"]'), + ).toContainText("Worker message for Generate and validate merge changes"); + await expect(page.getByRole("status")).toContainText("1 of 5 steps completed"); +}); + +test("loaded input cards remain usable and contained without loading a PBF", async ({ page }) => { + await page.setViewportSize({ width: 320, height: 900 }); + const harness = page.getByTestId("input-card-harness"); + const baseCard = harness + .locator('[data-slot="card"]') + .filter({ hasText: "Base OSM — authoritative existing dataset" }); + const patchCard = harness + .locator('[data-slot="card"]') + .filter({ hasText: "Patch OSM — imported additions and updates" }); + const longBaseName = + "an-extremely-long-base-osm-filename-that-must-not-push-actions-outside-the-card.pbf"; + + await expect(baseCard.locator('[data-slot="card-description"]')).toHaveText(longBaseName); + await expect(baseCard.locator('[data-slot="card-description"]')).toHaveAttribute( + "title", + longBaseName, + ); + await expect(baseCard.getByRole("button", { name: "Download base OSM" })).toBeVisible(); + await expect(baseCard.getByRole("button", { name: "Clear base OSM file" })).toBeVisible(); + await expect(patchCard.getByRole("button", { name: "Download patch OSM" })).toBeVisible(); + await expect(patchCard.getByRole("button", { name: "Clear patch OSM file" })).toBeVisible(); + await expect(patchCard.getByRole("button", { name: "Save to storage" })).toHaveCount(0); + await expect + .poll(() => harness.evaluate((element) => element.scrollWidth <= element.clientWidth)) + .toBe(true); + + await baseCard.getByRole("button", { name: "Download base OSM" }).click(); + await patchCard.getByRole("button", { name: "Download patch OSM" }).click(); + await expect + .poll(async () => (await page.evaluate(() => window.guidanceHarness.readState())).inputs) + .toMatchObject({ baseDownloads: 1, patchDownloads: 1 }); + + await baseCard.getByRole("button", { name: "Clear base OSM file" }).click(); + await expect(baseCard.getByRole("button", { name: "Open base OSM" })).toBeVisible(); + await expect(baseCard.locator('[data-slot="card-description"]')).toHaveCount(0); + await expect(patchCard.locator('[data-slot="card-description"]')).toHaveText("monaco.test.pbf"); + + await patchCard.getByRole("button", { name: "Clear patch OSM file" }).click(); + await expect(patchCard.getByRole("button", { name: "Open patch OSM" })).toBeVisible(); + await expect(patchCard.locator('[data-slot="card-description"]')).toHaveCount(0); +}); + +test("workflow step actions remain contained at supported sidebar widths", async ({ page }) => { + for (const width of [320, 512]) { + await page.setViewportSize({ width, height: 900 }); + const actionGroups = page.getByRole("group", { name: /step actions$/i }); + await expect(actionGroups).toHaveCount(3); + + for (const actionGroup of await actionGroups.all()) { + const measurements = await actionGroup.evaluate((group) => { + const groupBounds = group.getBoundingClientRect(); + const buttons = [...group.querySelectorAll('[data-slot="button"]')]; + const buttonBounds = buttons.map((button) => button.getBoundingClientRect()); + return { + buttons: buttonBounds.map((bounds) => ({ + bottom: bounds.bottom, + left: bounds.left, + right: bounds.right, + top: bounds.top, + })), + clientWidth: group.clientWidth, + groupLeft: groupBounds.left, + groupRight: groupBounds.right, + scrollWidth: group.scrollWidth, + }; + }); + + expect(measurements.scrollWidth).toBeLessThanOrEqual(measurements.clientWidth); + expect(measurements.buttons).toHaveLength(2); + expect(measurements.buttons[0].left).toBeGreaterThanOrEqual(measurements.groupLeft); + expect(measurements.buttons[0].right).toBeLessThanOrEqual(measurements.groupRight); + expect(measurements.buttons[1].left).toBeGreaterThanOrEqual(measurements.groupLeft); + expect(measurements.buttons[1].right).toBeLessThanOrEqual(measurements.groupRight); + expect(measurements.buttons[1].top).toBeGreaterThan(measurements.buttons[0].bottom); + } + + const reconciliationActions = page.getByRole("group", { + name: "Reconciliation step actions", + }); + const secondaryAction = reconciliationActions.getByRole("button", { + name: "Preview without exact reconciliation", + }); + const primaryAction = reconciliationActions.getByRole("button", { + name: "Preview with exact reconciliation", + }); + await expect(secondaryAction).toHaveClass(/border/); + await expect(primaryAction).toHaveClass(/bg-primary/); + await secondaryAction.focus(); + await expect(secondaryAction).toBeFocused(); + await page.keyboard.press("Tab"); + await expect(primaryAction).toBeFocused(); + } +}); + +test("guidance and diagrams remain contained at supported sidebar widths", async ({ page }) => { + for (const width of [320, 512]) { + await page.setViewportSize({ width, height: 800 }); + const trigger = page.getByRole("button", { name: "How this step works" }); + if ((await trigger.getAttribute("aria-expanded")) === "false") await trigger.click(); + + const diagram = page.locator('[data-slot="merge-step-guide"] svg[data-diagram]'); + await expect(diagram).toHaveCount(1); + await expect(diagram).toBeVisible(); + + const measurements = await page.evaluate(() => { + const sidebar = document.querySelector('[data-testid="guidance-sidebar"]'); + const svg = document.querySelector( + '[data-slot="merge-step-guide"] svg[data-diagram]', + ); + if (!sidebar || !svg) throw new Error("Guidance harness did not render"); + + const sidebarBounds = sidebar.getBoundingClientRect(); + const svgBounds = svg.getBoundingClientRect(); + const diagramTextHeights = [...svg.querySelectorAll("text")].map( + (label) => label.getBoundingClientRect().height, + ); + return { + documentClientWidth: document.documentElement.clientWidth, + documentScrollWidth: document.documentElement.scrollWidth, + minimumDiagramTextHeight: Math.min(...diagramTextHeights), + sidebarLeft: sidebarBounds.left, + sidebarRight: sidebarBounds.right, + svgHeight: svgBounds.height, + svgLeft: svgBounds.left, + svgRight: svgBounds.right, + }; + }); + + expect(measurements.documentScrollWidth).toBeLessThanOrEqual(measurements.documentClientWidth); + expect(measurements.minimumDiagramTextHeight).toBeGreaterThanOrEqual(9); + expect(measurements.svgHeight).toBeLessThanOrEqual(330); + expect(measurements.svgLeft).toBeGreaterThanOrEqual(measurements.sidebarLeft); + expect(measurements.svgRight).toBeLessThanOrEqual(measurements.sidebarRight); + } +}); diff --git a/apps/merge/e2e/merge-base-loading.spec.ts b/apps/merge/e2e/merge-base-loading.spec.ts new file mode 100644 index 00000000..040bb264 --- /dev/null +++ b/apps/merge/e2e/merge-base-loading.spec.ts @@ -0,0 +1,106 @@ +import { fileURLToPath } from "node:url"; + +import { expect, test, type Locator, type Page } from "@playwright/test"; + +async function loadPbf(card: Locator, page: Page, path: string) { + await card.getByRole("button", { name: "Open file" }).click(); + const fileChooserPromise = page.waitForEvent("filechooser"); + await page.getByRole("menuitem", { name: /^OSM PBF/ }).click(); + const fileChooser = await fileChooserPromise; + await fileChooser.setFiles(path); + const fileInfo = card.getByRole("button", { name: "File info" }); + const loadFailure = card.getByRole("alert"); + await expect(fileInfo.or(loadFailure)).toBeVisible({ + timeout: 120_000, + }); + if (await loadFailure.isVisible()) { + throw new Error(`OSM load failed: ${await loadFailure.innerText()}`); + } +} + +const MONACO_PBF = fileURLToPath(new URL("../../../fixtures/monaco.pbf", import.meta.url)); + +test("loads both inputs once and reaches exact reconciliation", async ({ page }) => { + // Keep this worker-backed journey to one load per input. Input-card actions, + // clearing, and responsive geometry run against the production header in the + // guidance harness instead of repeating PBF parsing and MapLibre resizing here. + // Multi-worker replication has dedicated coverage in worker-runtime.spec.ts; + // keeping this UI journey to one app worker avoids duplicating both inputs. + await page.addInitScript(() => { + Object.defineProperty(navigator, "hardwareConcurrency", { + configurable: true, + get: () => 1, + }); + }); + await page.goto("/"); + await expect.poll(() => page.evaluate(() => window.osmWorker?.workerCount ?? 0)).toBe(1); + await page.getByRole("tab", { name: "Merge" }).click(); + + const baseCard = page + .locator('[data-slot="card"]') + .filter({ hasText: "Base OSM — authoritative existing dataset" }) + .first(); + const patchCard = page + .locator('[data-slot="card"]') + .filter({ hasText: "Patch OSM — imported additions and updates" }) + .first(); + + await expect(baseCard.getByRole("button", { name: "Open file" })).toBeVisible(); + await expect(patchCard.getByRole("button", { name: "Open file" })).toBeVisible(); + + await loadPbf(baseCard, page, MONACO_PBF); + await expect(baseCard.locator('[data-slot="card-description"]')).toHaveText("monaco.pbf"); + await expect(baseCard.getByRole("button", { name: "Download base OSM" })).toBeVisible(); + await expect(baseCard.getByRole("button", { name: "Clear base OSM file" })).toBeVisible(); + const fileInfo = baseCard.getByRole("button", { name: "File info" }); + await fileInfo.click(); + await expect(baseCard.getByRole("row").filter({ hasText: "file name" })).toContainText( + "monaco.pbf", + ); + await expect(baseCard).toContainText("14,286"); + + // Use the one Monaco PBF tracked by Git for both roles. The guidance harness + // covers distinct displayed filenames without depending on local-only files. + await loadPbf(patchCard, page, MONACO_PBF); + await expect(patchCard.locator('[data-slot="card-description"]')).toHaveText("monaco.pbf"); + await expect(patchCard.getByRole("button", { name: "Download patch OSM" })).toBeVisible(); + await expect(patchCard.getByRole("button", { name: "Clear patch OSM file" })).toBeVisible(); + await expect(patchCard.getByRole("button", { name: "Save to storage" })).toHaveCount(0); + await patchCard.getByRole("button", { name: "File info" }).click(); + await expect(patchCard.getByRole("row").filter({ hasText: "file name" })).toContainText( + "monaco.pbf", + ); + + await page.getByRole("button", { name: /Review each merge stage/ }).click(); + await expect(page.getByText(/2: Inspect base OSM/i)).toBeVisible(); + await expect(page.getByRole("button", { name: "Skip base diagnostic" })).toBeVisible(); + + await page.getByRole("button", { name: "Skip base diagnostic" }).click(); + await expect(page.getByText(/4: Inspect patch OSM/i)).toBeVisible(); + await expect(page.getByRole("button", { name: "Skip patch diagnostic" })).toBeVisible(); + await expect(page.getByRole("button", { name: "Download JSON changes" })).toHaveCount(0); + + await page.getByRole("button", { name: "Skip patch diagnostic" }).click(); + await expect(page.getByText(/6: Direct merge/i)).toBeVisible(); + await expect(page.getByRole("button", { name: "Preview direct merge" })).toBeVisible(); + + await page.getByRole("button", { name: "Preview direct merge" }).click(); + await expect(page.getByText(/Review direct merge/i)).toBeVisible(); + await page.getByRole("button", { name: "Continue to matching and reconciliation" }).click(); + await expect(page.getByText(/Reconcile matching entities/i)).toBeVisible(); + + const reconciliationActions = page.getByRole("group", { + name: "Exact reconciliation actions", + }); + const withoutExact = reconciliationActions.getByRole("button", { + name: "Preview without exact reconciliation", + }); + const withExact = reconciliationActions.getByRole("button", { + name: "Preview with exact reconciliation", + }); + + await withoutExact.focus(); + await expect(withoutExact).toBeFocused(); + await page.keyboard.press("Tab"); + await expect(withExact).toBeFocused(); +}); diff --git a/apps/merge/playwright.config.ts b/apps/merge/playwright.config.ts index 8acffd65..5bc719d8 100644 --- a/apps/merge/playwright.config.ts +++ b/apps/merge/playwright.config.ts @@ -2,7 +2,7 @@ import { defineConfig, devices } from "@playwright/test"; export default defineConfig({ testDir: "./e2e", - testMatch: "worker-runtime.spec.ts", + testMatch: ["guidance.spec.ts", "merge-base-loading.spec.ts", "worker-runtime.spec.ts"], timeout: 120_000, use: { baseURL: "http://127.0.0.1:4173", @@ -15,7 +15,24 @@ export default defineConfig({ }, projects: [ { - name: "chromium", + name: "merge-integration", + testMatch: ["merge-base-loading.spec.ts"], + use: { ...devices["Desktop Chrome"] }, + }, + { + // Keep even the lightweight browser harness off the runner while the real + // Merge journey is parsing PBFs and rendering MapLibre. + name: "guidance", + dependencies: ["merge-integration"], + testMatch: ["guidance.spec.ts"], + use: { ...devices["Desktop Chrome"] }, + }, + { + // Worker restart and multi-worker tests run last so their nested workers + // cannot starve either UI project on small CI runners. + name: "worker-runtime", + dependencies: ["guidance"], + testMatch: ["worker-runtime.spec.ts"], use: { ...devices["Desktop Chrome"] }, }, ], diff --git a/apps/merge/src/blocks/inspect.tsx b/apps/merge/src/blocks/inspect.tsx index 79d15945..45b28af3 100644 --- a/apps/merge/src/blocks/inspect.tsx +++ b/apps/merge/src/blocks/inspect.tsx @@ -1,8 +1,7 @@ import { useAtom, useSetAtom } from "jotai"; -import { MergeIcon } from "lucide-react"; import type { OsmInfo } from "osmix"; import type { OsmFileType } from "osmix"; -import { Suspense, useMemo } from "react"; +import { Suspense } from "react"; import ActionButton from "../components/action-button"; import { Details, DetailsContent, DetailsSummary } from "../components/details"; @@ -18,9 +17,9 @@ import StoredOsmList from "../components/stored-osm-list"; import { Card, CardContent, CardHeader } from "../components/ui/card"; import { useFlyToEntity, useFlyToOsmBounds } from "../hooks/map"; import { useOsmFile } from "../hooks/osm"; +import { WITHIN_DATASET_DIAGNOSTIC_OPTIONS } from "../lib/merge-workflow"; import { BASE_OSM_KEY } from "../settings"; import { changesetStatsAtom } from "../state/changes"; -import { Log } from "../state/log"; import { selectOsmEntityAtom } from "../state/osm"; import { osmLoadingAbortControllerAtom } from "../state/status"; import { osmWorker } from "../state/worker"; @@ -37,21 +36,6 @@ export default function InspectBlock({ const setLoadingState = useSetAtom(osmLoadingAbortControllerAtom); const [changesetStats, setChangesetStats] = useAtom(changesetStatsAtom); - const applyChanges = async () => { - if (!baseOsm.osm) throw Error("Osm has not been loaded."); - const task = Log.startTask("Applying changes to OSM..."); - await osmWorker.applyChangesAndReplace(baseOsm.osm.id); - task.update("Refreshing OSM index..."); - const newOsm = await osmWorker.get(baseOsm.osm.id); - baseOsm.setOsm(newOsm); - setChangesetStats(null); - task.end("Changes applied!"); - }; - - const hasZeroChanges = useMemo(() => { - return changesetStats == null || changesetStats.totalChanges === 0; - }, [changesetStats]); - if (!baseOsm.osm || !baseOsm.osmInfo || !baseOsm.fileInfo) { return (
@@ -106,14 +90,20 @@ export default function InspectBlock({ return (
+

+ Scan for possible duplicates without changing the dataset. Nearby OSM entities may belong to + different roads, layers, or restrictions, so candidates must be investigated against the + source data instead of applied automatically. +

{ if (!baseOsm.osm) throw Error("Osm has not been loaded."); - const changes = await osmWorker.generateChangeset(baseOsm.osm.id, baseOsm.osm.id, { - deduplicateNodes: true, - deduplicateWays: true, - }); + const changes = await osmWorker.generateChangeset( + baseOsm.osm.id, + baseOsm.osm.id, + WITHIN_DATASET_DIAGNOSTIC_OPTIONS, + ); setChangesetStats(changes); }} > @@ -123,7 +113,7 @@ export default function InspectBlock({ {changesetStats != null && ( <> - Changeset + Diagnostic candidates }> @@ -144,12 +134,6 @@ export default function InspectBlock({ - - {!hasZeroChanges && ( - }> - Apply changes - - )} )}
diff --git a/apps/merge/src/blocks/merge.tsx b/apps/merge/src/blocks/merge.tsx index cfdc8129..ae4746f8 100644 --- a/apps/merge/src/blocks/merge.tsx +++ b/apps/merge/src/blocks/merge.tsx @@ -13,26 +13,39 @@ import { SearchCodeIcon, SkipForwardIcon, StopCircleIcon, - XIcon, } from "lucide-react"; -import { changeStatsSummary } from "osmix"; -import { Suspense, useMemo } from "react"; +import { + changeStatsSummary, + type OsmConflationBulkDecisionRequest, + type OsmConflationDecision, +} from "osmix"; +import { Suspense, useMemo, useState } from "react"; import ActionButton from "../components/action-button"; +import { + type AutomaticMergeProgressState, + CONFLATION_AUTOMATIC_MERGE_STEPS, + EXACT_AUTOMATIC_MERGE_STEPS, + LiveAutomaticMergeProgress, +} from "../components/automatic-merge-progress"; +import { ConflationConfig } from "../components/conflation-config"; +import { ConflationReview } from "../components/conflation-review"; +import { ConflationRoutingDiagnostics } from "../components/conflation-routing-diagnostics"; import { Details, DetailsContent, DetailsSummary } from "../components/details"; import EntityDetails from "../components/entity-details"; import { FullIndexRequired, hasFullNodeIndex } from "../components/full-index-required"; +import { MergeStepGuide, type MergeStepGuideId } from "../components/merge-step-guide"; import ChangesSummary, { ChangesExpandableList, ChangesFilters, ChangesPagination, } from "../components/osm-changes-summary"; import OsmInfoTable from "../components/osm-info-table"; +import { OsmInputCardHeader } from "../components/osm-input-card-header"; import { LoadingState } from "../components/section"; +import { StepActions } from "../components/step-actions"; import StoredOsmList from "../components/stored-osm-list"; -import TaskProgress from "../components/task-progress"; import { Button } from "../components/ui/button"; -import { ButtonGroup, ButtonGroupSeparator } from "../components/ui/button-group"; import { Card, CardAction, CardContent, CardHeader, CardTitle } from "../components/ui/card"; import { Item, @@ -44,10 +57,31 @@ import { } from "../components/ui/item"; import { useFlyToEntity, useFlyToOsmBounds } from "../hooks/map"; import { useOsmFile } from "../hooks/osm"; +import { toOsmConflationOptions, validateConflationForm } from "../lib/conflation-workflow"; +import { + completeMergeOptions, + finalizeVerifiedMerge, + INTERSECTION_OPTIONS, + recoverConflationRunAllFailure, + runConflationAllSteps, + verifiedBaseMergeOptions, + WITHIN_DATASET_DIAGNOSTIC_OPTIONS, +} from "../lib/merge-workflow"; import { showSaveFilePickerWithFallback } from "../lib/save-file-picker"; import { cn } from "../lib/utils"; import { BASE_OSM_KEY, PATCH_OSM_KEY } from "../settings"; import { changesetStatsAtom } from "../state/changes"; +import { + conflationCandidateFilterAtom, + conflationCandidatePageAtom, + conflationCandidatePageIndexAtom, + conflationComparisonAtom, + conflationDecisionsAtom, + conflationFormAtom, + conflationRoutingDiagnosticsAtom, + conflationSummaryAtom, + resetConflationReviewAtom, +} from "../state/conflation"; import { Log } from "../state/log"; import { selectedEntityAtom, selectOsmEntityAtom } from "../state/osm"; import { mergeAbortControllerAtom, osmLoadingAbortControllerAtom } from "../state/status"; @@ -61,6 +95,7 @@ const STEPS = [ "review-changeset", "direct-merge", "review-changeset", + "match-imported-data", "deduplicate-nodes", "review-changeset", "create-intersections", @@ -70,6 +105,19 @@ const STEPS = [ ] as const; const stepIndexAtom = atom(0); + +type ChangesetReviewContext = + | { kind: "base-diagnostic" } + | { kind: "patch-diagnostic" } + | { kind: "direct-preview" } + | { kind: "cumulative"; exactReconciliation: boolean } + | { kind: "intersections" }; + +const changesetReviewContextAtom = atom({ + kind: "cumulative", + exactReconciliation: true, +}); +const CONFLATION_PAGE_SIZE = 10; const stepAtom = atom<(typeof STEPS)[number] | null>((get) => { const stepIndex = get(stepIndexAtom); return STEPS[stepIndex]; @@ -94,36 +142,229 @@ const makeMergedDownloadName = (baseName?: string | null, patchName?: string | n return `${combined.slice(0, 120)}.pbf`; }; +function reviewGuideId(context: ChangesetReviewContext): MergeStepGuideId { + switch (context.kind) { + case "base-diagnostic": + return "review-base-diagnostic"; + case "patch-diagnostic": + return "review-patch-diagnostic"; + case "direct-preview": + return "review-direct"; + case "cumulative": + return context.exactReconciliation + ? "review-cumulative-exact" + : "review-cumulative-without-exact"; + case "intersections": + return "review-intersections"; + } +} + +function reviewStepTitle(context: ChangesetReviewContext): string { + switch (context.kind) { + case "base-diagnostic": + return "Review base diagnostic"; + case "patch-diagnostic": + return "Review patch diagnostic"; + case "direct-preview": + return "Review direct merge"; + case "cumulative": + return context.exactReconciliation + ? "Review cumulative merge" + : "Review merge without exact reconciliation"; + case "intersections": + return "Review intersections"; + } +} + +function reviewChangesetTitle(context: ChangesetReviewContext): string { + switch (context.kind) { + case "base-diagnostic": + return "Base diagnostic candidates"; + case "patch-diagnostic": + return "Patch diagnostic candidates"; + case "direct-preview": + return "Direct-merge preview"; + case "cumulative": + return context.exactReconciliation + ? "Cumulative merge changeset" + : "Merge changeset without exact reconciliation"; + case "intersections": + return "Intersection changeset"; + } +} + export default function MergeBlock() { const base = useOsmFile(BASE_OSM_KEY); const patch = useOsmFile(PATCH_OSM_KEY); const [changesetStats, setChangesetStats] = useAtom(changesetStatsAtom); + const [changesetReviewContext, setChangesetReviewContext] = useAtom(changesetReviewContextAtom); + const [conflationForm] = useAtom(conflationFormAtom); + const [conflationSummary, setConflationSummary] = useAtom(conflationSummaryAtom); + const [conflationCandidatePage, setConflationCandidatePage] = useAtom( + conflationCandidatePageAtom, + ); + const [conflationCandidatePageIndex, setConflationCandidatePageIndex] = useAtom( + conflationCandidatePageIndexAtom, + ); + const [conflationCandidateFilter, setConflationCandidateFilter] = useAtom( + conflationCandidateFilterAtom, + ); + const [isConflationFilterPending, setIsConflationFilterPending] = useState(false); + const [automaticMergeProgress, setAutomaticMergeProgress] = + useState(null); + const setConflationDecisions = useSetAtom(conflationDecisionsAtom); + const [conflationRoutingDiagnostics, setConflationRoutingDiagnostics] = useAtom( + conflationRoutingDiagnosticsAtom, + ); + const resetConflationReview = useSetAtom(resetConflationReviewAtom); + const setConflationComparison = useSetAtom(conflationComparisonAtom); const flyToEntity = useFlyToEntity(); const flyToOsmBounds = useFlyToOsmBounds(); const selectedEntity = useAtomValue(selectedEntityAtom); const selectEntity = useSetAtom(selectOsmEntityAtom); - const setStepIndex = useSetAtom(stepIndexAtom); + const [stepIndex, setStepIndex] = useAtom(stepIndexAtom); const [mergeAbortController, setMergeAbortController] = useAtom(mergeAbortControllerAtom); const setLoadingState = useSetAtom(osmLoadingAbortControllerAtom); - const prevStep = () => { + const moveStep = (direction: -1 | 1) => { selectEntity(null, null); - setStepIndex((s) => s - 1); + setConflationComparison({ type: "FeatureCollection", features: [] }); + setStepIndex((current) => { + let next = current + direction; + if (STEPS[next] === "match-imported-data" && !conflationForm.enabled) { + next += direction; + } + return next; + }); + }; + const prevStep = () => { + moveStep(-1); }; const nextStep = () => { - selectEntity(null, null); - setStepIndex((s) => s + 1); + moveStep(1); }; const goToStep = (step: number | (typeof STEPS)[number]) => { const stepIndex = typeof step === "number" ? step : STEPS.indexOf(step); selectEntity(null, null); + setConflationComparison({ type: "FeatureCollection", features: [] }); setStepIndex(stepIndex); }; + const showVerifiedMergeResult = () => + finalizeVerifiedMerge( + () => patch.setOsm(null), + () => goToStep("inspect-final-osm"), + ); + const completesVerifiedMerge = STEPS[stepIndex - 1] === "create-intersections"; const startStepTask = async (message: string, fn: () => Promise) => { const task = Log.startTask(message); - const endMessage = await fn(); - task.end(endMessage); - nextStep(); + try { + const endMessage = await fn(); + task.end(endMessage); + nextStep(); + } catch (error) { + task.end(`Task failed: ${error instanceof Error ? error.message : "Unknown error"}`, "error"); + } + }; + const conflationValidationMessage = validateConflationForm(conflationForm); + const conflationOptions = conflationValidationMessage + ? undefined + : toOsmConflationOptions(conflationForm); + const baseFileName = base.file?.name ?? base.fileInfo?.fileName; + const patchFileName = patch.file?.name ?? patch.fileInfo?.fileName; + + const resetMergeDerivedState = () => { + setChangesetStats(null); + resetConflationReview(); + selectEntity(null, null); + }; + + const clearBaseOsm = async () => { + resetMergeDerivedState(); + await base.loadOsmFile(null); + }; + + const clearPatchOsm = async () => { + resetMergeDerivedState(); + await patch.loadOsmFile(null); + }; + + const loadConflationPage = async (page: number) => { + if (!base.osm) throw Error("Base OSM is not loaded"); + const result = await osmWorker.getConflationPage(base.osm.id, page, CONFLATION_PAGE_SIZE); + setConflationCandidatePageIndex(page); + setConflationCandidatePage(result); + }; + + const updateConflationFilter = async (filter: typeof conflationCandidateFilter) => { + if (!base.osm) throw Error("Base OSM is not loaded"); + const previousFilter = conflationCandidateFilter; + setConflationCandidateFilter(filter); + setIsConflationFilterPending(true); + try { + await osmWorker.setConflationFilter(base.osm.id, filter); + await loadConflationPage(0); + } catch (error) { + // Keep the visible controls aligned with the still-displayed page when a + // worker failure prevents the requested filter from being applied. + try { + await osmWorker.setConflationFilter(base.osm.id, previousFilter); + } catch { + // Preserve the original refresh error; a later page request will surface + // any worker recovery failure through the ordinary error channel. + } + setConflationCandidateFilter(previousFilter); + throw error; + } finally { + setIsConflationFilterPending(false); + } + }; + + const updateConflationDecision = async (decision: OsmConflationDecision) => { + if (!base.osm) throw Error("Base OSM is not loaded"); + const summary = await osmWorker.setConflationDecision(base.osm.id, decision); + setConflationDecisions((current) => [ + ...current.filter((existing) => existing.candidateId !== decision.candidateId), + decision, + ]); + setConflationSummary(summary); + await loadConflationPage(conflationCandidatePageIndex); + }; + + const updateConflationBulkDecision = async (request: OsmConflationBulkDecisionRequest) => { + if (!base.osm) throw Error("Base OSM is not loaded"); + const result = await osmWorker.applyConflationBulkDecision(base.osm.id, request); + setConflationDecisions(result.decisions); + setConflationSummary(result.summary); + await loadConflationPage(0); + Log.addMessage( + `Updated ${result.preview.changedCandidates.toLocaleString()} filtered conflation decisions`, + ); + }; + + const generateVerifiedChangeset = async (reconcile: boolean) => { + if (!base.osm || !patch.osm) throw Error("Missing data to generate changes"); + setChangesetReviewContext({ kind: "cumulative", exactReconciliation: reconcile }); + if (conflationOptions) { + if (!conflationSummary) { + throw Error("Discover and review imported-data match candidates first"); + } + const result = await osmWorker.generateConflationChangeset( + base.osm.id, + verifiedBaseMergeOptions(reconcile), + ); + setChangesetStats(result.stats); + setConflationRoutingDiagnostics(result.routing); + return changeStatsSummary(result.stats); + } + + setConflationRoutingDiagnostics(null); + const result = await osmWorker.generateChangeset( + base.osm.id, + patch.osm.id, + verifiedBaseMergeOptions(reconcile), + ); + setChangesetStats(result); + return changeStatsSummary(result); }; const downloadJsonChanges = async () => { @@ -155,15 +396,18 @@ export default function MergeBlock() { const applyChanges = async () => { if (!changesetStats) throw Error("Changeset stats are not loaded"); await osmWorker.applyChangesAndReplace(changesetStats.osmId); - const osm = await osmWorker.get(changesetStats.osmId); setChangesetStats(null); - return osm; + return changesetStats.osmId; }; const hasZeroChanges = useMemo(() => { if (!changesetStats) return true; return changesetStats.totalChanges === 0; }, [changesetStats]); + const isDiagnosticReview = + changesetReviewContext.kind === "base-diagnostic" || + changesetReviewContext.kind === "patch-diagnostic"; + const isDirectPreviewReview = changesetReviewContext.kind === "direct-preview"; const baseNeedsFull = base.osmInfo !== null && !hasFullNodeIndex(base.osmInfo); const patchNeedsFull = patch.osmInfo !== null && !hasFullNodeIndex(patch.osmInfo); @@ -178,50 +422,99 @@ export default function MergeBlock() { return (
- + - Merge steps + Merge pipeline
    -
  1. Deduplicate nodes and ways in base OSM
    2. -
  2. Deduplicate nodes and ways in patch OSM
    3. -
  3. Merge patch OSM onto base OSM.
    4. -
  4. Deduplicate nodes and ways in newly merged OSM
    5. -
  5. Create new intersections in merged data where ways cross
    6. +
  6. Optionally inspect each input for possible internal duplicates
    7. +
  7. Add patch entities and apply same-ID patch updates
    8. +
  8. Optionally match nearby imported entities
    9. +
  9. Optionally reconcile exact, compatible entities across the inputs
    10. +
  10. Create safe intersections where eligible ways cross
    11. +
  11. Validate topology before exposing the merged result

- Note: entities from the patch file are prioritized over matching entities in the base - file. + The reviewed workflow pauses at diagnostic and changeset checkpoints. The automatic + workflow skips those checkpoints but uses the same safety validation.

- - Select patch OSM to merge - {patch.osm && ( - - - {!patch.isStored && patch.canStore && ( - } - title="Save to storage" - variant="ghost" - onAction={patch.saveToStorage} - /> - )} - } - title="Clear patch OSM file" - variant="ghost" - onAction={async () => { - await patch.loadOsmFile(null); - }} - /> - - + + + {!base.osm ? ( + { + const abortController = new AbortController(); + setLoadingState({ + controller: abortController, + osmKey: BASE_OSM_KEY, + }); + setChangesetStats(null); + resetConflationReview(); + selectEntity(null, null); + try { + const osmInfo = await base.loadOsmPbfUrl(url, abortController.signal); + if (osmInfo) flyToOsmBounds(osmInfo); + return osmInfo; + } finally { + setLoadingState(null); + } + }} + openOsmFile={async (file, fileType) => { + const abortController = new AbortController(); + setLoadingState({ + controller: abortController, + osmKey: BASE_OSM_KEY, + }); + setChangesetStats(null); + resetConflationReview(); + selectEntity(null, null); + try { + const osmInfo = + typeof file === "string" + ? await base.loadFromStorage(file, abortController.signal) + : await base.loadOsmFile(file, fileType, abortController.signal); + if (osmInfo) flyToOsmBounds(osmInfo); + return osmInfo; + } finally { + setLoadingState(null); + } + }} + /> + ) : ( + )} - + + + + + {!patch.osm ? ( + +
{ + setChangesetStats(null); + resetConflationReview(); nextStep(); }} /> @@ -292,8 +592,10 @@ export default function MergeBlock() { - Option 1: Verify each step - Verify changes before applying them. + Review each merge stage + + Inspect diagnostics and approve each changeset before the in-memory base changes. + @@ -303,44 +605,152 @@ export default function MergeBlock() { render={
- -

Monitor the activity log below for progress. This may take a few minutes to complete.

- + +

The active step may take a few minutes. Detailed worker messages remain in the log.

+ {automaticMergeProgress ? : null} {mergeAbortController && ( )} - -

- Each file is first scanned for duplicate entities inside the same dataset. We then look - for duplicates that appear in both files. -

-

- Duplicates are features that share an ID or occupy the same geometry. We prefer entities - with newer version metadata; if that information is missing we keep the feature with more - tags. -

-

- When a duplicate is detected we draft a changeset entry that removes the extra copy. - Review those proposals in the next step before applying them. -

+ Base OSM PBF @@ -413,31 +836,41 @@ export default function MergeBlock() { /> - } - onAction={() => - startStepTask("Inspecting base OSM for duplicate entities", async () => { - if (!base.osm) throw Error("Base OSM is not loaded"); - const changes = await osmWorker.generateChangeset(base.osm.id, base.osm.id, { - deduplicateNodes: true, - deduplicateWays: true, - }); - setChangesetStats(changes); - return changeStatsSummary(changes); - }) - } - > - Deduplicate base OSM - + + } + onAction={async () => { + setChangesetStats(null); + Log.addMessage("Skipped base duplicate diagnostic"); + goToStep("inspect-patch-osm"); + }} + variant="outline" + > + Skip base diagnostic + + } + onAction={() => + startStepTask("Inspecting base OSM for duplicate entities", async () => { + if (!base.osm) throw Error("Base OSM is not loaded"); + setChangesetReviewContext({ kind: "base-diagnostic" }); + const changes = await osmWorker.generateChangeset( + base.osm.id, + base.osm.id, + WITHIN_DATASET_DIAGNOSTIC_OPTIONS, + ); + setChangesetStats(changes); + return changeStatsSummary(changes); + }) + } + > + Scan base for duplicate candidates + + - -

- Generate a changeset that removes duplicate entities from the patch file before it is - merged into the base data. -

- + Patch OSM PBF @@ -449,31 +882,41 @@ export default function MergeBlock() { /> - } - onAction={() => - startStepTask("Inspecting patch OSM for duplicate entities", async () => { - if (!patch.osm) throw Error("Patch OSM is not loaded"); - const patchChanges = await osmWorker.generateChangeset(patch.osm.id, patch.osm.id, { - deduplicateNodes: true, - deduplicateWays: true, - }); - setChangesetStats(patchChanges); - return changeStatsSummary(patchChanges); - }) - } - > - Deduplicate patch OSM - + + } + onAction={async () => { + setChangesetStats(null); + Log.addMessage("Skipped patch duplicate diagnostic"); + goToStep("direct-merge"); + }} + variant="outline" + > + Skip patch diagnostic + + } + onAction={() => + startStepTask("Inspecting patch OSM for duplicate entities", async () => { + if (!patch.osm) throw Error("Patch OSM is not loaded"); + setChangesetReviewContext({ kind: "patch-diagnostic" }); + const patchChanges = await osmWorker.generateChangeset( + patch.osm.id, + patch.osm.id, + WITHIN_DATASET_DIAGNOSTIC_OPTIONS, + ); + setChangesetStats(patchChanges); + return changeStatsSummary(patchChanges); + }) + } + > + Scan patch for duplicate candidates + + - -

- Add the patch entities to the base dataset and replace any base features that share the - same IDs. -

- + Base OSM PBF @@ -515,54 +958,43 @@ export default function MergeBlock() { - - prevStep()} icon={}> + + } onAction={async () => prevStep()} variant="outline"> Back - } onAction={() => - startStepTask("Generating changeset", async () => { + startStepTask("Generating direct-merge preview", async () => { if (!base.osm || !patch.osm) throw Error("Missing data to generate changes"); - const results = await osmWorker.generateChangeset(base.osm.id, patch.osm.id, { - directMerge: true, - deduplicateNodes: false, - createIntersections: false, - }); + setChangesetReviewContext({ kind: "direct-preview" }); + setConflationRoutingDiagnostics(null); + const results = await osmWorker.generateChangeset( + base.osm.id, + patch.osm.id, + verifiedBaseMergeOptions(false), + ); setChangesetStats(results); return changeStatsSummary(results); }) } > - Generate direct changes + Preview direct merge - + - -

- Review the proposed edits produced in the previous step. Apply the changes to update the - base OSM before moving forward. -

- - } onAction={downloadJsonChanges}> - Download JSON changes - - - } - onAction={async () => {}} - > - Download .osc changes - - + + } onAction={downloadJsonChanges}> + Download JSON changes + {changesetStats && base.osm && ( - Changeset + {reviewChangesetTitle(changesetReviewContext)} }> @@ -578,40 +1010,141 @@ export default function MergeBlock() { )} + {conflationRoutingDiagnostics ? ( + + ) : null} + + + {isDiagnosticReview ? ( + { + setChangesetStats(null); + nextStep(); + }} + icon={} + > + Continue without applying + + ) : isDirectPreviewReview ? ( + { + setChangesetStats(null); + nextStep(); + }} + icon={} + > + Continue to matching and reconciliation + + ) : changesetStats == null || hasZeroChanges ? ( + { + if (completesVerifiedMerge) showVerifiedMergeResult(); + else nextStep(); + }} + icon={} + > + {changesetReviewContext.kind === "intersections" + ? "No intersections, finish merge" + : "No changes, go to next step"} + + ) : ( + } + onAction={() => + startStepTask("Applying changes to OSM", async () => { + if (!changesetStats) throw Error("Changes are not loaded"); + const changedOsmId = await applyChanges(); + if (changesetStats.osmId === base.osm?.id) { + const mergedName = makeMergedDownloadName( + base.fileInfo?.fileName, + patch.fileInfo?.fileName, + ); + await base.setMergedOsm(changedOsmId, mergedName); + } else if (changesetStats.osmId === patch.osm?.id) { + await patch.setMergedOsm(changedOsmId); + } else { + throw Error("Changeset OSM ID does not match base or patch OSM ID"); + } + if (completesVerifiedMerge) patch.setOsm(null); + return "Changes applied"; + }) + } + > + {changesetReviewContext.kind === "intersections" + ? "Apply intersections and finish" + : "Apply cumulative merge"} + + )} + + + + + } + onAction={async () => { + if (!base.osm || !patch.osm || !conflationOptions) { + throw Error("Valid proximity-matching options and both inputs are required"); + } + const task = Log.startTask("Discovering imported-data match candidates"); + try { + resetConflationReview(); + const summary = await osmWorker.discoverConflation( + base.osm.id, + patch.osm.id, + conflationOptions, + ); + setConflationSummary(summary); + const page = await osmWorker.getConflationPage(base.osm.id, 0, CONFLATION_PAGE_SIZE); + setConflationCandidatePage(page); + task.end(`Found ${summary.total.toLocaleString()} imported-data match candidates`); + } catch (error) { + task.end( + `Candidate discovery failed: ${error instanceof Error ? error.message : "Unknown error"}`, + "error", + ); + throw error; + } + }} + > + {conflationSummary ? "Run candidate discovery again" : "Discover match candidates"} + + + {conflationSummary && conflationCandidatePage && base.osm && patch.osm ? ( + + ) : null} - {changesetStats == null || hasZeroChanges ? ( - nextStep()} icon={}> - No changes, go to next step + + } + onAction={async () => prevStep()} + variant="outline" + > + Back - ) : ( } - onAction={() => - startStepTask("Applying changes to OSM", async () => { - if (!changesetStats) throw Error("Changes are not loaded"); - const newOsm = await applyChanges(); - if (changesetStats.osmId === base.osm?.id) { - base.setOsm(newOsm); - } else if (changesetStats.osmId === patch.osm?.id) { - patch.setOsm(newOsm); - } else { - throw Error("Changeset OSM ID does not match base or patch OSM ID"); - } - return "Changes applied"; - }) - } + disabled={!conflationSummary || isConflationFilterPending} + icon={} + onAction={async () => nextStep()} > - Apply all changes + Continue with current decisions - )} + - -

- Identify nodes that occupy the same location in both datasets and merge them, updating any - way or relation references that point to those nodes. -

- + Current OSM PBF @@ -631,96 +1164,70 @@ export default function MergeBlock() { - + } - onAction={async () => goToStep("inspect-final-osm")} + onAction={() => + startStepTask( + "Generating cumulative preview without exact reconciliation", + async () => { + return generateVerifiedChangeset(false); + }, + ) + } + variant="outline" > - Skip + Preview without exact reconciliation - } onAction={() => - startStepTask("De-duplicating nodes and ways", async () => { - if (!base.osm || !patch.osm) throw Error("Missing data to generate changes"); - const results = await osmWorker.generateChangeset(base.osm.id, patch.osm.id, { - deduplicateNodes: true, - deduplicateWays: true, - }); - setChangesetStats(results); - return changeStatsSummary(results); + startStepTask("Generating cumulative preview with exact reconciliation", async () => { + return generateVerifiedChangeset(true); }) } > - De-duplicate nodes + Preview with exact reconciliation - + - -
-

- Scan new ways for crossings with existing ways and flag the segments that should share - intersection nodes based on their tags. -

-

- We quickly search for nearby ways and keep only those whose tags allow an intersection: - both must be linear, share the same `layer` value if present, include a `highway` tag, - and avoid bridge or tunnel tags. -

-

- For each candidate we locate the precise crossover point. Existing nodes at that point - are reused, favoring nodes introduced by the patch; otherwise we create a new node. -

-

- Finally, we update the way geometries so they reference the chosen intersection node. - You can review and apply those edits in the next screen. -

-
- - + + } - onAction={async () => goToStep("inspect-final-osm")} + onAction={async () => showVerifiedMergeResult()} + variant="outline" > - Skip + Skip intersections and finish - } onAction={() => - startStepTask("Generating changeset", async () => { + startStepTask("Generating intersection preview", async () => { if (!base.osm || !patch.osm) throw Error("Missing data to generate changes"); - const results = await osmWorker.generateChangeset(base.osm.id, patch.osm.id, { - directMerge: false, - deduplicateNodes: false, - createIntersections: true, - }); + setChangesetReviewContext({ kind: "intersections" }); + setConflationRoutingDiagnostics(null); + const results = await osmWorker.generateChangeset( + base.osm.id, + patch.osm.id, + INTERSECTION_OPTIONS, + ); setChangesetStats(results); return changeStatsSummary(results); }) } > - Create intersections + Preview intersection changes - + - -

- Review the merged OSM dataset, explore the results on the map, and download the new PBF - when ready. Zoom in to inspect individual entities and confirm the applied changes. -

- + {base.osm && ( <> - New OSM PBF + Merged OSM — in-memory result + {conflationRoutingDiagnostics ? ( + + ) : null} + {selectedEntity && ( @@ -755,16 +1266,16 @@ export default function MergeBlock() { )} -
- } onAction={() => base.downloadOsm()}> - Download merged OSM PBF - + {!base.isStored && base.canStore && ( - } onAction={base.saveToStorage}> + } onAction={base.saveToStorage} variant="outline"> Save to storage )} -
+ } onAction={() => base.downloadOsm()}> + Download merged OSM PBF + + )} @@ -775,24 +1286,34 @@ export default function MergeBlock() { function Step({ step, title, + guideId, isTransitioning, children, }: { step: (typeof STEPS)[number]; title: string; + guideId: MergeStepGuideId; isTransitioning?: boolean; children: React.ReactNode; }) { const currentStep = useAtomValue(stepAtom); const stepIndex = useAtomValue(stepIndexAtom); + const conflationEnabled = useAtomValue(conflationFormAtom).enabled; + const hiddenConflationStepBeforeCurrent = + !conflationEnabled && STEPS.slice(0, stepIndex + 1).includes("match-imported-data") ? 1 : 0; if (step !== currentStep) return null; if (isTransitioning === true) return Please wait...; return ( <> - - {stepIndex + 1}: {title} + + {step === "run-all-steps" + ? `Automatic workflow: ${title}` + : `${stepIndex + 1 - hiddenConflationStepBeforeCurrent}: ${title}`} + + + {children} diff --git a/apps/merge/src/components/automatic-merge-progress.tsx b/apps/merge/src/components/automatic-merge-progress.tsx new file mode 100644 index 00000000..5d584f13 --- /dev/null +++ b/apps/merge/src/components/automatic-merge-progress.tsx @@ -0,0 +1,161 @@ +import { CheckIcon, CircleIcon } from "lucide-react"; +import { useEffect, useState } from "react"; + +import { useLog } from "../hooks/log"; +import { cn } from "../lib/utils"; +import { Spinner } from "./ui/spinner"; + +export interface AutomaticMergeStep { + id: AutomaticMergeStepId; + label: string; +} + +export type AutomaticMergeStepId = + | "apply-verified-merge" + | "create-intersections" + | "discover-imported-data" + | "generate-verified-merge" + | "merge-exact" + | "refresh-result"; + +export const EXACT_AUTOMATIC_MERGE_STEPS = [ + { + id: "merge-exact", + label: "Merge, reconcile, and create intersections", + }, + { + id: "refresh-result", + label: "Refresh merged dataset", + }, +] as const satisfies readonly AutomaticMergeStep[]; + +export const CONFLATION_AUTOMATIC_MERGE_STEPS = [ + { + id: "discover-imported-data", + label: "Discover imported-data matches", + }, + { + id: "generate-verified-merge", + label: "Generate and validate merge changes", + }, + { + id: "apply-verified-merge", + label: "Apply verified merge changes", + }, + { + id: "create-intersections", + label: "Create and apply safe intersections", + }, + { + id: "refresh-result", + label: "Refresh merged dataset", + }, +] as const satisfies readonly AutomaticMergeStep[]; + +export interface AutomaticMergeProgressState { + currentStepId: AutomaticMergeStepId; + steps: readonly AutomaticMergeStep[]; +} + +function formatElapsed(ms: number) { + const totalSeconds = Math.floor(ms / 1_000); + const minutes = Math.floor(totalSeconds / 60); + const seconds = totalSeconds % 60; + return `${minutes}:${String(seconds).padStart(2, "0")}`; +} + +export function AutomaticMergeProgress({ + currentStepId, + elapsedMs = 0, + latestMessage, + steps, +}: AutomaticMergeProgressState & { + elapsedMs?: number; + latestMessage?: string; +}) { + const currentIndex = steps.findIndex((step) => step.id === currentStepId); + if (currentIndex === -1) { + throw Error(`Unknown automatic merge step: ${currentStepId}`); + } + + const completedCount = currentIndex; + const currentStep = steps[currentIndex]; + + return ( +
+
+ Merge progress + + {formatElapsed(Math.max(0, elapsedMs))} + +
+

+ {currentStep.label} is running. {completedCount} of {steps.length} steps completed. +

+
    + {steps.map((step, index) => { + const status = + index < currentIndex ? "completed" : index === currentIndex ? "running" : "remaining"; + + return ( +
  1. + + {status === "completed" ? ( + + + {step.label} + + + {status === "completed" + ? "Completed" + : status === "running" + ? "Running" + : "Remaining"} + + {status === "running" && latestMessage ? ( +

    + {latestMessage} +

    + ) : null} +
    2. + ); + })} +
+
+ ); +} + +export function LiveAutomaticMergeProgress(props: AutomaticMergeProgressState) { + const { activeTasks, log, taskStartedAt } = useLog(); + const [now, setNow] = useState(() => Date.now()); + + useEffect(() => { + if (taskStartedAt == null) return; + setNow(Date.now()); + const interval = setInterval(() => setNow(Date.now()), 1_000); + return () => clearInterval(interval); + }, [taskStartedAt]); + + const latestMessage = activeTasks > 0 ? log[log.length - 1]?.message : undefined; + const elapsedMs = taskStartedAt == null ? 0 : Math.max(0, now - taskStartedAt); + + return ; +} diff --git a/apps/merge/src/components/conflation-comparison-layer.tsx b/apps/merge/src/components/conflation-comparison-layer.tsx new file mode 100644 index 00000000..2904fcb4 --- /dev/null +++ b/apps/merge/src/components/conflation-comparison-layer.tsx @@ -0,0 +1,32 @@ +import { useAtomValue } from "jotai"; +import type { CircleLayerSpecification, LineLayerSpecification } from "maplibre-gl"; +import { Layer, Source } from "react-map-gl/maplibre"; + +import { APPID } from "../settings"; +import { conflationComparisonAtom } from "../state/conflation"; + +const SOURCE_ID = `${APPID}:conflation-comparison`; + +const linePaint: LineLayerSpecification["paint"] = { + "line-color": ["case", ["==", ["get", "role"], "source"], "#e11d48", "#0284c7"], + "line-width": ["interpolate", ["linear"], ["zoom"], 12, 1, 14, 3, 18, 8], + "line-opacity": 0.9, +}; + +const circlePaint: CircleLayerSpecification["paint"] = { + "circle-color": ["case", ["==", ["get", "role"], "source"], "#e11d48", "#0284c7"], + "circle-radius": ["interpolate", ["linear"], ["zoom"], 12, 2, 14, 5, 18, 8], + "circle-stroke-color": "white", + "circle-stroke-width": 2, +}; + +export function ConflationComparisonLayer() { + const comparison = useAtomValue(conflationComparisonAtom); + + return ( + + + + + ); +} diff --git a/apps/merge/src/components/conflation-config.tsx b/apps/merge/src/components/conflation-config.tsx new file mode 100644 index 00000000..1fb246e4 --- /dev/null +++ b/apps/merge/src/components/conflation-config.tsx @@ -0,0 +1,146 @@ +import { useAtom, useSetAtom } from "jotai"; + +import { validateConflationForm } from "../lib/conflation-workflow"; +import { conflationFormAtom, resetConflationReviewAtom } from "../state/conflation"; +import { InfoTooltip } from "./info-tooltip"; +import { Card, CardContent, CardHeader } from "./ui/card"; +import { Checkbox, CheckboxLabel } from "./ui/checkbox"; +import { Input } from "./ui/input"; + +export function ConflationConfig() { + const [form, setForm] = useAtom(conflationFormAtom); + const resetReview = useSetAtom(resetConflationReviewAtom); + const validationMessage = validateConflationForm(form); + const updateForm = (update: (current: typeof form) => typeof form) => { + setForm(update); + resetReview(); + }; + + return ( + + Match imported data + +
+ + { + updateForm((current) => ({ ...current, enabled })); + }} + /> + Enable proximity matching + + + Opt in to match imported entities against nearby base OSM. Exact reconciliation remains + the default when this is disabled. + +
+ + {form.enabled ? ( +
+
+ + { + updateForm((current) => ({ ...current, transferProperties })); + }} + /> + Transfer selected properties + + + Copy only the selected OSM tags from an accepted imported match onto its base + entity. This does not move geometry, rewrite the imported network, or delete a base + tag when the imported value is absent. After tags transfer, an equivalent one-to-one + imported way may be suppressed. Cleanup removes only its newly imported, tagless + nodes that are no longer referenced by any way or relation. + +
+ +
+
+ + + Separate keys with commas or spaces. The defaults focus on crossing and kerb + accessibility data. Imported values replace base values only for these keys; + structural tags such as layer, bridge, tunnel, and area are protected, while + routing-affecting tags require review. + +
+ { + updateForm((current) => ({ + ...current, + propertyKeys: event.target.value, + })); + }} + /> +
+ +
+ + { + updateForm((current) => ({ ...current, attachNetwork })); + }} + /> + Attach compatible imported network nodes + + + Connect accepted imported ways by rewriting only patch-created way references to + preserved base nodes. Base coordinates, base way references, and relation membership + remain authoritative. + +
+ +
+
+ + + Nearby entities within this radius become candidates. Distance alone never + guarantees acceptance; geometry, routing context, grade separation, and ambiguity + checks still apply. + +
+ { + updateForm((current) => ({ + ...current, + maxDistanceMeters: event.target.valueAsNumber, + })); + }} + /> +
+ +
+ Automatic decisions + + High-confidence matches apply automatically. Ambiguous, routing-affecting, and + structurally uncertain candidates remain available for review. + +
+ + {validationMessage ? ( +

+ {validationMessage} +

+ ) : null} +
+ ) : null} + + + ); +} diff --git a/apps/merge/src/components/conflation-review.tsx b/apps/merge/src/components/conflation-review.tsx new file mode 100644 index 00000000..f203aa31 --- /dev/null +++ b/apps/merge/src/components/conflation-review.tsx @@ -0,0 +1,772 @@ +import { useSetAtom } from "jotai"; +import { LocateFixedIcon } from "lucide-react"; +import type { + Osm, + OsmConflationBulkAction, + OsmConflationBulkDecisionPreview, + OsmConflationBulkDecisionRequest, + OsmConflationCandidateFilter, + OsmConflationCandidateView, + OsmConflationDecision, + OsmConflationEffectiveStatus, + OsmConflationPage, + OsmConflationReasonCode, + OsmConflationSummary, +} from "osmix"; +import { osmEntityToGeoJSONFeature } from "osmix"; +import { useState } from "react"; + +import { useMap } from "../hooks/map"; +import { conflationBulkActionCopy } from "../lib/conflation-workflow"; +import { cn } from "../lib/utils"; +import { conflationComparisonAtom } from "../state/conflation"; +import ActionButton from "./action-button"; +import { Details, DetailsContent, DetailsSummary } from "./details"; +import { InfoTooltip } from "./info-tooltip"; +import { EmptyState } from "./section"; +import { StatusDot, type StatusDotStatus } from "./status-dot"; +import { Button } from "./ui/button"; +import { ButtonGroup, ButtonGroupSeparator } from "./ui/button-group"; +import { Card, CardAction, CardContent, CardHeader } from "./ui/card"; +import { + Dialog, + DialogContent, + DialogDescription, + DialogFooter, + DialogHeader, + DialogTitle, +} from "./ui/dialog"; +import { Item, ItemActions, ItemContent, ItemDescription, ItemGroup, ItemTitle } from "./ui/item"; +import { Spinner } from "./ui/spinner"; +import { Table, TableBody, TableCell, TableHead, TableHeader, TableRow } from "./ui/table"; + +const REASON_CODES = [ + "bearing-mismatch", + "drivable-network", + "exact-match", + "geometry-mismatch", + "grade-conflict", + "length-mismatch", + "many-to-one", + "multiple-targets", + "no-transferable-properties", + "node-context-conflict", + "non-routing-target", + "protected-tag", + "relation-member", + "routing-family-conflict", + "routing-property", + "same-id", + "unsupported-way-chain", + "would-collapse-way", +] as const satisfies readonly OsmConflationReasonCode[]; + +const STATUS_DOT: Record = { + accepted: "ok", + automatic: "ok", + blocked: "error", + rejected: "warn", + review: "warn", + unmatched: "error", +}; + +const STATUS_LABEL: Record = { + accepted: "Accepted", + automatic: "Automatic", + blocked: "Blocked", + rejected: "Rejected", + review: "Needs review", + unmatched: "Unmatched", +}; + +const STATUS_HELP: Record = { + accepted: "an explicit decision will apply the selected fuzzy action", + automatic: "at least one high-confidence action applies unless rejected", + blocked: "at least one action is prevented by a structural safety rule", + rejected: "fuzzy actions are disabled by an explicit decision", + review: "at least one action needs a decision; another action may already be automatic", + unmatched: "no compatible base target was found", +}; + +const REASON_LABEL: Record = { + "bearing-mismatch": "Direction does not align", + "drivable-network": "Drivable network requires review", + "exact-match": "Handled by exact reconciliation", + "geometry-mismatch": "Geometry differs", + "grade-conflict": "Grade separation conflicts", + "length-mismatch": "Lengths differ", + "many-to-one": "Multiple imported entities share one base target", + "multiple-targets": "Multiple possible base targets", + "no-transferable-properties": "No selected properties differ", + "node-context-conflict": "Connected-way context conflicts", + "non-routing-target": "Base target is not routable", + "protected-tag": "Protected structural property differs", + "relation-member": "Entity participates in a relation", + "routing-family-conflict": "Routing uses are incompatible", + "routing-property": "Routing property requires review", + "same-id": "Handled as a same-ID update", + "unsupported-way-chain": "One-to-many way matching is unsupported", + "would-collapse-way": "Attachment would collapse a way", +}; + +const ROUTING_FAMILY_LABEL = { + "bicycle-shared": "Bicycle or shared-use", + "motor-road": "Motor road", + "non-routable": "Non-routable", + pedestrian: "Pedestrian", +} as const; + +export function conflationStatusLabel(status: OsmConflationEffectiveStatus) { + return STATUS_LABEL[status]; +} + +export function conflationReasonLabel(reason: OsmConflationReasonCode) { + return REASON_LABEL[reason]; +} + +export function conflationCandidateTitle(candidate: OsmConflationCandidateView) { + const target = + candidate.targetId == null + ? "No compatible base target" + : `Base ${candidate.entityType} ${candidate.targetId}`; + return `Imported ${candidate.entityType} ${candidate.sourceId} → ${target}`; +} + +export interface ConflationReviewProps { + base: Osm; + patch: Osm; + summary: OsmConflationSummary; + page: OsmConflationPage; + filter: OsmConflationCandidateFilter; + isFilterPending: boolean; + onDecision: (decision: OsmConflationDecision) => Promise; + onBulkDecision: (request: OsmConflationBulkDecisionRequest) => Promise; + onFilterChange: (filter: OsmConflationCandidateFilter) => Promise; + onPageChange: (page: number) => Promise; +} + +function effectiveStatus(candidate: OsmConflationCandidateView) { + if (candidate.decision?.action === "accept") return "accepted" as const; + if (candidate.decision?.action === "reject") return "rejected" as const; + return candidate.status; +} + +function entityFeature( + osm: Osm, + candidate: OsmConflationCandidateView, + role: "source" | "target", +): GeoJSON.Feature | null { + const id = role === "source" ? candidate.sourceId : candidate.targetId; + if (id == null) return null; + const entity = candidate.entityType === "node" ? osm.nodes.getById(id) : osm.ways.getById(id); + if (!entity) return null; + const feature = osmEntityToGeoJSONFeature(osm, entity); + if (feature.type !== "Feature") return null; + return { + ...feature, + properties: { ...feature.properties, role }, + }; +} + +function entityBbox(osm: Osm, candidate: OsmConflationCandidateView, role: "source" | "target") { + const id = role === "source" ? candidate.sourceId : candidate.targetId; + if (id == null) return null; + if (candidate.entityType === "node") { + const node = osm.nodes.getById(id); + return node ? ([node.lon, node.lat, node.lon, node.lat] as const) : null; + } + return osm.ways.getEntityBbox({ id }); +} + +function SummaryTable({ summary }: { summary: OsmConflationSummary }) { + return ( + + + {( + ["total", "accepted", "automatic", "review", "blocked", "unmatched", "rejected"] as const + ).map((key) => ( + + {key === "total" ? "Total" : conflationStatusLabel(key)} + {summary[key].toLocaleString()} + + ))} + +
+ ); +} + +export function ConflationStatusLegend() { + return ( + +
+

+ Overall status summarizes the candidate. Property transfer and network attachment are + assessed independently. +

+ {(["automatic", "review", "blocked", "unmatched", "accepted", "rejected"] as const).map( + (status) => ( +

+ {conflationStatusLabel(status)}:{" "} + {STATUS_HELP[status]}. +

+ ), + )} +
+ + ); +} + +export function CandidateActionStatuses({ candidate }: { candidate: OsmConflationCandidateView }) { + return ( +
+ + Property transfer:{" "} + + {conflationStatusLabel(candidate.propertyTransfer.status)} + + + {candidate.networkAttachment ? ( + + Network attachment:{" "} + + {conflationStatusLabel(candidate.networkAttachment.status)} + + + ) : null} +
+ ); +} + +const BULK_ACTIONS = ["transfer-properties", "attach-network", "reject"] as const; + +function BulkPreviewTable({ preview }: { preview: OsmConflationBulkDecisionPreview }) { + const rows = [ + ["filtered matches", preview.filteredCandidates], + ["will change", preview.changedCandidates], + ["eligible", preview.eligibleCandidates], + ["automatic", preview.automaticCandidates], + ["review", preview.reviewCandidates], + ["blocked, ambiguous, or ineligible skipped", preview.skippedCandidates], + ["existing decisions replaced", preview.overriddenDecisions], + ] as const; + return ( + + + {rows.map(([label, count]) => ( + + {label} + {count.toLocaleString()} + + ))} + +
+ ); +} + +export function ConflationBulkActions({ + bulkActions, + disabled = false, + filter, + onBulkDecision, +}: { + bulkActions: OsmConflationPage["bulkActions"]; + disabled?: boolean; + filter: OsmConflationCandidateFilter; + onBulkDecision: (request: OsmConflationBulkDecisionRequest) => Promise; +}) { + const [selectedAction, setSelectedAction] = useState(null); + const selectedPreview = selectedAction ? bulkActions[selectedAction] : null; + const selectedCopy = selectedAction ? conflationBulkActionCopy(selectedAction) : null; + + return ( + <> +
+
+ Bulk decisions + + Bulk decisions apply to every match in the current filters across all pages. Automatic + matches already apply unless rejected. + +
+
+ {BULK_ACTIONS.map((action) => { + const preview = bulkActions[action]; + const copy = conflationBulkActionCopy(action); + return ( + + ); + })} +
+
+ + { + if (!open) setSelectedAction(null); + }} + > + {selectedAction && selectedPreview && selectedCopy ? ( + + + {selectedCopy.title} + + {selectedCopy.description} This applies across every filtered page and replaces + prior decisions shown below. + + + + + + { + await onBulkDecision({ action: selectedAction, filter: { ...filter } }); + setSelectedAction(null); + }} + > + {selectedCopy.confirmLabel} ({selectedPreview.changedCandidates.toLocaleString()}) + + + + ) : null} + + + ); +} + +export function CandidateEvidence({ candidate }: { candidate: OsmConflationCandidateView }) { + const { evidence } = candidate; + return ( +
+ Evidence and property diff + + + + + + + Evidence + + Distance finds nearby candidates. Routing families describe allowed network use; + bearing compares direction, length difference compares total geometry length, + and maximum geometry distance measures the worst sampled separation. + + + + Measured value + + + + + Candidate distance + {evidence.distanceMeters.toFixed(3)} m + + + Imported routing family + + {evidence.sourceRoutingFamilies + .map((family) => ROUTING_FAMILY_LABEL[family]) + .join(", ") || "None"} + + + + Base routing family + + {evidence.targetRoutingFamilies + .map((family) => ROUTING_FAMILY_LABEL[family]) + .join(", ") || "None"} + + + {evidence.bearingDifferenceDegrees !== undefined ? ( + + Bearing difference + {evidence.bearingDifferenceDegrees.toFixed(1)}° + + ) : null} + {evidence.lengthDifferenceRatio !== undefined ? ( + + Length difference + {(evidence.lengthDifferenceRatio * 100).toFixed(1)}% + + ) : null} + {evidence.maxGeometryDistanceMeters !== undefined ? ( + + Maximum geometry distance + {evidence.maxGeometryDistanceMeters.toFixed(3)} m + + ) : null} + +
+ + {evidence.tagDiff.length > 0 ? ( + + + + Property + Base value + Imported value + + + + {evidence.tagDiff.map((diff) => ( + + {diff.key} + {String(diff.baseValue ?? "not set")} + {String(diff.patchValue)} + + ))} + +
+ ) : ( + No selected property differences + )} + +
+ ); +} + +export function CandidateActions({ + candidate, + onDecision, +}: { + candidate: OsmConflationCandidateView; + onDecision: (decision: OsmConflationDecision) => Promise; +}) { + const canTransferProperties = + candidate.propertyTransfer.status !== "blocked" && + candidate.propertyTransfer.status !== "unmatched" && + candidate.evidence.tagDiff.length > 0; + const canAttachNetwork = + candidate.networkAttachment !== null && + candidate.networkAttachment.status !== "blocked" && + candidate.networkAttachment.status !== "unmatched"; + + return ( +
+ {canTransferProperties ? ( + + onDecision({ + candidateId: candidate.id, + action: "accept", + transferProperties: true, + attachNetwork: false, + }) + } + > + Transfer properties + + ) : null} + {canAttachNetwork ? ( + + onDecision({ + candidateId: candidate.id, + action: "accept", + transferProperties: false, + attachNetwork: true, + }) + } + > + Attach network + + ) : null} + {canTransferProperties && canAttachNetwork ? ( + + onDecision({ + candidateId: candidate.id, + action: "accept", + transferProperties: true, + attachNetwork: true, + }) + } + > + Transfer + attach + + ) : null} + onDecision({ candidateId: candidate.id, action: "reject" })} + > + Reject + +
+ ); +} + +export function ConflationResultsHeader({ + isFilterPending, + totalCandidates, +}: { + isFilterPending: boolean; + totalCandidates: number; +}) { + return ( + + Filtered matches ({totalCandidates.toLocaleString()} + {isFilterPending ? ", stale" : ""}) + {isFilterPending ? ( + + + Updating filters… + + ) : null} + + ); +} + +export function ConflationReview({ + base, + patch, + summary, + page, + filter, + isFilterPending, + onDecision, + onBulkDecision, + onFilterChange, + onPageChange, +}: ConflationReviewProps) { + const map = useMap(); + const setComparison = useSetAtom(conflationComparisonAtom); + const showCandidate = (candidate: OsmConflationCandidateView) => { + const sourceFeature = entityFeature(patch, candidate, "source"); + const targetFeature = entityFeature(base, candidate, "target"); + const features: GeoJSON.Feature[] = []; + if (sourceFeature) features.push(sourceFeature); + if (targetFeature) features.push(targetFeature); + setComparison({ + type: "FeatureCollection", + features, + }); + + const boxes = [ + entityBbox(patch, candidate, "source"), + entityBbox(base, candidate, "target"), + ].filter((bbox): bbox is readonly [number, number, number, number] => bbox !== null); + if (!map || boxes.length === 0) return; + const bounds = boxes.reduce( + (result, bbox) => [ + Math.min(result[0], bbox[0]), + Math.min(result[1], bbox[1]), + Math.max(result[2], bbox[2]), + Math.max(result[3], bbox[3]), + ], + [...boxes[0]], + ); + map.fitBounds( + [ + [bounds[0], bounds[1]], + [bounds[2], bounds[3]], + ], + { padding: 120, maxDuration: 200, maxZoom: 19 }, + ); + }; + + return ( +
+ + + Candidate summary + + + + + + + + + + + Candidate filters + + + + + + + + + + + + + + {page.candidates.length === 0 ? ( + No candidates match these filters + ) : ( + + {page.candidates.map((candidate) => { + const status = effectiveStatus(candidate); + return ( + + +
+ +
+ {conflationCandidateTitle(candidate)} + + {conflationStatusLabel(status)};{" "} + {candidate.evidence.distanceMeters.toFixed(3)} m + {candidate.reasons.length > 0 + ? `; ${candidate.reasons.map(conflationReasonLabel).join(", ")}` + : ""} + + +
+ + + +
+ + + + + ); + })} + + )} + + + + + + + + + + + +
+ + Map comparison + + The imported source is shown in destructive red and the proposed base target in + informational blue. + + + + Reject behavior + + Rejecting disables fuzzy property transfer and network attachment. It does not remove + the imported entity from the ordinary direct merge. + + +
+
+ ); +} diff --git a/apps/merge/src/components/conflation-routing-diagnostics.tsx b/apps/merge/src/components/conflation-routing-diagnostics.tsx new file mode 100644 index 00000000..52fd2e61 --- /dev/null +++ b/apps/merge/src/components/conflation-routing-diagnostics.tsx @@ -0,0 +1,86 @@ +import type { OsmConflationRoutingDiagnostics, OsmConflationRoutingGraphStats } from "osmix"; +import { useId } from "react"; + +import { Card, CardContent, CardHeader } from "./ui/card"; +import { Table, TableBody, TableCell, TableHead, TableHeader, TableRow } from "./ui/table"; + +const METRICS = [ + "nodes", + "routableNodes", + "edges", + "components", +] as const satisfies readonly (keyof OsmConflationRoutingGraphStats)[]; + +const METRIC_LABEL: Record<(typeof METRICS)[number], string> = { + components: "Connected components", + edges: "Directed edges", + nodes: "All graph nodes", + routableNodes: "Routable nodes", +}; + +function formatDelta(value: number) { + if (value > 0) return `+${value.toLocaleString()}`; + return value.toLocaleString(); +} + +export function ConflationRoutingDiagnostics({ + diagnostics, +}: { + diagnostics: OsmConflationRoutingDiagnostics; +}) { + const descriptionId = useId(); + return ( + + Routing topology impact + +
+

+ Before is the ordinary direct merge, + including exact reconciliation when selected.{" "} + After adds accepted fuzzy property + transfers and network attachments. +

+

+ All graph nodes include every node loaded into the mode-specific graph. Routable nodes + participate in at least one usable street; directed edges are traversable movements; + connected components are weakly connected groups calculated without edge direction. + Different components guarantee no route between them, but one component does not + guarantee travel in both directions. Delta is after minus before. +

+
+ + + + Mode / metric + Before ordinary merge + After fuzzy matching + Signed delta + + + + {(["car", "walk"] as const).flatMap((mode) => + METRICS.map((metric) => { + const value = diagnostics[mode]; + return ( + + + {mode.toUpperCase()} / {METRIC_LABEL[metric]} + + {value.before[metric].toLocaleString()} + {value.after[metric].toLocaleString()} + {formatDelta(value.delta[metric])} + + ); + }), + )} + +
+

+ A walk-only attachment should not change CAR topology. Fewer WALK components can indicate + the intended new connection, but topology counts alone do not prove that routing is + correct. +

+ + + ); +} diff --git a/apps/merge/src/components/details.tsx b/apps/merge/src/components/details.tsx index 8b446b14..94412d4b 100644 --- a/apps/merge/src/components/details.tsx +++ b/apps/merge/src/components/details.tsx @@ -1,5 +1,5 @@ import type { ClassValue } from "clsx"; -import { ChevronUp } from "lucide-react"; +import { ChevronDown } from "lucide-react"; import type { ReactNode } from "react"; import { cn } from "../lib/utils"; @@ -22,10 +22,6 @@ export function Details({ ); } -/** - * TODO: properly rotate the chevron when open. Right now, the state=open is applied to hte trigger, so we have to trickle it down to the icon somehow. - * ALSO: Only show the shadow on open - */ export function DetailsSummary({ className, children, @@ -36,12 +32,15 @@ export function DetailsSummary({ return ( {children} - + ); } diff --git a/apps/merge/src/components/info-tooltip.tsx b/apps/merge/src/components/info-tooltip.tsx new file mode 100644 index 00000000..a4180015 --- /dev/null +++ b/apps/merge/src/components/info-tooltip.tsx @@ -0,0 +1,55 @@ +import { Popover } from "@base-ui/react/popover"; +import { InfoIcon } from "lucide-react"; +import type { ReactNode } from "react"; + +import { cn } from "../lib/utils"; + +export function InfoTooltip({ + align = "center", + children, + className, + defaultOpen = false, + label, + side = "top", +}: { + align?: "center" | "end" | "start"; + children: ReactNode; + className?: string; + defaultOpen?: boolean; + label: string; + side?: "bottom" | "left" | "right" | "top"; +}) { + return ( + + + + + + + {children} + + + + + ); +} diff --git a/apps/merge/src/components/merge-guide-diagram.tsx b/apps/merge/src/components/merge-guide-diagram.tsx new file mode 100644 index 00000000..02644b28 --- /dev/null +++ b/apps/merge/src/components/merge-guide-diagram.tsx @@ -0,0 +1,398 @@ +import { useId, type ReactNode } from "react"; + +export const MERGE_GUIDE_DIAGRAM_IDS = [ + "pipeline", + "direct-merge", + "exact-reconciliation", + "fuzzy-conflation", + "intersections", +] as const; + +export type MergeGuideDiagramId = (typeof MERGE_GUIDE_DIAGRAM_IDS)[number]; + +interface DiagramFrameProps { + children: ReactNode; + description: string; + kind: MergeGuideDiagramId; + title: string; +} + +function DiagramFrame({ children, description, kind, title }: DiagramFrameProps) { + const titleId = useId(); + const descriptionId = useId(); + + return ( + + {title} + {description} + {children} + + ); +} + +type DiagramTone = "info" | "neutral" | "success" | "warning"; + +const boxToneClasses: Record = { + info: "fill-info/5 stroke-info", + neutral: "fill-card stroke-border", + success: "fill-success/10 stroke-success", + warning: "fill-warning/10 stroke-warning", +}; + +function DiagramBox({ + detail, + height = 54, + label, + tone = "neutral", + width, + x, + y, +}: { + detail?: string; + height?: number; + label: string; + tone?: DiagramTone; + width: number; + x: number; + y: number; +}) { + const center = x + width / 2; + return ( + + + + {label} + + {detail ? ( + + {detail} + + ) : null} + + ); +} + +function DiagramArrow({ + dashed = false, + endX, + endY, + startX, + startY, + tone = "info", +}: { + dashed?: boolean; + endX: number; + endY: number; + startX: number; + startY: number; + tone?: Exclude; +}) { + const strokeClass = + tone === "success" + ? "stroke-success fill-success" + : tone === "warning" + ? "stroke-warning fill-warning" + : "stroke-info fill-info"; + const angle = Math.atan2(endY - startY, endX - startX); + const arrowLength = 8; + const arrowSpread = 4; + const lineEndX = endX - Math.cos(angle) * arrowLength; + const lineEndY = endY - Math.sin(angle) * arrowLength; + const leftX = lineEndX + Math.cos(angle + Math.PI / 2) * arrowSpread; + const leftY = lineEndY + Math.sin(angle + Math.PI / 2) * arrowSpread; + const rightX = lineEndX + Math.cos(angle - Math.PI / 2) * arrowSpread; + const rightY = lineEndY + Math.sin(angle - Math.PI / 2) * arrowSpread; + + return ( + + + + + ); +} + +function PipelineDiagram() { + return ( + + + + + + + + + + + ); +} + +function DirectMergeDiagram() { + return ( + + + + + + + + Base-only IDs: retained + + + Patch-only IDs: added + + + Same IDs: patch wins + + + ); +} + +function ExactReconciliationDiagram() { + return ( + + + + + + + + + + ); +} + +function FuzzyConflationDiagram() { + return ( + + + + + + + + + + + ); +} + +function IntersectionsDiagram() { + return ( + + + Compatible grade + + + + + + Shared node: connected + + + + Grade separated + + + + + No shared node: disconnected + + + ); +} + +export function MergeGuideDiagram({ diagram }: { diagram: MergeGuideDiagramId }) { + switch (diagram) { + case "pipeline": + return ; + case "direct-merge": + return ; + case "exact-reconciliation": + return ; + case "fuzzy-conflation": + return ; + case "intersections": + return ; + } +} diff --git a/apps/merge/src/components/merge-step-guide.tsx b/apps/merge/src/components/merge-step-guide.tsx new file mode 100644 index 00000000..ee4d81e7 --- /dev/null +++ b/apps/merge/src/components/merge-step-guide.tsx @@ -0,0 +1,390 @@ +import { Details, DetailsContent, DetailsSummary } from "./details"; +import { MergeGuideDiagram, type MergeGuideDiagramId } from "./merge-guide-diagram"; +import { SectionTitle } from "./section"; + +export const MERGE_STEP_GUIDE_IDS = [ + "select", + "run-all", + "inspect-base", + "inspect-patch", + "direct", + "review-base-diagnostic", + "review-patch-diagnostic", + "review-direct", + "review-cumulative-exact", + "review-cumulative-without-exact", + "review-intersections", + "match-imported", + "reconcile", + "intersections", + "final", +] as const; + +export type MergeStepGuideId = (typeof MERGE_STEP_GUIDE_IDS)[number]; + +export interface MergeStepGuideDefinition { + /** The short explanation that remains visible while the detailed disclosure is closed. */ + summary: string; + /** Datasets, configuration, or generated changes inspected by this stage. */ + inputs: readonly string[]; + /** Mutations this stage may make now or after its changeset is accepted. */ + mutations: readonly string[]; + /** Safety guarantees and data that this stage must preserve. */ + invariants: readonly string[]; + /** The artifact or workflow state produced by the stage. */ + output: string; + /** A material caveat that should be visible within the detailed explanation. */ + warning?: string; + /** A diagram used only when it materially clarifies the transformation. */ + diagram?: MergeGuideDiagramId; +} + +export const MERGE_STEP_GUIDES = { + select: { + summary: + "Choose the authoritative base dataset and the imported patch, then choose a reviewed or automatic workflow.", + inputs: [ + "Base OSM: the existing dataset whose IDs and topology are authoritative except where a same-ID patch entity supplies an update.", + "Patch OSM: imported additions and updates that will be merged into the base.", + "Optional imported-data matching settings, including selected tag keys and search radius.", + ], + mutations: [ + "No OSM entities change here. The browser loads and indexes copies of both PBF files in memory.", + ], + invariants: [ + "The source files on disk are never overwritten.", + "Selecting the automatic workflow does not enable fuzzy matching unless it was explicitly configured.", + ], + output: + "Two indexed inputs and a workflow choice that determine which review checkpoints are shown.", + diagram: "pipeline", + }, + "run-all": { + summary: + "Run direct merge, exact reconciliation, optional automatic imported-data matches, and intersection creation without review checkpoints.", + inputs: [ + "The untouched base and patch loaded during input selection.", + "Imported-data settings only when fuzzy matching was explicitly enabled.", + ], + mutations: [ + "The verified cumulative changeset is applied to the in-memory base, followed by a separate intersection changeset.", + "Diagnostic scans and intermediate review screens are skipped.", + ], + invariants: [ + "Only high-confidence automatic fuzzy matches apply without review; unresolved candidates are not silently accepted.", + "The uploaded source files remain unchanged even after the in-memory base is replaced.", + ], + output: "A merged in-memory dataset ready for final inspection and PBF download.", + warning: + "Cancellation is best-effort. Osmix checks for it between supported stages, but a long worker operation may finish before the request is observed. Reload the untouched source inputs to undo a completed in-memory mutation.", + diagram: "pipeline", + }, + "inspect-base": { + summary: + "Scan the base for exact serialized-coordinate and ordered-reference duplicate candidates without changing the base.", + inputs: [ + "Only the authoritative base OSM and its spatial indexes.", + "Nodes at the same seven-decimal OSM coordinate and ways with identical ordered references and compatible routing semantics.", + ], + mutations: ["None. The scan creates a diagnostic changeset for display but never applies it."], + invariants: [ + "Nearby but non-identical entities are not reported as exact duplicates.", + "Base coordinates, references, tags, and relation members remain unchanged.", + ], + output: "A review-only list of possible exact duplicates inside the base dataset.", + }, + "inspect-patch": { + summary: + "Scan the patch for exact serialized-coordinate and ordered-reference duplicate candidates without normalizing the imported data.", + inputs: [ + "Only the imported patch OSM and its spatial indexes.", + "Nodes at the same seven-decimal OSM coordinate and ways with identical ordered references and compatible routing semantics.", + ], + mutations: ["None. The scan creates a diagnostic changeset for display but never applies it."], + invariants: [ + "The patch is passed untouched to later cross-dataset merge stages.", + "Nearby but non-identical entities belong in the separate, opt-in imported-data matching workflow.", + ], + output: "A review-only list of possible exact duplicates inside the patch dataset.", + }, + direct: { + summary: + "Preview patch additions and same-ID updates while retaining entities that exist only in the base.", + inputs: ["The original base and patch entities, compared by OSM entity type and ID."], + mutations: [ + "Patch-only entities are proposed as additions.", + "A patch entity with the same type and ID as a base entity is proposed as an authoritative update.", + "This screen only generates a preview; it does not apply those proposals.", + ], + invariants: [ + "Base-only entities remain in the result.", + "No proximity matching, topology attachment, or intersection creation occurs during direct merge.", + "Both uploaded inputs remain unchanged.", + ], + output: "A direct-merge changeset that can be inspected before later cumulative generation.", + diagram: "direct-merge", + }, + "review-base-diagnostic": { + summary: + "Inspect possible duplicates found inside the base; continuing discards this diagnostic changeset.", + inputs: ["The base-only scan results from the preceding diagnostic stage."], + mutations: ["None. Downloading or browsing the diagnostic records does not apply them."], + invariants: [ + "No base entity coordinates, references, tags, or relation members are changed.", + "A displayed candidate is evidence for human review, not an automatic merge instruction.", + ], + output: "A reviewed diagnostic record; the workflow continues with the unchanged base.", + }, + "review-patch-diagnostic": { + summary: + "Inspect possible duplicates found inside the patch; continuing keeps every patch entity unchanged.", + inputs: ["The patch-only scan results from the preceding diagnostic stage."], + mutations: ["None. Downloading or browsing the diagnostic records does not apply them."], + invariants: [ + "The imported patch remains unchanged for direct merge and cross-dataset reconciliation.", + "A displayed candidate is evidence for human review, not an automatic merge instruction.", + ], + output: "A reviewed diagnostic record; the workflow continues with the unchanged patch.", + }, + "review-direct": { + summary: + "Inspect the direct additions and same-ID updates before the verified merge is regenerated from untouched inputs.", + inputs: ["The direct-merge preview generated from the original base and patch."], + mutations: [ + "Approving this checkpoint does not apply the preview. It advances to later matching and reconciliation stages.", + ], + invariants: [ + "The original base and patch remain available so cumulative changes can be regenerated deterministically.", + "No proximity or exact cross-ID match has been included in this preview.", + ], + output: "Approval to continue; no OSM data changes at this checkpoint.", + diagram: "direct-merge", + }, + "review-cumulative-exact": { + summary: + "Inspect the cumulative direct, exact-reconciliation, and accepted imported-data changes before applying them atomically.", + inputs: [ + "A changeset regenerated from the untouched base and patch.", + "Enabled exact reconciliation and any saved imported-data match decisions.", + ], + mutations: [ + "Applying replaces the in-memory base with the validated cumulative result.", + "Patch additions, same-ID updates, reconciled references, and explicitly accepted fuzzy actions are applied together.", + ], + invariants: [ + "Pre-existing base coordinates, ordered way references, and ordered relation members remain protected from fuzzy conflation.", + "The integrity validator rejects dangling references, degenerate routable ways, invalid restrictions, and incompatible grade connections.", + "The source PBF files remain unchanged.", + ], + output: + "A verified in-memory base containing the accepted merge, ready for intersection creation.", + warning: + "Applying this changeset is the first irreversible in-memory workflow boundary. Return to the original inputs to undo it.", + diagram: "exact-reconciliation", + }, + "review-cumulative-without-exact": { + summary: + "Inspect the cumulative direct merge and accepted imported-data changes before applying them atomically.", + inputs: [ + "A changeset regenerated from the untouched base and patch with cross-ID reconciliation disabled.", + "Any saved imported-data match decisions.", + ], + mutations: [ + "Applying replaces the in-memory base with the validated cumulative result.", + "Patch additions, same-ID updates, and explicitly accepted fuzzy actions are applied together.", + ], + invariants: [ + "Coordinate-equal entities with different IDs remain separate unless an explicit imported-data decision matches them.", + "The integrity validator rejects dangling references, degenerate routable ways, invalid restrictions, and incompatible grade connections.", + "The source PBF files remain unchanged.", + ], + output: + "A verified in-memory base containing the accepted merge, ready for intersection creation.", + warning: + "Applying this changeset is the first irreversible in-memory workflow boundary. Return to the original inputs to undo it.", + diagram: "direct-merge", + }, + "review-intersections": { + summary: + "Inspect proposed shared nodes and way-reference edits before completing the merged dataset.", + inputs: [ + "An intersection-only changeset generated after the cumulative merge was applied and indexed.", + ], + mutations: [ + "Applying can reuse a compatible node, replace another nearby endpoint reference, merge non-conflicting node tags, rewrite an affected via-node restriction, and add crossing=yes.", + "When reuse would collapse a way, applying instead creates one exact node and inserts it into both ways in geometric order.", + ], + invariants: [ + "Grade-separated or tag-incompatible crossings remain disconnected.", + "Node reuse cannot collapse a way; Osmix creates a dedicated exact crossing node when reuse would make topology invalid.", + "Restriction and reference integrity must remain valid.", + ], + output: "The final in-memory merged dataset, or an unchanged dataset when there are no edits.", + diagram: "intersections", + }, + "match-imported": { + summary: + "Review safe nearby matches for selected property transfer, imported-network attachment, or both.", + inputs: [ + "The untouched patch compared only against the immutable original base.", + "Explicit tag keys, network-attachment choice, and candidate search radius.", + "Geometry, routing family, bearings, access, grade context, and relation participation used as evidence.", + ], + mutations: [ + "Discovery and decisions do not mutate OSM data.", + "Property transfer can overwrite only selected base tag values; an absent patch value never deletes a base value.", + "Network attachment can later rewrite only patch-created way references to a preserved base node.", + "An equivalent one-to-one patch way can be suppressed after transfer; only newly imported tagless nodes left unreferenced by every way and relation are cleaned up.", + ], + invariants: [ + "Fuzzy matching preserves original base IDs, coordinates, ordered way references, and relation membership; ordinary same-ID patch updates remain authoritative in the direct-merge baseline.", + "Protected structural tags cannot transfer fuzzily, and routing-affecting properties require review.", + "Ambiguous, many-to-one, grade-conflicting, restricted, or structurally invalid candidates are not accepted automatically.", + ], + output: + "Saved automatic, accepted, rejected, review, blocked, and unmatched candidate decisions for cumulative generation.", + warning: + "Rejecting a proposed match rejects only conflation. It does not delete the imported entity, which still proceeds through ordinary direct merge when otherwise unmatched.", + diagram: "fuzzy-conflation", + }, + reconcile: { + summary: + "Regenerate the merge and represent exact, uniquely compatible patch entities with preserved base entities.", + inputs: [ + "The untouched base and patch, plus any reviewed imported-data decisions.", + "Exact coordinates or ordered geometry together with routing, access, grade, and relation context.", + ], + mutations: [ + "The generated changeset can rewrite patch references to the surviving base entity.", + "Equivalent patch ways can be suppressed while the base way ID is preserved.", + "Base nodes receive missing non-conflicting patch tags; base ways receive missing non-conflicting descriptive patch tags.", + "No generated changes are applied until the following review screen.", + ], + invariants: [ + "Candidates must have a unique compatible base target; ambiguous matches remain separate.", + "Reconciliation never scans base-to-base or patch-to-patch and never follows transitive replacement chains.", + "Conflicting routing-critical tags, restrictions, or grade context prevent unsafe reconciliation.", + ], + output: + "A cumulative changeset with direct merge, optional accepted fuzzy actions, and optional exact reconciliation.", + diagram: "exact-reconciliation", + }, + intersections: { + summary: + "Find compatible same-grade highway crossings and propose shared nodes that connect their way geometry.", + inputs: [ + "The indexed cumulative merge result and patch ways that may cross existing ways.", + "Precise segment geometry plus highway, layer, level, bridge, tunnel, and covered context.", + ], + mutations: [ + "The generated changeset can reuse a compatible nearby node, replace the other nearby endpoint reference, merge non-conflicting node tags, rewrite affected via-node restrictions, and add crossing=yes.", + "Without safe reuse, it creates one exact crossing node and inserts it into both ways in geometric order.", + "No intersection edits are applied until the following review screen.", + ], + invariants: [ + "Grade-separated and incompatible crossings remain disconnected.", + "A nearby endpoint is reused only when the replacement cannot collapse or otherwise degenerate a way.", + "Multiple intersections on one segment are inserted in traversal order.", + ], + output: "An intersection-only changeset for final review and application.", + diagram: "intersections", + }, + final: { + summary: + "Inspect the in-memory merged result and download a new PBF when its topology and entities look correct.", + inputs: [ + "The applied cumulative merge and, unless skipped, the applied intersection changeset.", + "Optional routing diagnostics retained from imported-network attachment.", + ], + mutations: [ + "Map inspection and entity selection are read-only.", + "Downloading serializes the in-memory result as a new OSM PBF file.", + "Saving stores the indexed Osmix dataset in browser storage for later reuse; it does not create a PBF file.", + ], + invariants: [ + "The original base and patch files on disk remain untouched.", + "The merged dataset stays in browser memory until downloaded or saved to browser storage.", + ], + output: + "A downloaded merged OSM PBF, a reusable indexed browser copy, or both, depending on the selected actions.", + }, +} as const satisfies Record; + +function GuideList({ items }: { items: readonly string[] }) { + return ( +
    + {items.map((item) => ( +
  • {item}
    • + ))} +
+ ); +} + +function GuideHeading({ children }: { children: string }) { + return ( +
+ {children} +
+ ); +} + +export function MergeStepGuide({ + defaultOpen = false, + guideId, +}: { + defaultOpen?: boolean; + guideId: MergeStepGuideId; +}) { + const guide = MERGE_STEP_GUIDES[guideId]; + + return ( +
+

+ {guide.summary} +

+
+ How this step works + +
+ {"diagram" in guide ? ( +
+ +
+ ) : null} + +
+ Inputs + +
+ +
+ What can change + +
+ +
+ Safety guarantees + +
+ +
+ Output +

{guide.output}

+
+ + {"warning" in guide ? ( +
+ Caution +

{guide.warning}

+
+ ) : null} +
+ +
+
+ ); +} diff --git a/apps/merge/src/components/osm-changes-summary.tsx b/apps/merge/src/components/osm-changes-summary.tsx index 5cef0684..bac4f640 100644 --- a/apps/merge/src/components/osm-changes-summary.tsx +++ b/apps/merge/src/components/osm-changes-summary.tsx @@ -3,7 +3,7 @@ import { ArrowLeft, ArrowRight } from "lucide-react"; import type { OsmChange } from "osmix"; import type { OsmEntity, OsmNode, OsmRelation, OsmWay } from "osmix"; import { getEntityType, isNode, isRelation, isWay } from "osmix"; -import { useTransition } from "react"; +import { useId, useTransition } from "react"; import { cn } from "../lib/utils"; import { @@ -34,41 +34,57 @@ export default function ChangesSummary() { function ChangesSummaryTable() { const summary = useAtomValue(changesetStatsAtom); + const reconciliationHelpId = useId(); if (!summary || summary.totalChanges === 0) return No changes found; return ( - - - - total changes - {summary.totalChanges.toLocaleString()} - - - node changes - {summary.nodeChanges.toLocaleString()} - - - way changes - {summary.wayChanges.toLocaleString()} - - - relation changes - {summary.relationChanges.toLocaleString()} - - - - deduplicated nodes - {summary.deduplicatedNodes.toLocaleString()} - - - deduplicated nodes replaced - {summary.deduplicatedNodesReplaced.toLocaleString()} - - - intersection points found - {summary.intersectionPointsFound.toLocaleString()} - - -
+ <> + + + + Total changes + {summary.totalChanges.toLocaleString()} + + + Node changes + {summary.nodeChanges.toLocaleString()} + + + Way changes + {summary.wayChanges.toLocaleString()} + + + Relation changes + {summary.relationChanges.toLocaleString()} + + + + Reconciled nodes + {summary.deduplicatedNodes.toLocaleString()} + + + Node references rewritten + {summary.deduplicatedNodesReplaced.toLocaleString()} + + + Reconciled ways + {summary.deduplicatedWays.toLocaleString()} + + + Intersection points found + {summary.intersectionPointsFound.toLocaleString()} + + + Intersection nodes created + {summary.intersectionNodesCreated.toLocaleString()} + + +
+

+ Reconciliation resolves equivalent entities to one surviving entity instead of retaining + both. Node references rewritten counts way node references and relation node members changed + from a reconciled node ID to its surviving node ID. +

+ ); } @@ -110,22 +126,28 @@ export function ChangesFilters() { return (
- {(["create", "modify", "delete"] as const).map((value) => ( - - ))} - {(["node", "way", "relation"] as const).map((value) => ( - - ))} +
+ Change type + {(["create", "modify", "delete"] as const).map((value) => ( + + ))} +
+
+ Entity type + {(["node", "way", "relation"] as const).map((value) => ( + + ))} +
); } diff --git a/apps/merge/src/components/osm-info-table.tsx b/apps/merge/src/components/osm-info-table.tsx index 1d2d8d71..844ddc11 100644 --- a/apps/merge/src/components/osm-info-table.tsx +++ b/apps/merge/src/components/osm-info-table.tsx @@ -19,7 +19,9 @@ export default function OsmInfoTable({ /** Alternative to file - used when loading from storage */ fileInfo?: StoredFileInfo | null; }) { - // Get file name and size from either file or fileInfo + // Local files retain their browser-provided name. Stored and URL-loaded files + // use the metadata captured when the worker registered the dataset. + const fileName = file?.name ?? fileInfo?.fileName; const fileSize = file?.size ?? fileInfo?.fileSize; if (!osm || (!file && !fileInfo)) return null; @@ -37,6 +39,12 @@ export default function OsmInfoTable({ + {fileName ? ( + + file name + {fileName} + + ) : null} {fileSize != null && ( size diff --git a/apps/merge/src/components/osm-input-card-header.tsx b/apps/merge/src/components/osm-input-card-header.tsx new file mode 100644 index 00000000..f95d33bf --- /dev/null +++ b/apps/merge/src/components/osm-input-card-header.tsx @@ -0,0 +1,62 @@ +import { DownloadIcon, XIcon } from "lucide-react"; + +import ActionButton from "./action-button"; +import { ButtonGroup } from "./ui/button-group"; +import { CardAction, CardDescription, CardHeader, CardTitle } from "./ui/card"; + +/** + * Loaded-input chrome shared by the Merge workflow and its lightweight browser + * harness. Keeping this independent from OSM parsing lets responsive and action + * behavior use the production component without repeatedly loading Monaco. + */ +export function OsmInputCardHeader({ + fileName, + kind, + loaded, + onClear, + onDownload, + title, +}: { + fileName?: string; + kind: "base" | "patch"; + loaded: boolean; + onClear: () => Promise; + onDownload: () => Promise; + title: string; +}) { + const kindLabel = kind === "base" ? "Base" : "Patch"; + + return ( + +
+ {title} + {fileName ? ( + + {fileName} + + ) : null} +
+ {loaded ? ( + + + } + title={`Download ${kind} OSM`} + onAction={onDownload} + variant="ghost" + /> + } + title={`Clear ${kind} OSM file`} + onAction={onClear} + variant="ghost" + /> + + + ) : null} + + ); +} diff --git a/apps/merge/src/components/osmix-map-sources.tsx b/apps/merge/src/components/osmix-map-sources.tsx new file mode 100644 index 00000000..5b2666d3 --- /dev/null +++ b/apps/merge/src/components/osmix-map-sources.tsx @@ -0,0 +1,33 @@ +import type { Osm } from "osmix"; + +import OsmixRasterSource from "./osmix-raster-source"; +import OsmixVectorOverlay from "./osmix-vector-overlay"; + +export function OsmixMapSources({ + activeTab, + baseOsm, + extractOsm, + patchOsm, +}: { + activeTab: string; + baseOsm: Osm | null; + extractOsm: Osm | null; + patchOsm: Osm | null; +}) { + return ( + <> + {/* Dataset IDs are content hashes, so they change after a merge. react-map-gl + source and layer IDs are immutable, so each map role and ID needs its own key. */} + {baseOsm && } + {patchOsm && } + {baseOsm && } + {patchOsm && } + {activeTab === "Extract" && extractOsm ? ( + <> + + + + ) : null} + + ); +} diff --git a/apps/merge/src/components/osmix-raster-source.tsx b/apps/merge/src/components/osmix-raster-source.tsx index b499d04a..e745ace4 100644 --- a/apps/merge/src/components/osmix-raster-source.tsx +++ b/apps/merge/src/components/osmix-raster-source.tsx @@ -17,6 +17,10 @@ export default function OsmixRasterSource({ const id = `${APPID}:${osmId}:${tileSize}:raster`; return ( ) { + return ( +
[data-slot=button]]:h-auto [&>[data-slot=button]]:min-h-9", + "[&>[data-slot=button]]:w-full [&>[data-slot=button]]:min-w-0", + "[&>[data-slot=button]]:shrink [&>[data-slot=button]]:whitespace-normal", + "[&>[data-slot=button]]:py-2 [&>[data-slot=button]]:leading-tight", + className, + )} + {...props} + /> + ); +} diff --git a/apps/merge/src/components/task-progress.tsx b/apps/merge/src/components/task-progress.tsx deleted file mode 100644 index f84c6258..00000000 --- a/apps/merge/src/components/task-progress.tsx +++ /dev/null @@ -1,42 +0,0 @@ -import { useEffect, useState } from "react"; - -import { useLog } from "../hooks/log"; -import { Progress } from "./ui/progress"; - -function formatElapsed(ms: number) { - const totalSeconds = Math.floor(ms / 1_000); - const minutes = Math.floor(totalSeconds / 60); - const seconds = totalSeconds % 60; - return `${minutes}:${String(seconds).padStart(2, "0")}`; -} - -/** - * Progress feedback for long-running worker tasks. Worker progress messages do - * not include a numeric percentage yet, so the bar is indeterminate; the latest - * log message and a live elapsed timer show that work is advancing. - */ -export default function TaskProgress() { - const { log, activeTasks, taskStartedAt } = useLog(); - const [now, setNow] = useState(() => Date.now()); - - useEffect(() => { - if (taskStartedAt == null) return; - const interval = setInterval(() => setNow(Date.now()), 1_000); - return () => clearInterval(interval); - }, [taskStartedAt]); - - if (activeTasks === 0 || taskStartedAt == null) return null; - const latest = log[log.length - 1]; - - return ( -
- -
-
{latest?.message}
-
- {formatElapsed(Math.max(0, now - taskStartedAt))} -
-
-
- ); -} diff --git a/apps/merge/src/hooks/osm.ts b/apps/merge/src/hooks/osm.ts index c1c7ba0b..ec6455b3 100644 --- a/apps/merge/src/hooks/osm.ts +++ b/apps/merge/src/hooks/osm.ts @@ -10,6 +10,7 @@ import type { import { useEffect, useEffectEvent, useRef, useState } from "react"; import { getBrowserLoadCapabilities } from "../lib/browser-capabilities"; +import { prepareMergedOsmState } from "../lib/merged-osm-state"; import { describeOsmLoadFailure, type OsmLoadFailureContext } from "../lib/osm-load-failure"; import { ensureOsmPbfDownloadName } from "../lib/osm-pbf-download-name"; import { showSaveFilePickerWithFallback } from "../lib/save-file-picker"; @@ -141,6 +142,7 @@ export function useOsmFile(osmKey: string) { setFile(file); sourceUrlRef.current = null; setOsm(null); + setOsmInfo(null); setFileInfo(null); setIsStored(false); setLoadFailure(null); @@ -566,56 +568,35 @@ export function useOsmFile(osmKey: string) { * If the content hasn't changed (same content hash as original), keeps original file info. */ const setMergedOsm = useEffectEvent(async (newOsmId: string, mergedFileName?: string) => { - // Get the new Osm instance from the worker - const newOsm = await osmWorker.get(newOsmId); - const newOsmInfo = newOsm.info(); + const prepared = await prepareMergedOsmState({ + currentFileInfo: fileInfo, + currentOsm: osm, + mergedFileName, + newOsmId, + worker: osmWorker, + }); // Check if anything actually changed using isEqual - if (newOsm.isEqual(osm) && fileInfo) { + if (prepared.kind === "unchanged") { // No changes - keep the original file info and stored state - setOsm(newOsm); - setOsmInfo(newOsmInfo); - setSelectedOsm(newOsm); + setOsm(prepared.osm); + setOsmInfo(prepared.osmInfo); + setSelectedOsm(prepared.osm); setLoadFailure(null); - return newOsm; + return prepared.osm; } - // Generate a new file name based on merge context (fallback to timestamp) - const timestamp = new Date().toISOString().slice(0, 19).replace(/[:]/g, "-"); - const newFileName = mergedFileName ?? `osmix-merged-${timestamp}.pbf`; - - // File size is estimated from entity counts (will be accurate after serialization) - const estimatedSize = - newOsmInfo.stats.nodes * 20 + newOsmInfo.stats.ways * 100 + newOsmInfo.stats.relations * 200; - - // Use content hash as the new ID to keep worker ID and storage key in sync - const newFileHash = newOsm.contentHash(); - const newFileInfo: StoredFileInfo = { - fileHash: newFileHash, - fileName: newFileName, - fileSize: estimatedSize, - }; - - // Re-register the Osm in the worker under the new fileHash so that - // downloadOsm, storeCurrentOsm, and other calls that use osmInfo.id - // will find the correct worker entry after saving/loading from storage. - if (newOsmId !== newFileHash) { - await osmWorker.rename(newOsmId, newFileHash); - } - - // Update osmInfo.id to match the new fileHash (worker registration key) - const updatedOsmInfo = { ...newOsmInfo, id: newFileHash }; - - // Update all state + // The helper has refreshed the content-addressed worker instance and metadata. + sourceUrlRef.current = null; setFile(null); // No actual File object for merged results - setFileInfo(newFileInfo); - setOsm(newOsm); - setOsmInfo(updatedOsmInfo); + setFileInfo(prepared.fileInfo); + setOsm(prepared.osm); + setOsmInfo(prepared.osmInfo); setIsStored(false); // New file, not stored yet - setSelectedOsm(newOsm); + setSelectedOsm(prepared.osm); setLoadFailure(null); - return newOsm; + return prepared.osm; }); const clearLoadFailure = useEffectEvent(() => setLoadFailure(null)); diff --git a/apps/merge/src/lib/conflation-workflow.ts b/apps/merge/src/lib/conflation-workflow.ts new file mode 100644 index 00000000..bb5d5add --- /dev/null +++ b/apps/merge/src/lib/conflation-workflow.ts @@ -0,0 +1,106 @@ +import type { OsmConflationBulkAction, OsmConflationOptions } from "osmix"; + +export interface ConflationFormState { + enabled: boolean; + transferProperties: boolean; + propertyKeys: string; + attachNetwork: boolean; + maxDistanceMeters: number; +} + +// These node-level accessibility tags produced useful Yakima matches without the +// thousands of unmatched way candidates introduced by broad surface/geometry keys. +export const DEFAULT_CONFLATION_PROPERTY_KEYS = [ + "barrier", + "crossing", + "kerb", + "tactile_paving", +] as const; + +export const DEFAULT_CONFLATION_FORM_STATE: ConflationFormState = { + enabled: false, + transferProperties: true, + propertyKeys: DEFAULT_CONFLATION_PROPERTY_KEYS.join(", "), + attachNetwork: false, + maxDistanceMeters: 1, +}; + +/** Parse a comma- or whitespace-separated tag-key field into stable unique keys. */ +export function parseConflationPropertyKeys(value: string): string[] { + return [ + ...new Set( + value + .split(/[\s,]+/) + .map((key) => key.trim()) + .filter(Boolean), + ), + ].sort(); +} + +/** Return the first configuration problem that must be resolved before discovery. */ +export function validateConflationForm(state: ConflationFormState): string | null { + if (!state.enabled) return null; + if (!Number.isFinite(state.maxDistanceMeters) || state.maxDistanceMeters <= 0) { + return "Match distance must be greater than zero."; + } + if (!state.transferProperties && !state.attachNetwork) { + return "Enable property transfer, network attachment, or both."; + } + if (state.transferProperties && parseConflationPropertyKeys(state.propertyKeys).length === 0) { + return "Enter at least one property key to transfer."; + } + return null; +} + +/** Convert the opt-in form into deterministic worker options. */ +export function toOsmConflationOptions( + state: ConflationFormState, +): OsmConflationOptions | undefined { + if (!state.enabled) return undefined; + const validationMessage = validateConflationForm(state); + if (validationMessage) throw new Error(validationMessage); + return { + propertyKeys: state.transferProperties ? parseConflationPropertyKeys(state.propertyKeys) : [], + attachNetwork: state.attachNetwork, + maxDistanceMeters: state.maxDistanceMeters, + automatic: "high-confidence", + }; +} + +export interface ConflationBulkActionCopy { + buttonLabel: string; + confirmLabel: string; + description: string; + title: string; +} + +/** Keep filter-wide action wording consistent between the toolbar and confirmation dialog. */ +export function conflationBulkActionCopy( + action: OsmConflationBulkAction, +): ConflationBulkActionCopy { + if (action === "transfer-properties") { + return { + buttonLabel: "Transfer properties", + confirmLabel: "Transfer properties", + description: + "Transfer the selected patch properties to every eligible base match in the current filters.", + title: "Transfer properties to filtered matches?", + }; + } + if (action === "attach-network") { + return { + buttonLabel: "Attach network", + confirmLabel: "Attach network", + description: + "Attach imported way references to every eligible base match in the current filters.", + title: "Attach the filtered imported network?", + }; + } + return { + buttonLabel: "Reject filtered", + confirmLabel: "Reject filtered matches", + description: + "Reject every filtered match that is not already rejected, including blocked and unmatched rows.", + title: "Reject all filtered matches?", + }; +} diff --git a/apps/merge/src/lib/merge-workflow.ts b/apps/merge/src/lib/merge-workflow.ts new file mode 100644 index 00000000..0edec5c1 --- /dev/null +++ b/apps/merge/src/lib/merge-workflow.ts @@ -0,0 +1,186 @@ +import type { + OsmChangesetStats, + OsmConflationGenerationResult, + OsmConflationOptions, + OsmConflationSummary, + OsmMergeOptions, +} from "osmix"; + +export type ChangesetReviewPurpose = "apply" | "diagnostic" | "preview"; + +/** + * A same-dataset comparison may surface suspicious entities for review, but its + * proposed edits must never be applied automatically. + */ +export const WITHIN_DATASET_DIAGNOSTIC_OPTIONS = { + deduplicateNodes: true, + deduplicateWays: true, +} as const satisfies Partial; + +/** Reconcile entities from the patch against the base without normalizing either input. */ +export const CROSS_DATASET_RECONCILIATION_OPTIONS = { + deduplicateNodes: true, + deduplicateWays: true, +} as const satisfies Partial; + +export const DIRECT_MERGE_OPTIONS = { + directMerge: true, +} as const satisfies Partial; + +/** Build a verified base merge from the untouched base and patch. */ +export function verifiedBaseMergeOptions(reconcile: boolean): Partial { + return { + ...DIRECT_MERGE_OPTIONS, + ...(reconcile ? CROSS_DATASET_RECONCILIATION_OPTIONS : {}), + }; +} + +export const INTERSECTION_OPTIONS = { + createIntersections: true, +} as const satisfies Partial; + +/** Options shared by the non-interactive, high-level merge workflow. */ +export const COMPLETE_MERGE_OPTIONS = { + ...verifiedBaseMergeOptions(true), + ...INTERSECTION_OPTIONS, +} as const satisfies Partial; + +/** Add explicit fuzzy conflation without changing the exact-only default object. */ +export function completeMergeOptions(conflation?: OsmConflationOptions): Partial { + return conflation ? { ...COMPLETE_MERGE_OPTIONS, conflation } : COMPLETE_MERGE_OPTIONS; +} + +/** Build the cumulative direct, exact, and reviewed-fuzzy verified merge options. */ +export function verifiedConflationMergeOptions( + reconcile: boolean, + conflation: OsmConflationOptions, +): Partial { + return { + ...verifiedBaseMergeOptions(reconcile), + conflation, + }; +} + +interface ConflationRunAllWorker { + discoverConflation( + baseOsmId: string, + patchOsmId: string, + options: OsmConflationOptions, + ): Promise; + generateConflationChangeset( + baseOsmId: string, + options: Partial, + ): Promise; + generateChangeset( + baseOsmId: string, + patchOsmId: string, + options: Partial, + ): Promise; + applyChangesAndReplace(osmId: string): Promise; +} + +interface RunConflationAllStepsOptions { + baseOsmId: string; + conflation: OsmConflationOptions; + isCancelled: () => boolean; + onBaseApplied?: () => void; + onDiscovered?: (summary: OsmConflationSummary) => void; + onGenerated?: (result: OsmConflationGenerationResult) => void; + onStageChange?: (stage: ConflationRunAllStage) => void; + patchOsmId: string; + worker: ConflationRunAllWorker; +} + +export type ConflationRunAllStage = + | "apply-verified-merge" + | "create-intersections" + | "discover-imported-data" + | "generate-verified-merge"; + +export type RunConflationAllStepsResult = + | { + generation: OsmConflationGenerationResult | null; + status: "cancelled"; + summary: OsmConflationSummary; + } + | { + generation: OsmConflationGenerationResult; + intersections: OsmChangesetStats; + status: "completed"; + summary: OsmConflationSummary; + }; + +/** + * Run explicit conflation from untouched inputs, then create intersections on the applied result. + * + * Cancellation is honored until the first apply. Once the base changes, the intersection stage is + * completed before returning so callers never expose an incomplete result as a successful merge. + */ +export async function runConflationAllSteps({ + baseOsmId, + conflation, + isCancelled, + onBaseApplied, + onDiscovered, + onGenerated, + onStageChange, + patchOsmId, + worker, +}: RunConflationAllStepsOptions): Promise { + onStageChange?.("discover-imported-data"); + const summary = await worker.discoverConflation(baseOsmId, patchOsmId, conflation); + onDiscovered?.(summary); + if (isCancelled()) return { generation: null, status: "cancelled", summary }; + + onStageChange?.("generate-verified-merge"); + const generation = await worker.generateConflationChangeset( + baseOsmId, + verifiedBaseMergeOptions(true), + ); + onGenerated?.(generation); + if (isCancelled()) return { generation, status: "cancelled", summary }; + + // This is the first irreversible stage. After it succeeds, finish or explicitly + // expose the intersection retry state instead of pretending cancellation rolled back. + onStageChange?.("apply-verified-merge"); + await worker.applyChangesAndReplace(generation.stats.osmId); + onBaseApplied?.(); + + onStageChange?.("create-intersections"); + const intersections = await worker.generateChangeset(baseOsmId, patchOsmId, INTERSECTION_OPTIONS); + await worker.applyChangesAndReplace(intersections.osmId); + + return { generation, intersections, status: "completed", summary }; +} + +/** + * Restore any available candidate state, then leave the progress-only screen after a failed run. + * Showing the review in a `finally` block keeps discovery failures themselves retryable. + */ +export async function recoverConflationRunAllFailure({ + restoreReview, + showReview, +}: { + restoreReview?: () => Promise; + showReview: () => void; +}): Promise<{ error: unknown } | null> { + let restoreFailure: { error: unknown } | null = null; + try { + await restoreReview?.(); + } catch (error) { + restoreFailure = { error }; + } finally { + showReview(); + } + return restoreFailure; +} + +export function canApplyChangeset(purpose: ChangesetReviewPurpose): boolean { + return purpose === "apply"; +} + +/** Clear the patch overlay before showing the verified merged result. */ +export function finalizeVerifiedMerge(clearPatch: () => void, showFinalResult: () => void): void { + clearPatch(); + showFinalResult(); +} diff --git a/apps/merge/src/lib/merged-osm-state.ts b/apps/merge/src/lib/merged-osm-state.ts new file mode 100644 index 00000000..b73b1a99 --- /dev/null +++ b/apps/merge/src/lib/merged-osm-state.ts @@ -0,0 +1,78 @@ +import type { Osm, OsmInfo } from "osmix"; + +import type { StoredFileInfo } from "../workers/osm.worker"; + +interface MergedOsmWorker { + get(osmId: string): Promise; + rename(fromId: string, toId: string): Promise; +} + +interface PrepareMergedOsmStateOptions { + currentOsm: Osm | null; + currentFileInfo: StoredFileInfo | null; + mergedFileName?: string; + newOsmId: string; + now?: Date; + worker: MergedOsmWorker; +} + +export type PreparedMergedOsmState = + | { + kind: "unchanged"; + osm: Osm; + osmInfo: OsmInfo; + } + | { + fileInfo: StoredFileInfo; + kind: "changed"; + osm: Osm; + osmInfo: OsmInfo; + }; + +/** + * Resolve a merged dataset to its content-addressed worker ID and refreshed metadata. + * + * Renaming a worker dataset re-registers it as a new `Osm` instance. The post-rename + * lookup is required so callers never retain an object whose ID has been removed from + * the worker registry. + */ +export async function prepareMergedOsmState({ + currentOsm, + currentFileInfo, + mergedFileName, + newOsmId, + now = new Date(), + worker, +}: PrepareMergedOsmStateOptions): Promise { + let mergedOsm = await worker.get(newOsmId); + const initialInfo = mergedOsm.info(); + + if (mergedOsm.isEqual(currentOsm) && currentFileInfo) { + return { kind: "unchanged", osm: mergedOsm, osmInfo: initialInfo }; + } + + const contentHash = mergedOsm.contentHash(); + if (newOsmId !== contentHash) { + await worker.rename(newOsmId, contentHash); + mergedOsm = await worker.get(contentHash); + } + + const refreshedInfo = mergedOsm.info(); + const timestamp = now.toISOString().slice(0, 19).replace(/[:]/g, "-"); + const fileName = mergedFileName ?? `osmix-merged-${timestamp}.pbf`; + const fileInfo: StoredFileInfo = { + fileHash: contentHash, + fileName, + fileSize: + refreshedInfo.stats.nodes * 20 + + refreshedInfo.stats.ways * 100 + + refreshedInfo.stats.relations * 200, + }; + + return { + fileInfo, + kind: "changed", + osm: mergedOsm, + osmInfo: { ...refreshedInfo, id: contentHash }, + }; +} diff --git a/apps/merge/src/pages/merge.tsx b/apps/merge/src/pages/merge.tsx index 02fd1833..4d966608 100644 --- a/apps/merge/src/pages/merge.tsx +++ b/apps/merge/src/pages/merge.tsx @@ -8,13 +8,13 @@ import ExtractBlock from "../blocks/extract"; import InspectBlock from "../blocks/inspect"; import MergeBlock from "../blocks/merge"; import Basemap, { type MapInitialViewState } from "../components/basemap"; +import { ConflationComparisonLayer } from "../components/conflation-comparison-layer"; import CustomControl from "../components/custom-control"; import EntityDetailsMapControl from "../components/entity-details-map-control"; import ExtractMapLayers from "../components/extract-map-layers"; import { Main, MapContent, Sidebar } from "../components/layout"; import OsmFileMapControl from "../components/osm-file-map-control"; -import OsmixRasterSource from "../components/osmix-raster-source"; -import OsmixVectorOverlay from "../components/osmix-vector-overlay"; +import { OsmixMapSources } from "../components/osmix-map-sources"; import SelectedEntityLayer from "../components/selected-entity-layer"; import SidebarLog from "../components/sidebar-log"; import { buttonVariants } from "../components/ui/button"; @@ -186,19 +186,16 @@ export default function Merge() { - {base.osm && } - {patch.osm && } - {base.osm && } - {patch.osm && } - {activeTab === "Extract" && extract.osm ? ( - <> - - - - ) : null} + {activeTab === "Extract" ? : null} + {activeTab === "Merge" ? : null} ({ + ...DEFAULT_CONFLATION_FORM_STATE, +}); + +export const conflationComparisonAtom = atom({ + type: "FeatureCollection", + features: [], +}); + +export const conflationSummaryAtom = atom(null); +export const conflationCandidatePageAtom = atom(null); +export const conflationCandidatePageIndexAtom = atom(0); +export const conflationCandidateFilterAtom = atom({}); +export const conflationDecisionsAtom = atom([]); +export const conflationRoutingDiagnosticsAtom = atom(null); + +export const resetConflationReviewAtom = atom(null, (_get, set) => { + set(conflationSummaryAtom, null); + set(conflationCandidatePageAtom, null); + set(conflationCandidatePageIndexAtom, 0); + set(conflationCandidateFilterAtom, {}); + set(conflationDecisionsAtom, []); + set(conflationRoutingDiagnosticsAtom, null); + set(conflationComparisonAtom, { type: "FeatureCollection", features: [] }); +}); diff --git a/apps/merge/tests/automatic-merge-progress.test.ts b/apps/merge/tests/automatic-merge-progress.test.ts new file mode 100644 index 00000000..ae70f616 --- /dev/null +++ b/apps/merge/tests/automatic-merge-progress.test.ts @@ -0,0 +1,47 @@ +import { createElement } from "react"; +import { renderToStaticMarkup } from "react-dom/server"; +import { describe, expect, it } from "vitest"; + +import { + AutomaticMergeProgress, + CONFLATION_AUTOMATIC_MERGE_STEPS, + EXACT_AUTOMATIC_MERGE_STEPS, +} from "../src/components/automatic-merge-progress"; + +describe("automatic merge progress", () => { + it("marks earlier exact steps complete and the active step as running", () => { + const html = renderToStaticMarkup( + createElement(AutomaticMergeProgress, { + currentStepId: "refresh-result", + steps: EXACT_AUTOMATIC_MERGE_STEPS, + }), + ); + + expect(html).toContain('aria-label="Automatic merge progress"'); + expect(html).toContain('data-status="completed"'); + expect(html).toContain('aria-current="step"'); + expect(html).toContain("Merge, reconcile, and create intersections"); + expect(html).toContain("Refresh merged dataset"); + expect(html).toContain("1 of 2 steps completed"); + }); + + it("distinguishes completed, running, and remaining conflation stages", () => { + const html = renderToStaticMarkup( + createElement(AutomaticMergeProgress, { + currentStepId: "apply-verified-merge", + elapsedMs: 582_000, + latestMessage: "Applying verified imported-data changes", + steps: CONFLATION_AUTOMATIC_MERGE_STEPS, + }), + ); + + expect(html.match(/data-status="completed"/g)).toHaveLength(2); + expect(html.match(/data-status="running"/g)).toHaveLength(1); + expect(html.match(/data-status="remaining"/g)).toHaveLength(2); + expect(html).toContain("Apply verified merge changes is running"); + expect(html).toContain("2 of 5 steps completed"); + expect(html).toContain("9:42"); + expect(html).toContain("Applying verified imported-data changes"); + expect(html).not.toContain('role="progressbar"'); + }); +}); diff --git a/apps/merge/tests/conflation-workflow.test.ts b/apps/merge/tests/conflation-workflow.test.ts new file mode 100644 index 00000000..ab492f8b --- /dev/null +++ b/apps/merge/tests/conflation-workflow.test.ts @@ -0,0 +1,172 @@ +import { createElement } from "react"; +import { renderToStaticMarkup } from "react-dom/server"; +import { describe, expect, it } from "vitest"; + +import { + ConflationBulkActions, + ConflationResultsHeader, +} from "../src/components/conflation-review"; +import { + conflationBulkActionCopy, + DEFAULT_CONFLATION_FORM_STATE, + DEFAULT_CONFLATION_PROPERTY_KEYS, + parseConflationPropertyKeys, + toOsmConflationOptions, + validateConflationForm, +} from "../src/lib/conflation-workflow"; + +describe("conflation workflow configuration", () => { + it("keeps fuzzy matching disabled by default", () => { + expect(DEFAULT_CONFLATION_FORM_STATE).toEqual({ + enabled: false, + transferProperties: true, + propertyKeys: "barrier, crossing, kerb, tactile_paving", + attachNetwork: false, + maxDistanceMeters: 1, + }); + expect(parseConflationPropertyKeys(DEFAULT_CONFLATION_FORM_STATE.propertyKeys)).toEqual([ + ...DEFAULT_CONFLATION_PROPERTY_KEYS, + ]); + expect(validateConflationForm(DEFAULT_CONFLATION_FORM_STATE)).toBeNull(); + }); + + it("normalizes explicit property keys", () => { + expect(parseConflationPropertyKeys("name, surface name\noperator")).toEqual([ + "name", + "operator", + "surface", + ]); + }); + + it("requires at least one selected operation", () => { + expect( + validateConflationForm({ + ...DEFAULT_CONFLATION_FORM_STATE, + enabled: true, + transferProperties: false, + }), + ).toBe("Enable property transfer, network attachment, or both."); + }); + + it("requires explicit property keys when property transfer is enabled", () => { + expect( + validateConflationForm({ + ...DEFAULT_CONFLATION_FORM_STATE, + enabled: true, + propertyKeys: "", + }), + ).toBe("Enter at least one property key to transfer."); + }); + + it("accepts network-only matching without property keys", () => { + const state = { + ...DEFAULT_CONFLATION_FORM_STATE, + enabled: true, + transferProperties: false, + attachNetwork: true, + }; + expect(validateConflationForm(state)).toBeNull(); + expect(toOsmConflationOptions(state)).toEqual({ + propertyKeys: [], + attachNetwork: true, + maxDistanceMeters: 1, + automatic: "high-confidence", + }); + }); + + it("builds explicit high-confidence property-transfer options", () => { + expect( + toOsmConflationOptions({ + ...DEFAULT_CONFLATION_FORM_STATE, + enabled: true, + propertyKeys: "operator, name operator", + }), + ).toEqual({ + propertyKeys: ["name", "operator"], + attachNetwork: false, + maxDistanceMeters: 1, + automatic: "high-confidence", + }); + }); + + it("rejects invalid match distances", () => { + expect( + validateConflationForm({ + ...DEFAULT_CONFLATION_FORM_STATE, + enabled: true, + maxDistanceMeters: 0, + }), + ).toBe("Match distance must be greater than zero."); + }); + + it("uses action-specific labels and explicit filter-wide confirmation wording", () => { + expect(conflationBulkActionCopy("transfer-properties")).toMatchObject({ + buttonLabel: "Transfer properties", + title: "Transfer properties to filtered matches?", + }); + expect(conflationBulkActionCopy("attach-network")).toMatchObject({ + buttonLabel: "Attach network", + title: "Attach the filtered imported network?", + }); + expect(conflationBulkActionCopy("reject")).toEqual({ + buttonLabel: "Reject filtered", + confirmLabel: "Reject filtered matches", + description: + "Reject every filtered match that is not already rejected, including blocked and unmatched rows.", + title: "Reject all filtered matches?", + }); + }); + + it("renders filter-wide counts and disables actions with no decisions to change", () => { + const preview = { + action: "transfer-properties" as const, + filteredCandidates: 145, + eligibleCandidates: 145, + changedCandidates: 145, + skippedCandidates: 0, + automaticCandidates: 145, + reviewCandidates: 0, + overriddenDecisions: 0, + }; + const html = renderToStaticMarkup( + createElement(ConflationBulkActions, { + bulkActions: { + "transfer-properties": preview, + "attach-network": { + ...preview, + action: "attach-network", + changedCandidates: 12, + }, + reject: { + ...preview, + action: "reject", + changedCandidates: 0, + }, + }, + filter: { status: "automatic" }, + onBulkDecision: async () => {}, + }), + ); + + expect(html).toContain("Bulk decisions"); + expect(html).toContain('aria-label="About bulk decisions"'); + expect(html).not.toContain("every match in the current filters across all pages"); + expect(html).toContain("Transfer properties (145)"); + expect(html).toContain("Attach network (12)"); + expect(html).toMatch(/]*disabled=""[^>]*>Reject filtered \(0\)<\/button>/); + }); + + it("marks previous filtered results stale while the worker refreshes them", () => { + const html = renderToStaticMarkup( + createElement(ConflationResultsHeader, { + isFilterPending: true, + totalCandidates: 987_654, + }), + ); + + expect(html).toContain("Filtered matches (987,654, stale)"); + expect(html).toContain("Updating filters…"); + expect(html).toContain('role="status"'); + expect(html).toContain('aria-live="polite"'); + }); +}); diff --git a/apps/merge/tests/merge-inline-help.test.ts b/apps/merge/tests/merge-inline-help.test.ts new file mode 100644 index 00000000..8e778426 --- /dev/null +++ b/apps/merge/tests/merge-inline-help.test.ts @@ -0,0 +1,179 @@ +import { createStore, Provider } from "jotai"; +import type { OsmConflationCandidateView, OsmConflationRoutingDiagnostics } from "osmix"; +import { createElement } from "react"; +import { renderToStaticMarkup } from "react-dom/server"; +import { describe, expect, it, vi } from "vitest"; + +vi.mock("../src/state/worker", () => ({ + osmWorker: { + getChangesetPage: vi.fn(), + setChangesetFilters: vi.fn(), + }, +})); + +import { ConflationConfig } from "../src/components/conflation-config"; +import { + CandidateActionStatuses, + CandidateActions, + CandidateEvidence, + conflationCandidateTitle, + conflationReasonLabel, + ConflationStatusLegend, +} from "../src/components/conflation-review"; +import { ConflationRoutingDiagnostics } from "../src/components/conflation-routing-diagnostics"; +import ChangesSummary, { ChangesFilters } from "../src/components/osm-changes-summary"; +import { changesetStatsAtom } from "../src/state/changes"; +import { conflationFormAtom } from "../src/state/conflation"; + +const CANDIDATE: OsmConflationCandidateView = { + id: "node:11:22", + entityType: "node", + sourceId: 11, + targetId: 22, + status: "review", + reasons: ["routing-property"], + propertyTransfer: { status: "review", reasons: ["routing-property"] }, + networkAttachment: { status: "automatic", reasons: [] }, + evidence: { + distanceMeters: 0.25, + sourceRoutingFamilies: ["pedestrian"], + targetRoutingFamilies: ["bicycle-shared"], + tagDiff: [ + { + key: "crossing", + baseValue: "unmarked", + patchValue: "marked", + protected: false, + routing: false, + }, + ], + bearingDifferenceDegrees: 4, + lengthDifferenceRatio: 0.02, + maxGeometryDistanceMeters: 0.4, + }, +}; + +function renderWithStore( + element: React.ReactNode, + configure: (store: ReturnType) => void, +) { + const store = createStore(); + configure(store); + return renderToStaticMarkup(createElement(Provider, { store }, element)); +} + +describe("merge inline guidance", () => { + it("keeps proximity labels visible and moves detailed help into info tooltips", () => { + const html = renderWithStore(createElement(ConflationConfig), (store) => { + store.set(conflationFormAtom, { + enabled: true, + transferProperties: true, + propertyKeys: "barrier, crossing, kerb, tactile_paving", + attachNetwork: true, + maxDistanceMeters: 1, + }); + }); + + expect(html).toContain("OSM tag keys to transfer"); + expect(html).toContain("Candidate search radius (meters)"); + expect(html).toContain('aria-label="About proximity matching"'); + expect(html).toContain('aria-label="About property transfer"'); + expect(html).toContain('aria-label="About transferable OSM tags"'); + expect(html).toContain('aria-label="About network attachment"'); + expect(html).toContain('aria-label="About candidate search radius"'); + expect(html).toContain('aria-label="About automatic matching decisions"'); + expect(html).not.toContain("Distance alone never guarantees acceptance"); + expect(html).not.toContain("routing-affecting tags require review"); + expect(html).not.toContain("equivalent one-to-one imported"); + expect(html).not.toContain("tagless nodes"); + expect(html).not.toContain("referenced by any way or relation"); + }); + + it("humanizes candidate statuses, reasons, evidence, and actions", () => { + const legend = renderToStaticMarkup(createElement(ConflationStatusLegend)); + const evidence = renderToStaticMarkup( + createElement(CandidateEvidence, { candidate: CANDIDATE }), + ); + const actions = renderToStaticMarkup( + createElement(CandidateActions, { candidate: CANDIDATE, onDecision: async () => {} }), + ); + const actionStatuses = renderToStaticMarkup( + createElement(CandidateActionStatuses, { candidate: CANDIDATE }), + ); + + expect(legend).toContain('aria-label="About candidate statuses"'); + expect(legend).not.toContain("at least one action needs a decision"); + expect(conflationReasonLabel("would-collapse-way")).toBe("Attachment would collapse a way"); + expect(conflationCandidateTitle(CANDIDATE)).toBe("Imported node 11 → Base node 22"); + expect(evidence).toContain('aria-label="About candidate evidence metrics"'); + expect(evidence).not.toContain("Distance finds nearby candidates"); + expect(evidence).toContain("Imported routing family"); + expect(evidence).toContain("Base routing family"); + expect(evidence).toContain("Property"); + expect(evidence).toContain("Base value"); + expect(evidence).toContain("Imported value"); + expect(actions).toContain("Transfer + attach"); + expect(actionStatuses).toContain("Property transfer"); + expect(actionStatuses).toContain("Needs review"); + expect(actionStatuses).toContain("Network attachment"); + expect(actionStatuses).toContain("Automatic"); + + const wayStatuses = renderToStaticMarkup( + createElement(CandidateActionStatuses, { + candidate: { ...CANDIDATE, entityType: "way", networkAttachment: null }, + }), + ); + expect(wayStatuses).not.toContain("Network attachment"); + }); + + it("defines the routing baseline, metrics, signed deltas, and mode invariants", () => { + const mode = { + before: { components: 2, edges: 2, nodes: 3, routableNodes: 3 }, + after: { components: 1, edges: 4, nodes: 4, routableNodes: 4 }, + delta: { components: -1, edges: 2, nodes: 1, routableNodes: 1 }, + }; + const diagnostics: OsmConflationRoutingDiagnostics = { car: mode, walk: mode }; + const html = renderToStaticMarkup(createElement(ConflationRoutingDiagnostics, { diagnostics })); + + expect(html).toContain("including exact reconciliation when selected"); + expect(html).toContain("Routable nodes"); + expect(html).toContain("Directed edges"); + expect(html).toContain("Connected components"); + expect(html).toContain("weakly connected groups"); + expect(html).toContain("does not guarantee travel in both directions"); + expect(html).toContain("Signed delta"); + expect(html).toContain(">+2<"); + expect(html).toContain("walk-only attachment should not change CAR topology"); + expect(html).toContain("do not prove that routing is correct"); + }); + + it("shows reconciliation and intersection statistics with labeled filter groups", () => { + const html = renderWithStore( + createElement("div", null, createElement(ChangesSummary), createElement(ChangesFilters)), + (store) => { + store.set(changesetStatsAtom, { + osmId: "merged", + totalChanges: 25, + nodeChanges: 10, + wayChanges: 9, + relationChanges: 6, + deduplicatedNodes: 3, + deduplicatedNodesReplaced: 7, + deduplicatedWays: 2, + intersectionPointsFound: 5, + intersectionNodesCreated: 4, + }); + }, + ); + + expect(html).toContain("Reconciled nodes"); + expect(html).toContain("Node references rewritten"); + expect(html).toContain("Reconciled ways"); + expect(html).toContain("Intersection nodes created"); + expect(html).toContain("way node references and relation node members changed"); + expect(html).toContain("one surviving entity"); + expect(html).toContain(" { + it("defines detailed guidance for every workflow and review variant", () => { + expect(MERGE_STEP_GUIDE_IDS).toEqual(expectedGuideIds); + expect(Object.keys(MERGE_STEP_GUIDES)).toEqual(expectedGuideIds); + + for (const guideId of expectedGuideIds) { + const guide = MERGE_STEP_GUIDES[guideId]; + expect(guide.summary.length, `${guideId} summary`).toBeGreaterThan(20); + expect(guide.inputs.length, `${guideId} inputs`).toBeGreaterThan(0); + expect(guide.mutations.length, `${guideId} mutations`).toBeGreaterThan(0); + expect(guide.invariants.length, `${guideId} safety guarantees`).toBeGreaterThan(0); + expect(guide.output.length, `${guideId} output`).toBeGreaterThan(20); + expect("diagram" in guide ? guide.diagram : undefined).toBe(expectedDiagrams[guideId]); + } + }); + + it("renders the short summary and a closed detailed disclosure", () => { + const html = renderToStaticMarkup(createElement(MergeStepGuide, { guideId: "select" })); + + expect(html).toContain('data-slot="merge-step-guide"'); + expect(html).toContain('data-guide-id="select"'); + expect(html).toContain('data-slot="merge-step-guide-summary"'); + expect(html).toContain(MERGE_STEP_GUIDES.select.summary); + expect(html).toContain("How this step works"); + expect(html).toContain('aria-expanded="false"'); + expect(html).not.toContain('data-slot="merge-step-guide-details"'); + }); + + it("keeps the exact-off review free of exact-reconciliation behavior", () => { + const guide = MERGE_STEP_GUIDES["review-cumulative-without-exact"]; + const copy = [ + guide.summary, + ...guide.inputs, + ...guide.mutations, + ...guide.invariants, + guide.output, + ].join(" "); + + expect(copy).not.toMatch(/exact[- ]reconcil/i); + expect(copy).not.toContain("reconciled references"); + expect(guide.diagram).toBe("direct-merge"); + }); + + it("renders semantic level-three headings for every detailed section", () => { + const html = renderToStaticMarkup( + createElement(MergeStepGuide, { defaultOpen: true, guideId: "run-all" }), + ); + + expect(html.match(/role="heading" aria-level="3"/g)).toHaveLength(5); + for (const heading of ["Inputs", "What can change", "Safety guarantees", "Output", "Caution"]) { + expect(html).toContain(heading); + } + }); + + it.each(MERGE_GUIDE_DIAGRAM_IDS)("renders an accessible, responsive %s diagram", (diagram) => { + const html = renderToStaticMarkup(createElement(MergeGuideDiagram, { diagram })); + const fontSizes = [...html.matchAll(/font-size="([0-9]+)"/g)].map((match) => Number(match[1])); + + expect(html).toContain('role="img"'); + expect(html).toContain(`data-diagram="${diagram}"`); + expect(html).toContain('viewBox="0 0 240 300"'); + expect(html).toContain('class="h-auto w-full max-w-full"'); + expect(html).toMatch(/aria-labelledby="[^"]+ [^"]+"/); + expect(html).toMatch(/[^<]+<\/title>/); + expect(html).toMatch(/<desc id="[^"]+">[^<]+<\/desc>/); + expect(Math.min(...fontSizes)).toBeGreaterThanOrEqual(10); + expect(html).not.toContain("foreignObject"); + }); + + it("uses Base UI open-state styling and hides the decorative chevron", () => { + const html = renderToStaticMarkup( + createElement( + Details, + { defaultOpen: true }, + createElement(DetailsSummary, null, "Technical details"), + createElement(DetailsContent, null, "Expanded content"), + ), + ); + + expect(html).toContain("data-panel-open:shadow-sm"); + expect(html).toContain("group-data-panel-open:rotate-180"); + expect(html).toMatch(/<svg[^>]*aria-hidden="true"/); + expect(html).toContain('aria-expanded="true"'); + expect(html).toContain("Expanded content"); + }); +}); diff --git a/apps/merge/tests/merge-worker-hash.test.ts b/apps/merge/tests/merge-worker-hash.test.ts index 84e6b12c..7b15bc8f 100644 --- a/apps/merge/tests/merge-worker-hash.test.ts +++ b/apps/merge/tests/merge-worker-hash.test.ts @@ -1,5 +1,7 @@ import { afterAll, beforeAll, describe, expect, it, vi } from "vitest"; +import { MergeWorker } from "../src/workers/osm.worker"; + vi.mock("comlink", async (importOriginal) => ({ ...(await importOriginal<typeof import("comlink")>()), expose: vi.fn(), @@ -12,11 +14,8 @@ class MockBroadcastChannel { } describe("MergeWorker streaming hashing", () => { - let MergeWorker: typeof import("../src/workers/osm.worker").MergeWorker; - - beforeAll(async () => { + beforeAll(() => { vi.stubGlobal("BroadcastChannel", MockBroadcastChannel); - ({ MergeWorker } = await import("../src/workers/osm.worker")); }); afterAll(() => { diff --git a/apps/merge/tests/merge-workflow.test.ts b/apps/merge/tests/merge-workflow.test.ts new file mode 100644 index 00000000..a4fa88c3 --- /dev/null +++ b/apps/merge/tests/merge-workflow.test.ts @@ -0,0 +1,247 @@ +import type { OsmChangesetStats, OsmConflationGenerationResult, OsmConflationSummary } from "osmix"; +import { describe, expect, it, vi } from "vitest"; + +import { + canApplyChangeset, + COMPLETE_MERGE_OPTIONS, + completeMergeOptions, + CROSS_DATASET_RECONCILIATION_OPTIONS, + finalizeVerifiedMerge, + INTERSECTION_OPTIONS, + recoverConflationRunAllFailure, + runConflationAllSteps, + verifiedConflationMergeOptions, + verifiedBaseMergeOptions, + WITHIN_DATASET_DIAGNOSTIC_OPTIONS, +} from "../src/lib/merge-workflow"; + +const changesetStats = (osmId: string, totalChanges: number): OsmChangesetStats => ({ + deduplicatedNodes: 0, + deduplicatedNodesReplaced: 0, + deduplicatedWays: 0, + intersectionNodesCreated: 0, + intersectionPointsFound: 0, + nodeChanges: totalChanges, + osmId, + relationChanges: 0, + totalChanges, + wayChanges: 0, +}); + +const summary: OsmConflationSummary = { + accepted: 0, + automatic: 1, + blocked: 0, + rejected: 0, + review: 1, + total: 3, + unmatched: 1, +}; + +const generation: OsmConflationGenerationResult = { + stats: changesetStats("base", 3), + routing: { + car: { + before: { components: 1, edges: 2, nodes: 2, routableNodes: 2 }, + after: { components: 1, edges: 2, nodes: 2, routableNodes: 2 }, + delta: { components: 0, edges: 0, nodes: 0, routableNodes: 0 }, + }, + walk: { + before: { components: 2, edges: 2, nodes: 3, routableNodes: 3 }, + after: { components: 1, edges: 4, nodes: 4, routableNodes: 4 }, + delta: { components: -1, edges: 2, nodes: 1, routableNodes: 1 }, + }, + }, +}; + +describe("merge workflow policy", () => { + it("keeps within-dataset duplicate scans diagnostic", () => { + expect(WITHIN_DATASET_DIAGNOSTIC_OPTIONS).toEqual({ + deduplicateNodes: true, + deduplicateWays: true, + }); + expect(canApplyChangeset("diagnostic")).toBe(false); + expect(canApplyChangeset("preview")).toBe(false); + }); + + it("uses the same cross-dataset reconciliation options in a complete merge", () => { + expect(COMPLETE_MERGE_OPTIONS).toMatchObject(CROSS_DATASET_RECONCILIATION_OPTIONS); + expect(COMPLETE_MERGE_OPTIONS).toEqual({ + deduplicateNodes: true, + deduplicateWays: true, + directMerge: true, + createIntersections: true, + }); + expect(canApplyChangeset("apply")).toBe(true); + }); + + it("keeps exact-only defaults while adding explicitly configured conflation", () => { + const conflation = { + propertyKeys: ["name"], + attachNetwork: false, + maxDistanceMeters: 1, + automatic: "high-confidence" as const, + }; + + expect(completeMergeOptions()).toBe(COMPLETE_MERGE_OPTIONS); + expect(completeMergeOptions(conflation)).toEqual({ + ...COMPLETE_MERGE_OPTIONS, + conflation, + }); + expect(verifiedConflationMergeOptions(true, conflation)).toEqual({ + directMerge: true, + deduplicateNodes: true, + deduplicateWays: true, + conflation, + }); + }); + + it("regenerates the verified base merge from the original inputs", () => { + expect(verifiedBaseMergeOptions(false)).toEqual({ + directMerge: true, + }); + expect(verifiedBaseMergeOptions(true)).toEqual({ + directMerge: true, + deduplicateNodes: true, + deduplicateWays: true, + }); + expect(INTERSECTION_OPTIONS).toEqual({ createIntersections: true }); + }); + + it("clears the patch before verified final inspection", () => { + const transitions: string[] = []; + + finalizeVerifiedMerge( + () => transitions.push("clear-patch"), + () => transitions.push("show-final"), + ); + + expect(transitions).toEqual(["clear-patch", "show-final"]); + }); + + it("runs enabled run-all through session generation before intersections", async () => { + const calls: string[] = []; + const stages: string[] = []; + const intersections = changesetStats("base", 2); + const conflation = { + propertyKeys: ["name"], + attachNetwork: true, + maxDistanceMeters: 1, + automatic: "high-confidence" as const, + }; + const worker = { + discoverConflation: vi.fn(async () => { + calls.push("discover"); + return summary; + }), + generateConflationChangeset: vi.fn(async () => { + calls.push("generate-conflation"); + return generation; + }), + applyChangesAndReplace: vi.fn(async () => { + calls.push("apply"); + }), + generateChangeset: vi.fn(async () => { + calls.push("generate-intersections"); + return intersections; + }), + }; + + const result = await runConflationAllSteps({ + baseOsmId: "base", + conflation, + isCancelled: () => false, + onStageChange: (stage) => stages.push(stage), + patchOsmId: "patch", + worker, + }); + + expect(result).toEqual({ + generation, + intersections, + status: "completed", + summary, + }); + expect(calls).toEqual([ + "discover", + "generate-conflation", + "apply", + "generate-intersections", + "apply", + ]); + expect(stages).toEqual([ + "discover-imported-data", + "generate-verified-merge", + "apply-verified-merge", + "create-intersections", + ]); + expect(worker.discoverConflation).toHaveBeenCalledWith("base", "patch", conflation); + expect(worker.generateConflationChangeset).toHaveBeenCalledWith( + "base", + verifiedBaseMergeOptions(true), + ); + expect(worker.generateChangeset).toHaveBeenCalledWith("base", "patch", INTERSECTION_OPTIONS); + expect(worker.applyChangesAndReplace).toHaveBeenNthCalledWith(1, "base"); + expect(worker.applyChangesAndReplace).toHaveBeenNthCalledWith(2, "base"); + }); + + it("cancels enabled run-all before mutating either input", async () => { + const worker = { + discoverConflation: vi.fn(async () => summary), + generateConflationChangeset: vi.fn(async () => generation), + applyChangesAndReplace: vi.fn(async () => {}), + generateChangeset: vi.fn(async () => changesetStats("base", 0)), + }; + + const result = await runConflationAllSteps({ + baseOsmId: "base", + conflation: { + propertyKeys: ["name"], + attachNetwork: false, + automatic: "high-confidence", + }, + isCancelled: () => true, + patchOsmId: "patch", + worker, + }); + + expect(result).toEqual({ generation: null, status: "cancelled", summary }); + expect(worker.generateConflationChangeset).not.toHaveBeenCalled(); + expect(worker.applyChangesAndReplace).not.toHaveBeenCalled(); + expect(worker.generateChangeset).not.toHaveBeenCalled(); + }); + + it("returns failed conflation run-all work to a retryable review screen", async () => { + const transitions: string[] = []; + + const restoreError = await recoverConflationRunAllFailure({ + restoreReview: async () => { + transitions.push("restore-candidate-session"); + }, + showReview: () => transitions.push("show-match-imported-data"), + }); + + expect(restoreError).toBeNull(); + expect(transitions).toEqual(["restore-candidate-session", "show-match-imported-data"]); + + transitions.length = 0; + const discoveryFailure = await recoverConflationRunAllFailure({ + restoreReview: async () => { + transitions.push("restore-failed"); + throw Error("candidate discovery did not create a session"); + }, + showReview: () => transitions.push("show-match-imported-data"), + }); + + expect(discoveryFailure).toEqual({ + error: Error("candidate discovery did not create a session"), + }); + expect(transitions).toEqual(["restore-failed", "show-match-imported-data"]); + + transitions.length = 0; + await recoverConflationRunAllFailure({ + showReview: () => transitions.push("show-match-imported-data"), + }); + expect(transitions).toEqual(["show-match-imported-data"]); + }); +}); diff --git a/apps/merge/tests/merged-osm-state.test.ts b/apps/merge/tests/merged-osm-state.test.ts new file mode 100644 index 00000000..eb252a9e --- /dev/null +++ b/apps/merge/tests/merged-osm-state.test.ts @@ -0,0 +1,112 @@ +import type { Osm, OsmInfo } from "osmix"; +import { describe, expect, it, vi } from "vitest"; + +import { prepareMergedOsmState } from "../src/lib/merged-osm-state"; + +const info = (id: string): OsmInfo => ({ + bbox: [7.4, 43.7, 7.5, 43.8], + header: {}, + id, + spatialIndexes: { + nodes: { all: true, tagged: true }, + ways: true, + }, + stats: { nodes: 10, relations: 2, ways: 3 }, +}); + +const osm = (id: string, contentHash: string, equal = false) => + ({ + contentHash: () => contentHash, + id, + info: () => info(id), + isEqual: () => equal, + }) as unknown as Osm; + +describe("merged OSM state", () => { + it("re-fetches a renamed dataset and returns content-addressed metadata", async () => { + const beforeRename = osm("base", "merged-hash"); + const afterRename = osm("merged-hash", "merged-hash"); + const registry = new Map<string, Osm>([["base", beforeRename]]); + const get = vi.fn(async (id: string) => { + const registered = registry.get(id); + if (!registered) throw Error(`Missing OSM ${id}`); + return registered; + }); + const rename = vi.fn(async (fromId: string, toId: string) => { + if (!registry.delete(fromId)) throw Error(`Missing OSM ${fromId}`); + registry.set(toId, afterRename); + }); + + const result = await prepareMergedOsmState({ + currentFileInfo: { + fileHash: "base", + fileName: "base.pbf", + fileSize: 1, + }, + currentOsm: osm("base", "base"), + mergedFileName: "merged.pbf", + newOsmId: "base", + worker: { get, rename }, + }); + + expect(rename).toHaveBeenCalledWith("base", "merged-hash"); + expect(get).toHaveBeenNthCalledWith(1, "base"); + expect(get).toHaveBeenNthCalledWith(2, "merged-hash"); + expect(await get(result.osm.id)).toBe(afterRename); + expect(registry.has("base")).toBe(false); + expect(result).toEqual({ + fileInfo: { + fileHash: "merged-hash", + fileName: "merged.pbf", + fileSize: 900, + }, + kind: "changed", + osm: afterRename, + osmInfo: info("merged-hash"), + }); + }); + + it("keeps source metadata when the applied changeset does not change content", async () => { + const unchanged = osm("base", "base", true); + const get = vi.fn<(id: string) => Promise<Osm>>().mockResolvedValue(unchanged); + const rename = vi.fn<(fromId: string, toId: string) => Promise<void>>(); + + const result = await prepareMergedOsmState({ + currentFileInfo: { + fileHash: "base", + fileName: "base.pbf", + fileSize: 1, + }, + currentOsm: osm("base", "base"), + newOsmId: "base", + worker: { get, rename }, + }); + + expect(result).toEqual({ kind: "unchanged", osm: unchanged, osmInfo: info("base") }); + expect(rename).not.toHaveBeenCalled(); + expect(get).toHaveBeenCalledOnce(); + }); + + it("does not rename a dataset that already uses its content hash", async () => { + const merged = osm("merged-hash", "merged-hash"); + const get = vi.fn<(id: string) => Promise<Osm>>().mockResolvedValue(merged); + const rename = vi.fn<(fromId: string, toId: string) => Promise<void>>(); + + const result = await prepareMergedOsmState({ + currentFileInfo: null, + currentOsm: null, + newOsmId: "merged-hash", + now: new Date("2026-07-21T01:02:03Z"), + worker: { get, rename }, + }); + + expect(result.kind).toBe("changed"); + if (result.kind === "changed") { + expect(result.fileInfo.fileName).toBe("osmix-merged-2026-07-21T01-02-03.pbf"); + expect(result.osm.id).toBe("merged-hash"); + expect(result.osmInfo.id).toBe("merged-hash"); + } + expect(rename).not.toHaveBeenCalled(); + expect(get).toHaveBeenCalledOnce(); + }); +}); diff --git a/apps/merge/tests/osmix-raster-source.test.ts b/apps/merge/tests/osmix-raster-source.test.ts new file mode 100644 index 00000000..322e6be9 --- /dev/null +++ b/apps/merge/tests/osmix-raster-source.test.ts @@ -0,0 +1,48 @@ +import { Osm } from "osmix"; +import { describe, expect, it, vi } from "vitest"; + +vi.mock("../src/state/worker", () => ({ osmWorker: {} })); + +import { OsmixMapSources } from "../src/components/osmix-map-sources"; +import OsmixRasterSource from "../src/components/osmix-raster-source"; + +function childKeys(element: ReturnType<typeof OsmixMapSources>) { + return (element.props.children as React.ReactElement[]).filter(Boolean).map((child) => child.key); +} + +describe("Osmix map sources", () => { + it("remounts when a merge replaces the dataset ID", () => { + const beforeMerge = OsmixRasterSource({ osmId: "yakima-base", tileSize: 512 }); + const afterMerge = OsmixRasterSource({ osmId: "yakima-merged", tileSize: 512 }); + + expect(beforeMerge.key).toBe(beforeMerge.props.id); + expect(afterMerge.key).toBe(afterMerge.props.id); + expect(afterMerge.key).not.toBe(beforeMerge.key); + }); + + it("replaces base and patch source wrappers after a merge", () => { + const beforeMerge = OsmixMapSources({ + activeTab: "Merge", + baseOsm: new Osm({ id: "yakima-base" }), + extractOsm: null, + patchOsm: new Osm({ id: "yakima-osw" }), + }); + const afterMerge = OsmixMapSources({ + activeTab: "Merge", + baseOsm: new Osm({ id: "yakima-merged" }), + extractOsm: null, + patchOsm: null, + }); + + expect(childKeys(beforeMerge)).toEqual([ + "base:raster:yakima-base", + "patch:raster:yakima-osw", + "base:overlay:yakima-base", + "patch:overlay:yakima-osw", + ]); + expect(childKeys(afterMerge)).toEqual([ + "base:raster:yakima-merged", + "base:overlay:yakima-merged", + ]); + }); +}); diff --git a/packages/change/README.md b/packages/change/README.md index ad76e53a..7abc68bd 100644 --- a/packages/change/README.md +++ b/packages/change/README.md @@ -6,7 +6,7 @@ - Construct repeatable `OsmChangeset`s that track creates, modifies, and deletes with origin metadata and per-entity refs. - **Augmented diffs**: Automatically captures both old and new entity states for modifications and deletions, following the [Overpass API Augmented Diffs](https://wiki.openstreetmap.org/wiki/Overpass_API/Augmented_Diffs) format. -- Deduplicate coincident nodes or overlapping ways, replace references, and optionally create intersection points where geometry meets. +- Conservatively reconcile compatible nodes or overlapping ways, replace references, and optionally create intersection points where geometry meets. - Generate summary stats and OSC-friendly XML fragments so downstream systems can audit each change step. - Run `merge(base, patch, options)` to execute the full dedupe/merge workflow with a single call. - Export lightweight utilities for measuring distances, pruning duplicate refs, and deciding when ways should connect. @@ -30,8 +30,6 @@ const base = await fromPbf(monacoPbf); const patch = await fromPbf(patchPbf); const changeset = new OsmChangeset(base); -changeset.deduplicateNodes(base.nodes); -changeset.deduplicateWays(base.ways); changeset.generateDirectChanges(patch); console.log(changeStatsSummary(changeset.stats)); @@ -40,7 +38,11 @@ const merged = applyChangesetToOsm(changeset); console.log(merged.id); ``` -`OsmChangeset` keeps track of creates/modifies/deletes per entity type. Call the helpers (`deduplicateNodes`, `deduplicateWays`, `generateDirectChanges`, `createIntersectionsForWays`, etc.) in whatever order your workflow requires, then use `applyChangesetToOsm()` to produce a new `Osm` instance with the edits applied. +`OsmChangeset` keeps track of creates/modifies/deletes per entity type. Prefer `merge()` for the complete +pipeline. When composing it manually, generate direct changes and then reconcile patch nodes before patch +ways in one changeset rooted in the original base. Apply that changeset before creating intersections so the +new patch ways are present in the rebuilt spatial index. For that reason, `generateChangeset()` rejects +`directMerge: true` combined with `createIntersections: true`; use `merge()` for the staged pipeline. ### Run the bundled merge pipeline @@ -56,7 +58,60 @@ const combined = await merge(base, patch, { console.log(combined.id); ``` -`merge` wraps a sequence of changesets that deduplicate each dataset, optionally create intersections, and (when `directMerge` is true) generate modifications that reconcile the patch into the base. All options default to `false`, so you can enable only the stages you need. +`merge` preserves the two source datasets and uses deduplication only to reconcile compatible patch entities +with the base. It optionally creates intersections and, when `directMerge` is true, generates modifications +that merge the patch into the base. All options default to `false`, so you can enable only the stages you need. +An empty patch is therefore an identity operation; the high-level pipeline does not normalize either input as +a hidden preliminary step. + +### Match imported data within one meter + +Exact reconciliation remains the default. For imported GeoJSON, Shapefile, OSW, or other independently +created data, opt into proximity conflation with explicit property keys and an explicit network-attachment +choice. The historical radius is one meter unless `maxDistanceMeters` is supplied. + +```ts check-docs change-context +import { + applyChangesetToOsm, + discoverConflationCandidates, + generateConflationChangeset, +} from "osmix"; + +const conflation = { + propertyKeys: ["name", "operator", "surface"], + attachNetwork: true, +}; +const discovery = discoverConflationCandidates(base, patch, conflation); + +// Review discovery.candidates and persist decisions by stable candidate ID. +const decisions = discovery.candidates + .filter((candidate) => candidate.status === "review") + .map((candidate) => ({ candidateId: candidate.id, action: "reject" as const })); + +const changeset = generateConflationChangeset( + base, + patch, + { + directMerge: true, + deduplicateNodes: true, + deduplicateWays: true, + conflation, + }, + decisions, + discovery, +); +const conflated = applyChangesetToOsm(changeset); +``` + +Discovery compares only the untouched patch with the immutable original base. High-confidence candidates +apply automatically by default; set `automatic: "none"` when every match should require a decision. Property +transfer changes only selected tags on the base entity. Network attachment changes only patch-created way +references. Base IDs, coordinates, ordered way references, and ordered relation members stay authoritative. + +Structural properties cannot transfer. Routing-affecting properties, motor-road attachments, ambiguous +targets, relation membership, and uncertain geometry require review. Grade conflicts, restrictions, dangling +references, and way collapse remain blocked even when an accept decision is supplied. Equivalent one-to-one +patch ways may be suppressed after property transfer; segmented way chains are reported but unsupported. ## API @@ -72,8 +127,8 @@ constructor(base: Osm) #### Core methods -- `deduplicateNodes(nodes: Nodes)`: Check a set of nodes (usually from the base or patch) for duplicates against the base dataset. Deletes duplicates and maps their IDs to the surviving node. -- `deduplicateWays(ways: Ways)`: Check a set of ways for geometric duplicates. Deletes duplicates and preserves the one with more tags/metadata. +- `deduplicateNodes(nodes: Nodes)`: Check candidate nodes (normally from a patch) against the base dataset and map safe duplicates to the surviving base node. Proximity alone is not sufficient. +- `deduplicateWays(ways: Ways)`: Check candidate ways against the base and reconcile only matching geometry with compatible routing and grade-separation tags. - `generateDirectChanges(patch: Osm)`: Merge a patch dataset into the changeset. Handles creates and updates. - `createIntersectionsForWays(ways: Ways)`: Checks provided ways for intersections with existing ways in the base dataset. Splits ways and inserts nodes where they cross. - `applyNodeReplacementsToWays(replacementMap)`: Updates way references based on a map of replaced node IDs (generated by `deduplicateNodes`). @@ -86,13 +141,32 @@ High-level pipeline to merge `patch` into `base`. Returns a new `Osm` instance. Options: - `directMerge` (boolean): Apply creates/updates from patch. -- `deduplicateNodes` (boolean): Run node deduplication. -- `deduplicateWays` (boolean): Run way deduplication. +- `deduplicateNodes` (boolean): Reconcile compatible patch nodes with unique base matches. +- `deduplicateWays` (boolean): Reconcile compatible patch ways with matching base geometry. - `createIntersections` (boolean): Split intersecting ways. +- `conflation` (optional): Explicit imported-data matching configuration. `propertyKeys` and + `attachNetwork` are required when supplied; `maxDistanceMeters` defaults to `1`, and `automatic` defaults + to `"high-confidence"`. + +### Conflation discovery and generation + +- `discoverConflationCandidates(base, patch, options)`: Return deterministic node and one-to-one-way + candidates with action-specific status, evidence, tag diffs, and reason codes. +- `filterConflationCandidates(candidates, filter, decisions?)`: Filter discovery rows without rerunning the + spatial search. +- `summarizeConflationCandidates(candidates, decisions?)`: Count automatic, review, blocked, unmatched, and + rejected rows. +- `generateConflationChangeset(base, patch, mergeOptions, decisions?, discovery?)`: Generate one cumulative + direct, exact, and fuzzy changeset from untouched inputs. +- `generateConflationApplicationChangeset(baseline, patch, discovery, originalBase, decisions?)`: Apply only + reviewed fuzzy actions to an already materialized ordinary-merge baseline. The immutable original base is + required so generation can rediscover and validate candidates instead of trusting mutable review records. ### `applyChangesetToOsm(changeset: OsmChangeset): Osm` Applies all pending changes in the changeset to produce a **new** `Osm` instance. The original `base` is immutable. +Application rejects new dangling references, degenerate highways, and detached turn-restriction topology +before returning the result. ### Augmented Diffs @@ -145,6 +219,11 @@ Options: - Requires runtimes compatible with `@osmix/core` (Node 20+, Bun, or modern browsers) since the same typed-array data structures are used. - Deduplication helpers assume datasets store dense node blocks and rely on spatial indexes built via `Osm.buildIndexes()`. - Intersections are generated only for highway/footway-style features; polygonal ways are ignored. +- A scan that compares a dataset with itself is useful for diagnostics, but its proposed proximity matches + should not be applied automatically. Use the high-level cross-dataset merge for reconciliation. +- PBFs produced by older Osmix versions may already contain topology changes caused by automatic + within-input deduplication. Those files cannot be repaired reliably without their source inputs and should + be regenerated from the original base and patch files. ## Development diff --git a/packages/change/src/apply-changeset.ts b/packages/change/src/apply-changeset.ts index fedc5a9b..11024ffa 100644 --- a/packages/change/src/apply-changeset.ts +++ b/packages/change/src/apply-changeset.ts @@ -10,6 +10,22 @@ import { Osm } from "@osmix/core"; import type { OsmChangeset } from "./changeset.ts"; +import { assertNoNewRoutingIntegrityIssues, reuseRoutingIntegrityAnalysis } from "./integrity.ts"; + +function hasOwnChanges(changes: Record<number, unknown>) { + for (const key in changes) { + if (Object.hasOwn(changes, key)) return true; + } + return false; +} + +function isEmptyChangeset(changeset: OsmChangeset) { + return ( + !hasOwnChanges(changeset.nodeChanges) && + !hasOwnChanges(changeset.wayChanges) && + !hasOwnChanges(changeset.relationChanges) + ); +} /** * Apply a changeset to an Osm index, producing a new Osm index. @@ -30,12 +46,29 @@ import type { OsmChangeset } from "./changeset.ts"; * @example * ```ts * const changeset = new OsmChangeset(baseOsm) - * changeset.deduplicateNodes(baseOsm.nodes) + * changeset.generateDirectChanges(patchOsm) + * changeset.deduplicateNodes(patchOsm.nodes) * const newOsm = applyChangesetToOsm(changeset) * ``` */ export function applyChangesetToOsm(changeset: OsmChangeset, newOsmId?: string) { const baseOsm = changeset.osm; + if (isEmptyChangeset(changeset)) { + // Keep the documented fresh-result behavior while reusing finalized, + // immutable typed buffers and spatial indexes for a true no-op. This is + // important for empty-patch identity merges on large base datasets. + const osm = new Osm({ + ...baseOsm.transferables(), + id: newOsmId ?? baseOsm.id, + }); + if (!osm.hasSpatialIndexes()) osm.buildSpatialIndexes(); + // The wrapper above references the exact same finalized entity buffers. + // Carry the source analysis forward so the next merge stage can reuse it. + reuseRoutingIntegrityAnalysis(baseOsm, osm); + assertNoNewRoutingIntegrityIssues(changeset.routingIntegrityBaselineKeys, osm); + return osm; + } + const osm = new Osm({ id: newOsmId ?? baseOsm.id, header: baseOsm.header, @@ -43,17 +76,10 @@ export function applyChangesetToOsm(changeset: OsmChangeset, newOsmId?: string) const { nodeChanges, wayChanges, relationChanges } = changeset; - // Work on shallow copies so applying a changeset never consumes its change - // records. The change entities are only read while indexing them below. - const pendingNodeChanges = { ...nodeChanges }; - const pendingWayChanges = { ...wayChanges }; - const pendingRelationChanges = { ...relationChanges }; - // Add nodes from base, modifying and deleting as needed for (const node of baseOsm.nodes) { const change = nodeChanges[node.id]; if (change) { - delete pendingNodeChanges[node.id]; if (change.changeType === "delete") continue; // Don't add deleted nodes if (change.changeType === "create") throw Error("Changeset contains create changes for existing entities"); @@ -63,18 +89,22 @@ export function applyChangesetToOsm(changeset: OsmChangeset, newOsmId?: string) // All remaining node changes should be create // Add nodes from patch - for (const change of Object.values(pendingNodeChanges)) { - if (change.changeType !== "create") { + for (const idText in nodeChanges) { + if (!Object.hasOwn(nodeChanges, idText)) continue; + const change = nodeChanges[Number(idText)]!; + if (change.changeType === "create") { + osm.nodes.addNode(change.entity); + continue; + } + if (!baseOsm.nodes.ids.has(Number(idText))) { throw Error("Changeset still contains node changes in incorrect stage."); } - osm.nodes.addNode(change.entity); } // Add ways from base, modifying and deleting as needed for (const way of baseOsm.ways) { const change = wayChanges[way.id]; if (change) { - delete pendingWayChanges[way.id]; if (change.changeType === "delete") continue; // Don't add deleted ways if (change.changeType === "create") { throw Error("Changeset contains create changes for existing entities"); @@ -86,17 +116,22 @@ export function applyChangesetToOsm(changeset: OsmChangeset, newOsmId?: string) // All remaining way changes should be create // Add ways from patch - for (const change of Object.values(pendingWayChanges)) { - if (change.changeType !== "create") + for (const idText in wayChanges) { + if (!Object.hasOwn(wayChanges, idText)) continue; + const change = wayChanges[Number(idText)]!; + if (change.changeType === "create") { + osm.ways.addWay(change.entity); + continue; + } + if (!baseOsm.ways.ids.has(Number(idText))) { throw Error("Changeset still contains way changes in incorrect stage."); - osm.ways.addWay(change.entity); + } } // Add relations from base, modifying and deleting as needed for (const relation of baseOsm.relations) { const change = relationChanges[relation.id]; if (change) { - delete pendingRelationChanges[relation.id]; if (change.changeType === "delete") continue; // Don't add deleted relations if (change.changeType === "create") { throw Error("Changeset contains create changes for existing entities"); @@ -106,10 +141,16 @@ export function applyChangesetToOsm(changeset: OsmChangeset, newOsmId?: string) } // Add relations from patch - for (const change of Object.values(pendingRelationChanges)) { - if (change.changeType !== "create") + for (const idText in relationChanges) { + if (!Object.hasOwn(relationChanges, idText)) continue; + const change = relationChanges[Number(idText)]!; + if (change.changeType === "create") { + osm.relations.addRelation(change.entity); + continue; + } + if (!baseOsm.relations.ids.has(Number(idText))) { throw Error("Changeset still contains relation changes in incorrect stage."); - osm.relations.addRelation(change.entity); + } } // Everything should be added now, finish the osm @@ -118,5 +159,7 @@ export function applyChangesetToOsm(changeset: OsmChangeset, newOsmId?: string) // Build spatial indexes osm.buildSpatialIndexes(); + assertNoNewRoutingIntegrityIssues(changeset.routingIntegrityBaselineKeys, osm); + return osm; } diff --git a/packages/change/src/changeset.ts b/packages/change/src/changeset.ts index 1d75b93d..15c6ba1a 100644 --- a/packages/change/src/changeset.ts +++ b/packages/change/src/changeset.ts @@ -9,27 +9,214 @@ */ import type { IdOrIndex, Nodes, Osm, Ways } from "@osmix/core"; +import { toMicroDegrees } from "@osmix/geo/coordinates"; import type { OsmEntity, OsmEntityType, OsmEntityTypeMap, OsmNode, OsmWay } from "@osmix/types"; -import { entityPropertiesEqual, getEntityType, isWayEqual } from "@osmix/types/utils"; +import { entityPropertiesEqual, getEntityType } from "@osmix/types/utils"; import { dequal } from "dequal"; // dequal/lite does not work with `TypedArray`s +import { inheritedRoutingIntegrityIssueKeys, routingIntegrityIssueKeys } from "./integrity.ts"; import type { OsmChange, OsmChanges, OsmChangesetStats, OsmEntityRef } from "./types.ts"; import { + areWayTagsIntersectionCandidate, cleanCoords, entityHasTagValue, - getEntityVersion, - isWayIntersectionCandidate, nearestNodeOnWay, removeDuplicateAdjacentRelationMembers, removeDuplicateAdjacentWayRefs, + routingGradeSignature, waysIntersect, waysShouldConnect, } from "./utils.ts"; type ReplacementMap = Map<number, number>; type IdIndex = Nodes["ids"]; +type ExactWayIndex = Map<number, number | number[]>; +type WaysByNode = ReadonlyMap<number, readonly OsmWay[]>; + +interface NodeCandidate { + baseNodes: OsmNode[]; + patchNode: OsmNode; +} + +interface WayCoordinateCacheEntry { + cleaned?: [number, number][]; + coordinates: [number, number][] | null; + nodeCoordinateRevision: number; + wayRevision: number; +} + +interface IntersectionMetadata { + eligible: Uint8Array; + gradeIds: Int32Array; +} const EMPTY_ID = -1; +const DESCRIPTIVE_WAY_TAGS = new Set([ + "alt_name", + "int_name", + "loc_name", + "name", + "note", + "official_name", + "old_name", + "operator", + "ref", + "short_name", + "source", + "wikidata", + "wikipedia", +]); +const DESCRIPTIVE_WAY_TAG_PREFIXES = [ + "alt_name:", + "name:", + "note:", + "official_name:", + "old_name:", + "operator:", + "source:", +] as const; +const GRADE_AND_ACCESS_TAGS = [ + "access", + "barrier", + "bicycle", + "foot", + "horse", + "motor_vehicle", + "motorcar", + "vehicle", +] as const; +const GRADE_TAG_DEFAULTS = { + bridge: "no", + covered: "no", + layer: "0", + level: "", + tunnel: "no", +} as const; +const NODE_ROUTING_CRITICAL_TAGS = [ + "access", + "barrier", + "bicycle", + "foot", + "ford", + "highway", + "horse", + "motor_vehicle", + "motorcar", + "vehicle", +] as const; + +function sameOsmCoordinate(a: OsmNode, b: OsmNode) { + return ( + toMicroDegrees(a.lon) === toMicroDegrees(b.lon) && + toMicroDegrees(a.lat) === toMicroDegrees(b.lat) + ); +} + +function hasAnyTagConflict(a: OsmEntity["tags"], b: OsmEntity["tags"]) { + if (!a || !b) return false; + return Object.entries(a).some(([key, value]) => b[key] != null && b[key] !== value); +} + +function isDescriptiveWayTag(key: string) { + return ( + DESCRIPTIVE_WAY_TAGS.has(key) || + DESCRIPTIVE_WAY_TAG_PREFIXES.some((prefix) => key.startsWith(prefix)) + ); +} + +function routingSemanticTagsEqual(a: OsmEntity["tags"], b: OsmEntity["tags"]) { + const keys = new Set([...Object.keys(a ?? {}), ...Object.keys(b ?? {})]); + return [...keys].every((key) => isDescriptiveWayTag(key) || a?.[key] === b?.[key]); +} + +function hashText(hash: number, value: string) { + let nextHash = hash; + for (let index = 0; index < value.length; index++) { + nextHash ^= value.charCodeAt(index); + nextHash = Math.imul(nextHash, 16_777_619); + } + return nextHash >>> 0; +} + +/** + * Produce a compact lookup key for exact way reconciliation. Hash collisions are + * expected and harmless because candidates still pass the complete refs and tag + * predicates before they can be accepted. + */ +function exactWayHash(way: OsmWay) { + let hash = 2_166_136_261; + hash = hashText(hash, `${way.refs.length}:`); + for (const ref of way.refs) hash = hashText(hash, `${ref},`); + for (const [key, value] of Object.entries(way.tags ?? {}).toSorted(([a], [b]) => + a < b ? -1 : a > b ? 1 : 0, + )) { + if (isDescriptiveWayTag(key)) continue; + hash = hashText(hash, `${key.length}:${key}${String(value).length}:${String(value)}`); + } + return hash; +} + +function hasConflictingGradeOrAccessTags(a: OsmEntity["tags"], b: OsmEntity["tags"]) { + if (GRADE_AND_ACCESS_TAGS.some((key) => String(a?.[key] ?? "") !== String(b?.[key] ?? ""))) { + return true; + } + return Object.entries(GRADE_TAG_DEFAULTS).some( + ([key, defaultValue]) => String(a?.[key] ?? defaultValue) !== String(b?.[key] ?? defaultValue), + ); +} + +function withNonConflictingTags<T extends OsmEntity>(base: T, patch: T): T { + if (!patch.tags) return base; + const tags = { ...base.tags }; + let changed = false; + for (const [key, value] of Object.entries(patch.tags)) { + if (tags[key] != null) continue; + tags[key] = value; + changed = true; + } + return changed ? { ...base, tags } : base; +} + +function withNonConflictingDescriptiveTags<T extends OsmEntity>(base: T, patch: T): T { + if (!patch.tags) return base; + const tags = { ...base.tags }; + let changed = false; + for (const [key, value] of Object.entries(patch.tags)) { + if (!isDescriptiveWayTag(key) || tags[key] != null) continue; + tags[key] = value; + changed = true; + } + return changed ? { ...base, tags } : base; +} + +function nodeRoutingTagCount(node: OsmNode) { + return NODE_ROUTING_CRITICAL_TAGS.reduce( + (count, key) => count + (node.tags?.[key] == null ? 0 : 1), + 0, + ); +} + +function wayBbox(coordinates: [number, number][]): [number, number, number, number] { + let minLon = Number.POSITIVE_INFINITY; + let minLat = Number.POSITIVE_INFINITY; + let maxLon = Number.NEGATIVE_INFINITY; + let maxLat = Number.NEGATIVE_INFINITY; + for (const [lon, lat] of coordinates) { + minLon = Math.min(minLon, lon); + minLat = Math.min(minLat, lat); + maxLon = Math.max(maxLon, lon); + maxLat = Math.max(maxLat, lat); + } + return [minLon, minLat, maxLon, maxLat]; +} + +function bboxesIntersect( + a: readonly [number, number, number, number], + b: readonly [number, number, number, number], +) { + if (a[0] > a[2] || a[1] > a[3] || b[0] > b[2] || b[1] > b[3]) return false; + return a[0] <= b[2] && a[2] >= b[0] && a[1] <= b[3] && a[3] >= b[1]; +} /** Return the true maximum ID regardless of insertion order. */ function maximumId(ids: IdIndex): number | null { @@ -88,6 +275,8 @@ export class OsmChangeset { relationChanges: Record<number, OsmChange<OsmEntityTypeMap["relation"]>> = {}; osm: Osm; + /** @internal Integrity issues inherited from merge inputs rather than introduced by changes. */ + routingIntegrityBaselineKeys: Set<string>; // Next node ID tracker for generating new IDs during intersection creation currentNodeId: number; @@ -98,17 +287,26 @@ export class OsmChangeset { intersectionPointsFound = 0; intersectionNodesCreated = 0; + /** Revisions keep geometry caches correct while intersections rewrite ways in place. */ + private nodeCoordinateRevision = 0; + private readonly wayGeometryRevisions = new Map<number, number>(); + private readonly wayCoordinateCache = new Map<number, WayCoordinateCacheEntry>(); + static fromJson(base: Osm, json: OsmChanges) { const changeset = new OsmChangeset(base); changeset.nodeChanges = json.nodes; changeset.wayChanges = json.ways; changeset.relationChanges = json.relations; + // Serialized node changes may move, delete, or supply a previously missing + // ref. Conservatively disable packed base-coordinate reuse for this instance. + if (Object.keys(json.nodes).length > 0) changeset.nodeCoordinateRevision++; return changeset; } constructor(base: Osm) { this.osm = base; this.currentNodeId = maximumId(base.nodes.ids) ?? EMPTY_ID; + this.routingIntegrityBaselineKeys = routingIntegrityIssueKeys(base); } get stats(): OsmChangesetStats { @@ -156,6 +354,13 @@ export class OsmChangeset { } create(entity: OsmEntity, osmId: string, refs?: OsmEntityRef[]) { + this.recordCreate(entity, osmId, refs); + const type = getEntityType(entity); + if (type === "node") this.nodeCoordinateRevision++; + if (type === "way") this.invalidateWayGeometry(entity.id); + } + + private recordCreate(entity: OsmEntity, osmId: string, refs?: OsmEntityRef[]) { this.changes(getEntityType(entity))[entity.id] = { changeType: "create", entity, @@ -191,12 +396,25 @@ export class OsmChangeset { // If we already have a change, preserve the original oldEntity. const oldEntity = change?.oldEntity ?? (changeEntity ? undefined : existingEntity); + const modifiedEntity = modify(existingEntity); changes[id] = { changeType: change?.changeType ?? "modify", - entity: modify(existingEntity), + entity: modifiedEntity, osmId: this.osm.id, // If we're modifying an entity, it must exist in the base OSM oldEntity, }; + + if (type === "node") { + const previous = existingEntity as OsmNode; + const next = modifiedEntity as OsmNode; + if (previous.lon !== next.lon || previous.lat !== next.lat) { + this.nodeCoordinateRevision++; + } + } else if (type === "way") { + const previous = existingEntity as OsmWay; + const next = modifiedEntity as OsmWay; + if (!dequal(previous.refs, next.refs)) this.invalidateWayGeometry(id); + } } getEntity<T extends OsmEntityType>(type: T, id: number): OsmEntityTypeMap[T] | undefined { @@ -219,94 +437,182 @@ export class OsmChangeset { osmId: this.osm.id, oldEntity: entity, // For augmented diffs: capture the entity being deleted }; + const type = getEntityType(entity); + if (type === "node") this.nodeCoordinateRevision++; + if (type === "way") this.invalidateWayGeometry(entity.id); + } + + private invalidateWayGeometry(wayId: number) { + this.wayGeometryRevisions.set(wayId, (this.wayGeometryRevisions.get(wayId) ?? 0) + 1); + this.wayCoordinateCache.delete(wayId); + } + + private *currentWays() { + for (const way of this.osm.ways) { + const current = this.getCurrentWay(way); + if (current) yield current; + } + for (const change of Object.values(this.wayChanges)) { + // Existing IDs were already yielded in base insertion order. Object-key + // order matches the former Map append order for patch-created entities. + if (this.osm.ways.ids.has(change.entity.id) || change.changeType === "delete") continue; + yield change.entity; + } + } + + private *currentRelations() { + for (const relation of this.osm.relations) { + const change = this.relationChanges[relation.id]; + if (change?.changeType === "delete") continue; + yield change?.entity ?? relation; + } + for (const change of Object.values(this.relationChanges)) { + if (this.osm.relations.ids.has(change.entity.id) || change.changeType === "delete") continue; + yield change.entity; + } + } + + private nodeContextsCompatible(patchNode: OsmNode, baseNode: OsmNode, waysByNode: WaysByNode) { + if (hasAnyTagConflict(patchNode.tags, baseNode.tags)) return false; + if (hasConflictingGradeOrAccessTags(patchNode.tags, baseNode.tags)) return false; + + const patchWays = waysByNode.get(patchNode.id) ?? []; + const baseWays = waysByNode.get(baseNode.id) ?? []; + if (patchWays.length === 0 || baseWays.length === 0) return true; + + return patchWays.every((patchWay) => + baseWays.every( + (baseWay) => + !hasConflictingGradeOrAccessTags(patchWay.tags, baseWay.tags) && + (patchWay.tags?.["highway"] == null) === (baseWay.tags?.["highway"] == null), + ), + ); } /** - * Check nodes for duplicates and consolidate them within this OSM dataset. - * This process helps merge disparate datasets that share common geometry. - * - * The algorithm: - * 1. Find all pairs of nodes at the same geographic location (within a tiny radius). - * 2. For each pair, determine which node to keep: - * - Prefer higher version number. - * - If versions are equal, prefer the node with more tags. - * - If tags are equal, prefer the higher ID (deterministic tie-breaker). - * 3. Build a replacement map (deleted ID -> kept ID). - * 4. Flatten chains (e.g., if A->B and B->C, then A->C). - * 5. Schedule duplicate nodes for deletion. - * 6. Update all ways and relations to reference the kept nodes. + * Build incident-way context in one pass, but only for nodes that already have + * an exact-coordinate candidate. This replaces a full way scan per candidate. */ - deduplicateNodes(nodes: Nodes) { - const checkedIdPairs = new IdPairs(); - const replacementMap = new Map<number, number>(); - - // Find overlapping nodes and determine which to keep - for (const node of nodes) { - if (!this.osm.nodes.ids.has(node.id)) continue; - if (this.nodeChanges[node.id]?.changeType === "delete") continue; - - // Use a tiny radius (1 meter = 0.001 km) to find nodes at effectively the same location - const existingNodes = this.osm.nodes.findIndexesWithinRadius(node.lon, node.lat, 0.001); - const existingNodeIds = existingNodes - .map((index) => ({ id: this.osm.nodes.ids.at(index), index })) - .filter((n) => n.id !== node.id && !checkedIdPairs.has(n.id, node.id)); - - for (const { index: existingNodeIndex } of existingNodeIds) { - const existingNode = this.osm.nodes.getByIndex(existingNodeIndex); - if (existingNode == null) continue; - - checkedIdPairs.add(existingNode.id, node.id); - - // Determine which node to keep using version/tags logic (same as deduplicateWay) - const nodeVersion = getEntityVersion(node); - const existingNodeVersion = getEntityVersion(existingNode); - - let nodeToKeep: number = node.id; - let nodeToDelete: number = existingNode.id; - - // Check version - prefer higher version - if (existingNodeVersion > nodeVersion) { - // Existing node has higher version, keep existing node - nodeToKeep = existingNode.id; - nodeToDelete = node.id; - } else if (nodeVersion === existingNodeVersion) { - // Same version, keep node with more tags (>= comparison to match deduplicateWay) - const nodeTagCount = Object.keys(node.tags ?? {}).length; - const existingNodeTagCount = Object.keys(existingNode.tags ?? {}).length; - if (existingNodeTagCount >= nodeTagCount) { - // Existing node has same or more tags, keep existing node - // If equal tags, use higher ID for normalization - if (existingNodeTagCount === nodeTagCount) { - nodeToKeep = Math.max(node.id, existingNode.id); - nodeToDelete = Math.min(node.id, existingNode.id); - } else { - nodeToKeep = existingNode.id; - nodeToDelete = node.id; - } - } - } + private currentWaysByNode(nodeIds: ReadonlySet<number>): WaysByNode { + const waysByNode = new Map<number, OsmWay[]>(); + for (const nodeId of nodeIds) waysByNode.set(nodeId, []); + if (waysByNode.size === 0) return waysByNode; + + for (const way of this.currentWays()) { + let matchedRefs: Set<number> | undefined; + for (const ref of way.refs) { + const incidentWays = waysByNode.get(ref); + if (!incidentWays || matchedRefs?.has(ref)) continue; + incidentWays.push(way); + (matchedRefs ??= new Set()).add(ref); + } + } + return waysByNode; + } - // Add to replacement map (deleted node -> kept node) - replacementMap.set(nodeToDelete, nodeToKeep); + private removeUnsafeNodeReplacements(replacementMap: ReplacementMap) { + if (replacementMap.size === 0) return; + let changed = true; + while (changed) { + changed = false; + for (const way of this.currentWays()) { + if (way.tags?.["highway"] == null || new Set(way.refs).size < 2) continue; + const replacedRefs = way.refs.map((ref) => replacementMap.get(ref) ?? ref); + if (new Set(replacedRefs).size >= 2) continue; + for (const ref of way.refs) { + if (!replacementMap.delete(ref)) continue; + changed = true; + } } } + } - // Flatten deletion chains before updating higher-layer references. - const flattenedMap = flattenReplacementMap(replacementMap); - this.applyNodeReplacementsToWays(flattenedMap); - this.applyNodeReplacementsToRelations(flattenedMap); + private reconcileNodeTags(patchNode: OsmNode, baseNodeId: number) { + const baseNode = this.getCurrentNode(baseNodeId); + if (!baseNode) return; + const mergedNode = withNonConflictingTags(baseNode, patchNode); + if (mergedNode !== baseNode) this.modify("node", baseNodeId, () => mergedNode); + } - // Schedule nodes for deletion only after all references are updated. - for (const fromId of flattenedMap.keys()) { - const nodeToDelete = this.osm.nodes.getById(fromId); - if (nodeToDelete) { - this.deduplicatedNodes++; - this.delete(nodeToDelete, [ - { type: "node", id: flattenedMap.get(fromId)!, osmId: this.osm.id }, - ]); - } + private deleteReconciledNode(node: OsmNode, survivorId: number) { + const pendingChange = this.nodeChanges[node.id]; + if (pendingChange?.changeType === "create") { + delete this.nodeChanges[node.id]; + } else { + const storedNode = this.osm.nodes.getById(node.id); + if (!storedNode) return; + this.delete(storedNode, [{ type: "node", id: survivorId, osmId: this.osm.id }]); } - return flattenedMap; + this.deduplicatedNodes++; + } + + /** + * Reconcile incoming nodes with unambiguous base nodes at the exact OSM coordinate. + * Cross-dataset reconciliation always preserves the base ID. Same-dataset diagnostic + * scans use the highest compatible ID as a deterministic candidate survivor. + */ + deduplicateNodes(nodes: Nodes) { + const sameDataset = nodes === this.osm.nodes; + const replacementMap: ReplacementMap = new Map(); + const exactCandidates: NodeCandidate[] = []; + const contextNodeIds = new Set<number>(); + + for (const patchNode of nodes) { + if (this.nodeChanges[patchNode.id]?.changeType === "delete") continue; + if (!sameDataset && this.nodeChanges[patchNode.id]?.changeType !== "create") continue; + const currentPatchNode = this.getCurrentNode(patchNode.id); + if (!currentPatchNode) continue; + + // Exact reconciliation only accepts equality at OSM's seven-decimal storage + // precision. Query that exact coordinate rather than calculating and sorting + // haversine distances for candidates that could never be accepted. + const candidateNodes = this.osm.nodes + .findIndexesWithinBbox([patchNode.lon, patchNode.lat, patchNode.lon, patchNode.lat]) + .map((index) => this.osm.nodes.getByIndex(index)) + .map((baseNode) => this.getCurrentNode(baseNode.id) ?? baseNode) + .filter( + (baseNode) => + baseNode.id !== patchNode.id && + (!sameDataset || baseNode.id > patchNode.id) && + this.nodeChanges[baseNode.id]?.changeType !== "delete" && + sameOsmCoordinate(currentPatchNode, baseNode) && + !hasAnyTagConflict(currentPatchNode.tags, baseNode.tags) && + !hasConflictingGradeOrAccessTags(currentPatchNode.tags, baseNode.tags), + ); + + if (candidateNodes.length === 0) continue; + exactCandidates.push({ baseNodes: candidateNodes, patchNode: currentPatchNode }); + contextNodeIds.add(currentPatchNode.id); + for (const baseNode of candidateNodes) contextNodeIds.add(baseNode.id); + } + + if (exactCandidates.length === 0) return replacementMap; + + const waysByNode = this.currentWaysByNode(contextNodeIds); + for (const { baseNodes, patchNode } of exactCandidates) { + const compatibleNodes = baseNodes.filter((baseNode) => + this.nodeContextsCompatible(patchNode, baseNode, waysByNode), + ); + if (compatibleNodes.length === 0 || (!sameDataset && compatibleNodes.length !== 1)) continue; + const baseNode = sameDataset + ? compatibleNodes.toSorted((a, b) => b.id - a.id)[0] + : compatibleNodes[0]; + replacementMap.set(patchNode.id, baseNode!.id); + } + + if (replacementMap.size === 0) return replacementMap; + this.removeUnsafeNodeReplacements(replacementMap); + if (replacementMap.size === 0) return replacementMap; + this.applyNodeReplacementsToWays(replacementMap); + this.applyNodeReplacementsToRelations(replacementMap); + + for (const [patchNodeId, baseNodeId] of replacementMap) { + const patchNode = this.getCurrentNode(patchNodeId); + if (!patchNode) continue; + this.reconcileNodeTags(patchNode, baseNodeId); + this.deleteReconciledNode(patchNode, baseNodeId); + } + return replacementMap; } /** @@ -314,11 +620,10 @@ export class OsmChangeset { * Returns the total number of node references replaced. */ private applyNodeReplacementsToWays(replacementMap: Map<number, number>): number { + if (replacementMap.size === 0) return 0; let replacedCount = 0; - for (let wayIndex = 0; wayIndex < this.osm.ways.size; wayIndex++) { - const way = this.osm.ways.getByIndex(wayIndex); - if (this.wayChanges[way.id]?.changeType === "delete") continue; + for (const way of this.currentWays()) { let hasReplacement = false; const newRefs = way.refs.map((ref) => { const replacement = replacementMap.get(ref); @@ -349,14 +654,10 @@ export class OsmChangeset { * Returns the total number of node member references replaced. */ private applyNodeReplacementsToRelations(replacementMap: Map<number, number>): number { + if (replacementMap.size === 0) return 0; let replacedCount = 0; - for (let relationIndex = 0; relationIndex < this.osm.relations.size; relationIndex++) { - const baseRelation = this.osm.relations.getByIndex(relationIndex); - const relation = - (this.relationChanges[baseRelation.id]?.entity as - | OsmEntityTypeMap["relation"] - | undefined) ?? baseRelation; + for (const relation of this.currentRelations()) { let hasReplacement = false; const newMembers = relation.members.map((member) => { if (member.type !== "node") return member; @@ -369,7 +670,7 @@ export class OsmChangeset { return member; }); - if (hasReplacement && this.relationChanges[relation.id]?.changeType !== "delete") { + if (hasReplacement) { this.modify("relation", relation.id, (relation) => removeDuplicateAdjacentRelationMembers({ ...relation, @@ -383,18 +684,122 @@ export class OsmChangeset { return replacedCount; } + private replaceRestrictionViaNode(fromId: number, toId: number) { + for (const relation of this.currentRelations()) { + if ( + relation.tags?.["type"] !== "restriction" || + !relation.members.some( + (member) => member.type === "node" && member.role === "via" && member.ref === fromId, + ) + ) { + continue; + } + this.modify("relation", relation.id, (relation) => + removeDuplicateAdjacentRelationMembers({ + ...relation, + members: relation.members.map((member) => + member.type === "node" && member.role === "via" && member.ref === fromId + ? { ...member, ref: toId } + : member, + ), + }), + ); + } + } + + private chooseIntersectionNode( + wayNode: OsmNode, + intersectingWayNode: OsmNode, + wayIsPatch: boolean, + intersectingWayIsPatch: boolean, + ) { + if (hasAnyTagConflict(wayNode.tags, intersectingWayNode.tags)) return null; + + const wayRoutingTags = nodeRoutingTagCount(wayNode); + const intersectingRoutingTags = nodeRoutingTagCount(intersectingWayNode); + let keepWayNode: boolean; + if (wayIsPatch !== intersectingWayIsPatch) { + keepWayNode = !wayIsPatch; + } else if (wayRoutingTags !== intersectingRoutingTags) { + keepWayNode = wayRoutingTags > intersectingRoutingTags; + } else { + const wayTagCount = Object.keys(wayNode.tags ?? {}).length; + const intersectingTagCount = Object.keys(intersectingWayNode.tags ?? {}).length; + keepWayNode = wayTagCount >= intersectingTagCount; + } + + const survivor = keepWayNode ? wayNode : intersectingWayNode; + const replaced = keepWayNode ? intersectingWayNode : wayNode; + return { keepWayNode, replaced, survivor }; + } + + /** + * Endpoint reuse is a local rewrite of exactly one of the intersecting ways. + * Reject it before mutation when snapping two nearby crossings to the same + * endpoint would create invalid topology. + */ + private intersectionReplacementIsUnsafe( + way: OsmWay, + replacedNodeId: number, + survivorNodeId: number, + ) { + const refs = way.refs.map((ref) => (ref === replacedNodeId ? survivorNodeId : ref)); + if (refs.some((ref, index) => index > 0 && ref === refs[index - 1])) return true; + return new Set(refs).size < 2; + } + + private mergeNodeTags(survivor: OsmNode, replaced: OsmNode) { + const merged = withNonConflictingTags(survivor, replaced); + if (merged !== survivor) this.modify("node", survivor.id, () => merged); + return merged; + } + + private markNodeAsCrossing(nodeId: number) { + const node = this.getCurrentNode(nodeId); + if (!node || entityHasTagValue(node, "crossing", "yes")) return; + this.modify("node", node.id, (node) => ({ + ...node, + tags: { ...node.tags, crossing: "yes" }, + })); + } + /** * De-duplicate the ways within this OSM changeset. */ *deduplicateWaysGenerator(ways: Ways, replacementMap: ReplacementMap = new Map()) { const dedupedIdPairs = new IdPairs(); + const sameDataset = ways === this.osm.ways; + const exactWayIndex = sameDataset ? undefined : this.buildCrossDatasetExactWayIndex(); for (const way of ways) { - if (!this.osm.ways.ids.has(way.id)) continue; if (this.wayChanges[way.id]?.changeType === "delete") continue; - yield this.deduplicateWay(way, dedupedIdPairs, replacementMap); + yield this.deduplicateWayAgainstBase( + way, + sameDataset, + dedupedIdPairs, + replacementMap, + exactWayIndex, + ); } } + /** + * Index immutable base targets by ordered refs and routing semantics. Candidate + * buckets are collision-checked with the complete reconciliation predicates. + */ + private buildCrossDatasetExactWayIndex(): ExactWayIndex { + const index: ExactWayIndex = new Map(); + for (let wayIndex = 0; wayIndex < this.osm.ways.size; wayIndex++) { + const currentWay = this.getCurrentWay(this.osm.ways.getByIndex(wayIndex)); + if (!currentWay) continue; + const hash = exactWayHash(currentWay); + const indexed = index.get(hash); + if (indexed === undefined) index.set(hash, wayIndex); + else if (typeof indexed === "number") index.set(hash, [indexed, wayIndex]); + else indexed.push(wayIndex); + } + return index; + } + deduplicateWays(ways: Ways) { const replacementMap: ReplacementMap = new Map(); for (const _ of this.deduplicateWaysGenerator(ways, replacementMap)); @@ -406,14 +811,10 @@ export class OsmChangeset { * Returns the total number of way member references replaced. */ private applyWayReplacementsToRelations(replacementMap: ReplacementMap): number { + if (replacementMap.size === 0) return 0; let replacedCount = 0; - for (let relationIndex = 0; relationIndex < this.osm.relations.size; relationIndex++) { - const baseRelation = this.osm.relations.getByIndex(relationIndex); - const relation = - (this.relationChanges[baseRelation.id]?.entity as - | OsmEntityTypeMap["relation"] - | undefined) ?? baseRelation; + for (const relation of this.currentRelations()) { let hasReplacement = false; const newMembers = relation.members.map((member) => { if (member.type !== "way") return member; @@ -426,7 +827,7 @@ export class OsmChangeset { return member; }); - if (hasReplacement && this.relationChanges[relation.id]?.changeType !== "delete") { + if (hasReplacement) { this.modify("relation", relation.id, (relation) => removeDuplicateAdjacentRelationMembers({ ...relation, @@ -439,75 +840,84 @@ export class OsmChangeset { return replacedCount; } - /** - * Deduplicate a way by comparing it with existing ways in the OSM dataset. - * When a duplicate way is found, the patch way is deleted and references point to the kept way. - * - * Duplication criteria: - * - Geometrically identical (same coordinates). - * - Properties (except ID) must be roughly compatible. - * - Keeps the way with the higher version or more tags. - * - */ + private deleteReconciledWay(way: OsmWay, survivorId: number) { + const pendingChange = this.wayChanges[way.id]; + if (pendingChange?.changeType === "create") { + delete this.wayChanges[way.id]; + } else { + const storedWay = this.osm.ways.getById(way.id); + if (!storedWay) return; + this.delete(storedWay, [{ type: "way", id: survivorId, osmId: this.osm.id }]); + } + this.deduplicatedWays++; + } + + private deduplicateWayAgainstBase( + patchWay: OsmWay, + sameDataset: boolean, + dedupedIdPairs: IdPairs, + replacementMap: ReplacementMap, + exactWayIndex?: ExactWayIndex, + ) { + if (!this.osm.ways.ids.has(patchWay.id) && this.wayChanges[patchWay.id] == null) return 0; + if (!sameDataset && this.wayChanges[patchWay.id]?.changeType !== "create") return 0; + const currentPatchWay = + this.getCurrentWay(patchWay) ?? this.wayChanges[patchWay.id]?.entity ?? patchWay; + const indexed = exactWayIndex?.get(exactWayHash(currentPatchWay)); + if (exactWayIndex && indexed === undefined) return 0; + const indexedCandidates = + typeof indexed === "number" ? [indexed] : indexed === undefined ? [] : indexed; + + const wayCoords = this.getWayCoordinates(currentPatchWay); + if (!wayCoords || wayCoords.length < 2) return 0; + + const patchBbox = wayBbox(wayCoords); + const closeWayIndexes = exactWayIndex + ? indexedCandidates.filter((index) => + bboxesIntersect(patchBbox, this.osm.ways.getEntityBbox({ index })), + ) + : this.osm.ways.intersects(patchBbox); + const candidates = closeWayIndexes + .map((index) => this.osm.ways.getByIndex(index)) + .filter((baseWay) => { + if (baseWay.id === patchWay.id) return false; + if (sameDataset) { + if (baseWay.id < patchWay.id) return false; + } + if (dedupedIdPairs.has(patchWay.id, baseWay.id)) return false; + dedupedIdPairs.add(patchWay.id, baseWay.id); + const currentBaseWay = this.getCurrentWay(baseWay); + if (!currentBaseWay) return false; + if (!dequal(currentPatchWay.refs, currentBaseWay.refs)) return false; + return routingSemanticTagsEqual(currentPatchWay.tags, currentBaseWay.tags); + }); + + if (candidates.length === 0 || (!sameDataset && candidates.length !== 1)) return 0; + const baseWay = sameDataset ? candidates.toSorted((a, b) => b.id - a.id)[0] : candidates[0]; + const currentBaseWay = this.getCurrentWay(baseWay!); + if (!currentBaseWay) return 0; + + const mergedWay = withNonConflictingDescriptiveTags(currentBaseWay, currentPatchWay); + if (mergedWay !== currentBaseWay) this.modify("way", currentBaseWay.id, () => mergedWay); + + replacementMap.set(patchWay.id, currentBaseWay.id); + this.applyWayReplacementsToRelations(replacementMap); + this.deleteReconciledWay(patchWay, currentBaseWay.id); + return 1; + } + + /** Reconcile one incoming way with a unique, equivalent base way. */ deduplicateWay( patchWay: OsmWay, dedupedIdPairs: IdPairs, replacementMap: ReplacementMap = new Map(), ) { - const wayIndex = this.osm.ways.ids.getIndexFromId(patchWay.id); - const wayCoords = this.osm.ways.getCoordinates(wayIndex); - - // Look for duplicate ways in OSM index - const closeWayIndexes = this.osm.ways.intersects(this.osm.ways.getEntityBbox(patchWay)); - const wayVersion = getEntityVersion(patchWay); - const wayTagCount = Object.keys(patchWay.tags ?? {}).length; - const candidateDuplicateWays: OsmWay[] = closeWayIndexes - .map((index) => { - const otherWay = this.osm.ways.getByIndex(index); - if (otherWay.id === patchWay.id) return null; - - // Has this pair been deduped or checked already? - if (dedupedIdPairs.has(patchWay.id, otherWay.id)) return null; - dedupedIdPairs.add(patchWay.id, otherWay.id); - - // Check if all way properties other than the ID are equal - if (isWayEqual(patchWay, otherWay)) return otherWay; - - // Check geometry - const coords = this.osm.ways.getCoordinates(index); - if (!dequal(wayCoords, coords)) return null; - - // Check version - const otherWayVersion = getEntityVersion(otherWay); - if (otherWayVersion < wayVersion) return null; - if (otherWayVersion > wayVersion) return otherWay; - - // Ways are geometrically equal, with same version. Keep the way with more tags - const tagCount = Object.keys(otherWay.tags ?? {}).length; - return tagCount >= wayTagCount ? otherWay : null; - }) - .filter((way) => way != null); - - if (candidateDuplicateWays.length === 0) return 0; - - const survivorIds = candidateDuplicateWays - .map((way) => resolveReplacement(way.id, replacementMap)) - .filter((id, index, ids) => ids.indexOf(id) === index); - if (survivorIds.length === 0) return 0; - const survivorId = survivorIds.length === 1 ? survivorIds[0]! : Math.min(...survivorIds); - const finalSurvivorId = resolveReplacement(survivorId, replacementMap); - if (finalSurvivorId === patchWay.id) { - throw Error(`Replacement cycle detected at way ${patchWay.id}`); - } - replacementMap.set(patchWay.id, finalSurvivorId); - const flattenedMap = flattenReplacementMap(replacementMap); - this.applyWayReplacementsToRelations(flattenedMap); - - // Delete this way - this.delete(patchWay, [{ type: "way", id: finalSurvivorId, osmId: this.osm.id }]); - this.deduplicatedWays++; - - return candidateDuplicateWays.length; + return this.deduplicateWayAgainstBase( + patchWay, + this.osm.ways.ids.has(patchWay.id), + dedupedIdPairs, + replacementMap, + ); } /** @@ -519,9 +929,22 @@ export class OsmChangeset { */ *createIntersectionsForWaysGenerator(ways: Ways) { const wayIdPairs = new IdPairs(); + const patchWayIds = new Set<number>(); + for (const way of ways) patchWayIds.add(way.id); + const metadata = this.buildIntersectionMetadata(); for (const way of ways) { - if (!this.osm.ways.ids.has(way.id)) continue; - yield this.createIntersectionsForWay({ id: way.id }, wayIdPairs); + // Yield once per input way so callers can report complete progress even + // when exact reconciliation already removed an equivalent patch way. + if (!this.osm.ways.ids.has(way.id)) { + yield; + continue; + } + yield this.createIntersectionsForWayInternal( + { id: way.id }, + wayIdPairs, + patchWayIds, + metadata, + ); } } @@ -541,20 +964,91 @@ export class OsmChangeset { return change?.entity ?? this.osm.nodes.getById(id); } + /** + * Precompute the immutable tag checks used for every spatial candidate. Way + * refs change during insertion, but intersection eligibility and grade do not. + */ + private buildIntersectionMetadata(): IntersectionMetadata { + const eligible = new Uint8Array(this.osm.ways.size); + const gradeIds = new Int32Array(this.osm.ways.size); + const grades = new Map<string, number>(); + let nextGradeId = 1; + + for (let index = 0; index < this.osm.ways.size; index++) { + const wayId = this.osm.ways.ids.at(index); + const change = this.wayChanges[wayId]; + if (change?.changeType === "delete") continue; + const tags = change ? change.entity.tags : this.osm.ways.tags.getTags(index); + if (!areWayTagsIntersectionCandidate(tags)) continue; + eligible[index] = 1; + const grade = routingGradeSignature(tags); + let gradeId = grades.get(grade); + if (gradeId === undefined) { + gradeId = nextGradeId++; + grades.set(grade, gradeId); + } + gradeIds[index] = gradeId; + } + + return { eligible, gradeIds }; + } + /** * Resolve way coordinates from the base dataset plus pending node changes. * Returns null when any ref is genuinely unavailable instead of substituting geometry. */ private getWayCoordinates(way: OsmWay): [number, number][] | null { + const wayRevision = this.wayGeometryRevisions.get(way.id) ?? 0; + const cached = this.wayCoordinateCache.get(way.id); + if ( + cached && + cached.wayRevision === wayRevision && + cached.nodeCoordinateRevision === this.nodeCoordinateRevision + ) { + return cached.coordinates; + } + + // Unchanged base geometry can resolve packed node indexes directly. This + // avoids one binary ID lookup per ref in the intersection hot path. Match + // the fallback's missing-ref behavior by requiring every ref to resolve. + if (this.nodeCoordinateRevision === 0 && this.wayChanges[way.id] === undefined) { + const [wayIndex] = this.osm.ways.ids.idOrIndex({ id: way.id }); + if (wayIndex !== -1) { + const coordinates = this.osm.ways.getResolvedCoordinates(wayIndex); + if (coordinates.length !== way.refs.length) return null; + this.wayCoordinateCache.set(way.id, { + coordinates, + nodeCoordinateRevision: this.nodeCoordinateRevision, + wayRevision, + }); + return coordinates; + } + } + const coordinates: [number, number][] = []; for (const ref of way.refs) { const node = this.getCurrentNode(ref); + // Do not cache unresolved geometry: a later node creation can make this + // same set of refs resolvable without changing the way revision. if (!node) return null; coordinates.push([node.lon, node.lat]); } + this.wayCoordinateCache.set(way.id, { + coordinates, + nodeCoordinateRevision: this.nodeCoordinateRevision, + wayRevision, + }); return coordinates; } + private getCleanWayCoordinates(way: OsmWay): [number, number][] | null { + const coordinates = this.getWayCoordinates(way); + if (!coordinates) return null; + const cached = this.wayCoordinateCache.get(way.id); + if (!cached) return cleanCoords(coordinates); + return (cached.cleaned ??= cleanCoords(coordinates)); + } + /** * Create intersections for a single way. * - Finds other ways that intersect the given way's bounding box. @@ -563,127 +1057,195 @@ export class OsmChangeset { * - Inserts existing nodes or creates new intersection nodes at the crossing points. */ createIntersectionsForWay(wayIdOrIndex: IdOrIndex, wayIdPairs: IdPairs) { + return this.createIntersectionsForWayInternal( + wayIdOrIndex, + wayIdPairs, + null, + this.buildIntersectionMetadata(), + ); + } + + private createIntersectionsForWayInternal( + wayIdOrIndex: IdOrIndex, + wayIdPairs: IdPairs, + patchWayIds: ReadonlySet<number> | null, + metadata: IntersectionMetadata, + ) { let intersectionsFound = 0; let intersectionsCreated = 0; // Get the actual way from the OSM data (which may have been modified by deduplication) const [wayIndex] = this.osm.ways.ids.idOrIndex(wayIdOrIndex); - const way = this.getCurrentWay(this.osm.ways.getByIndex(wayIndex)); - if (!way) return; - if (!isWayIntersectionCandidate(way)) return; + if (wayIndex >= 0 && metadata.eligible[wayIndex] !== 1) return; + const baseWay = this.osm.ways.getByIndex(wayIndex); + const initialWay = this.getCurrentWay(baseWay); + if (!initialWay) return; - const wayCoordinates = this.getWayCoordinates(way); - if (!wayCoordinates || wayCoordinates.length < 2) return; + const initialWayCoordinates = this.getWayCoordinates(initialWay); + if (!initialWayCoordinates || initialWayCoordinates.length < 2) return; // Check for intersecting ways. Since the way exists in the base OSM, there will always be at least one way. const bbox = this.osm.ways.getEntityBbox({ index: wayIndex }); - const intersectingWayIndexes = this.osm.ways.intersects(bbox); - if (intersectingWayIndexes.length <= 1) return; // No candidates + const initialGradeId = metadata.gradeIds[wayIndex]; + const intersectingWayIndexes = this.osm.ways.intersects(bbox, (intersectingWayIndex) => { + const intersectingWayId = this.osm.ways.ids.at(intersectingWayIndex); + if (intersectingWayId == null || intersectingWayId === initialWay.id) return false; + if (wayIdPairs.has(initialWay.id, intersectingWayId)) return false; + + // The old loop recorded every spatial pair before checking routing and + // grade compatibility. Keep that side effect while avoiding entity and + // coordinate work for pairs that can never connect. + if ( + metadata.eligible[intersectingWayIndex] !== 1 || + metadata.gradeIds[intersectingWayIndex] !== initialGradeId + ) { + wayIdPairs.add(initialWay.id, intersectingWayId); + return false; + } + return true; + }); + if (intersectingWayIndexes.length === 0) return; - const coordinates = cleanCoords(wayCoordinates); for (const intersectingWayIndex of intersectingWayIndexes) { const intersectingWayId = this.osm.ways.ids.at(intersectingWayIndex); // Skip self and null ways - if (intersectingWayId == null || intersectingWayId === way.id) continue; - if (wayIdPairs.has(way.id, intersectingWayId)) continue; - wayIdPairs.add(way.id, intersectingWayId); + if (intersectingWayId == null || intersectingWayId === initialWay.id) continue; + if (wayIdPairs.has(initialWay.id, intersectingWayId)) continue; + wayIdPairs.add(initialWay.id, intersectingWayId); // Skip ways that aren't applicable for connecting + const way = this.getCurrentWay(baseWay); const intersectingWay = this.getCurrentWay(this.osm.ways.getByIndex(intersectingWayIndex)); - if (!intersectingWay) continue; + if (!way || !intersectingWay) continue; if (!waysShouldConnect(way.tags, intersectingWay.tags)) continue; + const wayCoordinates = this.getWayCoordinates(way); const intersectingWayCoordinates = this.getWayCoordinates(intersectingWay); - if (!intersectingWayCoordinates || intersectingWayCoordinates.length < 2) continue; - const intersectingWayCoords = cleanCoords(intersectingWayCoordinates); + if ( + !wayCoordinates || + wayCoordinates.length < 2 || + !intersectingWayCoordinates || + intersectingWayCoordinates.length < 2 + ) { + continue; + } + const coordinates = this.getCleanWayCoordinates(way); + const intersectingWayCoords = this.getCleanWayCoordinates(intersectingWay); + if (!coordinates || !intersectingWayCoords) continue; // Skip ways that are geometrically equal if (dequal(coordinates, intersectingWayCoords)) continue; const intersectingPoints = waysIntersect(coordinates, intersectingWayCoords); for (const pt of intersectingPoints) { + const currentWay = this.getCurrentWay(baseWay); + // Reuse the already decoded base entity; getCurrentWay still selects any + // pending rewrite made by an earlier point in this same pair. + const currentIntersectingWay = this.getCurrentWay(intersectingWay); + if (!currentWay || !currentIntersectingWay) continue; + const currentWayCoordinates = this.getWayCoordinates(currentWay); + const currentIntersectingWayCoordinates = this.getWayCoordinates(currentIntersectingWay); + if (!currentWayCoordinates || !currentIntersectingWayCoordinates) continue; + const intersectingWayNodeId = nearestNodeOnWay( - intersectingWay, - intersectingWayCoords, + currentIntersectingWay, + currentIntersectingWayCoordinates, pt, ).nodeId; - const wayNodeId = nearestNodeOnWay(way, coordinates, pt).nodeId; + const wayNodeId = nearestNodeOnWay(currentWay, currentWayCoordinates, pt).nodeId; // If both ways already share the same node at this intersection, // just add the crossing tag (if needed) but don't count as an intersection. - if (wayNodeId && intersectingWayNodeId && wayNodeId === intersectingWayNodeId) { - const sharedNode = this.getCurrentNode(wayNodeId); - if (sharedNode && !entityHasTagValue(sharedNode, "crossing", "yes")) { - this.modify("node", sharedNode.id, (node) => { - return { - ...node, - tags: { ...node.tags, crossing: "yes" }, - }; - }); - } + if ( + wayNodeId != null && + intersectingWayNodeId != null && + wayNodeId === intersectingWayNodeId + ) { + this.markNodeAsCrossing(wayNodeId); continue; } - intersectionsFound++; - - // Prefer the incoming way node, then the intersecting way node, then a new node. - if (wayNodeId) { + let endpointResolution: ReturnType<OsmChangeset["chooseIntersectionNode"]> | undefined; + let createDedicatedIntersection = false; + if (wayNodeId != null && intersectingWayNodeId != null) { const wayNode = this.getCurrentNode(wayNodeId); - if (wayNode == null) throw Error(`Way node ${String(wayNodeId)} not found`); - if (intersectingWayNodeId) { - // Replace in intersecting way - this.modify("way", intersectingWay.id, (way) => { - return { - ...way, - refs: way.refs.map((ref) => (ref === intersectingWayNodeId ? wayNodeId : ref)), - }; - }); - } else { - this.spliceNodeIntoWay(intersectingWay, wayNode); + const intersectingWayNode = this.getCurrentNode(intersectingWayNodeId); + if (!wayNode || !intersectingWayNode) continue; + endpointResolution = this.chooseIntersectionNode( + wayNode, + intersectingWayNode, + patchWayIds?.has(currentWay.id) ?? false, + patchWayIds?.has(currentIntersectingWay.id) ?? false, + ); + if (!endpointResolution) continue; + const rewrittenWay = endpointResolution.keepWayNode ? currentIntersectingWay : currentWay; + if ( + this.intersectionReplacementIsUnsafe( + rewrittenWay, + endpointResolution.replaced.id, + endpointResolution.survivor.id, + ) + ) { + endpointResolution = undefined; + createDedicatedIntersection = true; } + } - if (!entityHasTagValue(wayNode, "crossing", "yes")) { - this.modify("node", wayNode.id, (node) => { - return { - ...node, - tags: { ...node.tags, crossing: "yes" }, - }; - }); + intersectionsFound++; + + if (endpointResolution) { + const survivor = this.mergeNodeTags( + endpointResolution.survivor, + endpointResolution.replaced, + ); + if (endpointResolution.keepWayNode) { + this.modify("way", currentIntersectingWay.id, (way) => ({ + ...way, + refs: way.refs.map((ref) => + ref === endpointResolution!.replaced.id ? survivor.id : ref, + ), + })); + } else { + this.modify("way", currentWay.id, (way) => ({ + ...way, + refs: way.refs.map((ref) => + ref === endpointResolution!.replaced.id ? survivor.id : ref, + ), + })); } - } else if (intersectingWayNodeId) { + this.replaceRestrictionViaNode(endpointResolution.replaced.id, survivor.id); + this.markNodeAsCrossing(survivor.id); + } else if (createDedicatedIntersection) { + intersectionsCreated++; + const newIntersectionNode = this.createIntersectionNode( + currentWay, + currentIntersectingWay, + pt, + ); + this.spliceNodeIntoWay(currentWay, newIntersectionNode); + this.spliceNodeIntoWay(currentIntersectingWay, newIntersectionNode); + } else if (wayNodeId != null) { + const wayNode = this.getCurrentNode(wayNodeId); + if (wayNode == null) throw Error(`Way node ${String(wayNodeId)} not found`); + this.spliceNodeIntoWay(currentIntersectingWay, wayNode); + this.markNodeAsCrossing(wayNode.id); + } else if (intersectingWayNodeId != null) { const intersectingWayNode = this.getCurrentNode(intersectingWayNodeId); if (intersectingWayNode == null) throw Error(`Intersecting way node ${String(intersectingWayNodeId)} not found`); - this.spliceNodeIntoWay(way, intersectingWayNode); - if (!entityHasTagValue(intersectingWayNode, "crossing", "yes")) { - this.modify("node", intersectingWayNode.id, (node) => { - return { - ...node, - tags: { ...node.tags, crossing: "yes" }, - }; - }); - } + this.spliceNodeIntoWay(currentWay, intersectingWayNode); + this.markNodeAsCrossing(intersectingWayNode.id); } else { intersectionsCreated++; - - const newIntersectionNode: OsmNode = { - id: this.nextNodeId(), - lon: pt[0], - lat: pt[1], - tags: { - crossing: "yes", - }, - }; - this.create(newIntersectionNode, this.osm.id, [ - { type: "way", id: way.id, osmId: this.osm.id }, - { type: "way", id: intersectingWay.id, osmId: this.osm.id }, - ]); - - // Splice into the existing ways - this.spliceNodeIntoWay(way, newIntersectionNode); - this.spliceNodeIntoWay(intersectingWay, newIntersectionNode); + const newIntersectionNode = this.createIntersectionNode( + currentWay, + currentIntersectingWay, + pt, + ); + this.spliceNodeIntoWay(currentWay, newIntersectionNode); + this.spliceNodeIntoWay(currentIntersectingWay, newIntersectionNode); } } } @@ -697,6 +1259,28 @@ export class OsmChangeset { }; } + private createIntersectionNode( + way: OsmWay, + intersectingWay: OsmWay, + point: [number, number], + ): OsmNode { + const node: OsmNode = { + id: this.nextNodeId(), + lon: point[0], + lat: point[1], + tags: { + crossing: "yes", + }, + }; + // The new maximum ID cannot be referenced by existing base ways, so adding + // it does not invalidate any cached geometry until each way is spliced. + this.recordCreate(node, this.osm.id, [ + { type: "way", id: way.id, osmId: this.osm.id }, + { type: "way", id: intersectingWay.id, osmId: this.osm.id }, + ]); + return node; + } + /** * We do not pass coordinates here because the way may have already been modified. */ @@ -704,32 +1288,50 @@ export class OsmChangeset { const currentWay = this.getCurrentWay(way); if (!currentWay) return; const coordinates = this.getWayCoordinates(currentWay); - if (!coordinates || coordinates.length === 0) return; - const { refIndex } = nearestNodeOnWay( - currentWay, - coordinates, - [node.lon, node.lat], - Number.POSITIVE_INFINITY, - ); - if (refIndex < 0) return; + if (!coordinates || coordinates.length < 2 || currentWay.refs.includes(node.id)) return; + + let closestSegment = -1; + let closestDistance = Number.POSITIVE_INFINITY; + for (let index = 0; index < coordinates.length - 1; index++) { + const start = coordinates[index]!; + const end = coordinates[index + 1]!; + const dx = end[0] - start[0]; + const dy = end[1] - start[1]; + const lengthSquared = dx * dx + dy * dy; + if (lengthSquared === 0) continue; + const projection = ((node.lon - start[0]) * dx + (node.lat - start[1]) * dy) / lengthSquared; + const parameter = Math.max(0, Math.min(1, projection)); + const projectedLon = start[0] + parameter * dx; + const projectedLat = start[1] + parameter * dy; + const distance = (node.lon - projectedLon) ** 2 + (node.lat - projectedLat) ** 2; + if (distance >= closestDistance) continue; + closestDistance = distance; + closestSegment = index; + } + if (closestSegment < 0) return; this.modify("way", way.id, (way) => ({ ...way, - refs: way.refs.toSpliced(refIndex, 0, node.id), + refs: way.refs.toSpliced(closestSegment + 1, 0, node.id), })); } /** - * Create changes to merge nodes, ways, and relations from a patch OSM file into the base OSM. - * - Check for duplicate nodes in the patch, replace the existing nodes where appropriate. - * - Check for duplicate incoming ways, only add single instances of geometrically equal ways. + * Create direct same-ID modifications and new-entity changes from a patch OSM file. * * Implementation notes: - * - Ways are processed before nodes to improve node deduplication accuracy (see comment on line 633). - * - Node replacements in relations are handled by `applyNodeReplacementsToRelations()` when - * deduplicating nodes, but relation member updates during direct merge are not automatically - * handled. Use `deduplicateNodes()` after `generateDirectChanges()` if relation updates are needed. + * - Ways are processed before nodes so subsequent node reconciliation can inspect pending ways. + * - Call `deduplicateNodes()` and `deduplicateWays()` afterward for conservative cross-dataset + * reconciliation and relation-member rewrites. */ generateDirectChanges(patch: Osm) { + for (const key of inheritedRoutingIntegrityIssueKeys( + this.osm, + patch, + this.routingIntegrityBaselineKeys, + )) { + this.routingIntegrityBaselineKeys.add(key); + } + // Reset the current node ID to the highest node ID in the base or patch. const maximums = [maximumId(this.osm.nodes.ids), maximumId(patch.nodes.ids)].filter( (id): id is number => id !== null, @@ -781,18 +1383,25 @@ export class OsmChangeset { } class IdPairs { - #idPairs = new Set<string>(); - - #makeIdsKey(wayIds: number[]) { - return wayIds.toSorted((a, b) => a - b).join(","); - } - - add(...wayIds: number[]) { - this.#idPairs.add(this.#makeIdsKey(wayIds)); + #idPairs = new Map<number, number | Set<number>>(); + + add(firstId: number, secondId: number) { + const lowerId = Math.min(firstId, secondId); + const higherId = Math.max(firstId, secondId); + const partners = this.#idPairs.get(lowerId); + if (partners === undefined) this.#idPairs.set(lowerId, higherId); + else if (typeof partners === "number") { + if (partners !== higherId) this.#idPairs.set(lowerId, new Set([partners, higherId])); + } else partners.add(higherId); } - has(...wayIds: number[]) { - return this.#idPairs.has(this.#makeIdsKey(wayIds)); + has(firstId: number, secondId: number) { + const lowerId = Math.min(firstId, secondId); + const higherId = Math.max(firstId, secondId); + const partners = this.#idPairs.get(lowerId); + return typeof partners === "number" + ? partners === higherId + : (partners?.has(higherId) ?? false); } clear() { diff --git a/packages/change/src/conflation.ts b/packages/change/src/conflation.ts new file mode 100644 index 00000000..2a6c4d6a --- /dev/null +++ b/packages/change/src/conflation.ts @@ -0,0 +1,1645 @@ +/** Safe, explicit proximity conflation for imported OSM-like datasets. */ + +import type { Osm } from "@osmix/core"; +import { haversineDistance } from "@osmix/geo/haversine-distance"; +import type { ProgressEvent } from "@osmix/shared/progress"; +import type { LonLat, OsmEntity, OsmNode, OsmRelation, OsmTags, OsmWay } from "@osmix/types"; + +import { applyChangesetToOsm } from "./apply-changeset.ts"; +import { OsmChangeset } from "./changeset.ts"; +import { generateChangeset } from "./generate-changeset.ts"; +import { assertConflationPreservesBaseTopology } from "./integrity.ts"; +import type { + OsmConflationActionAssessment, + OsmConflationBulkDecisionRequest, + OsmConflationBulkDecisionResult, + OsmConflationCandidate, + OsmConflationCandidateFilter, + OsmConflationDecision, + OsmConflationDiscovery, + OsmConflationEffectiveStatus, + OsmConflationEvidence, + OsmConflationOptions, + OsmConflationReasonCode, + OsmConflationRoutingFamily, + OsmConflationSummary, + OsmConflationTagDiff, + OsmMergeOptions, + ResolvedOsmConflationOptions, +} from "./types.ts"; +import { routingGradeSignature } from "./utils.ts"; + +// Preserve the historical one-meter matching radius, but only inside this explicit, +// cross-dataset workflow. Proximity alone never authorizes a topology change. +const DEFAULT_MAX_DISTANCE_METERS = 1; +const MAX_BEARING_DIFFERENCE_DEGREES = 30; +const MAX_LENGTH_DIFFERENCE_RATIO = 0.05; +const SAMPLE_INTERVAL_METERS = 5; + +const PEDESTRIAN_HIGHWAYS = new Set(["corridor", "footway", "path", "pedestrian", "steps"]); +const BICYCLE_HIGHWAYS = new Set(["cycleway"]); +const NON_MOTOR_HIGHWAYS = new Set([...PEDESTRIAN_HIGHWAYS, ...BICYCLE_HIGHWAYS, "bridleway"]); +// Access and routing checks also recognize namespaced variants (for example +// `access:conditional` and `maxspeed:forward`) so they cannot bypass review. +const ACCESS_KEYS = [ + "access", + "agricultural", + "atv", + "bicycle", + "bus", + "caravan", + "carriage", + "coach", + "emergency", + "foot", + "forestry", + "golf_cart", + "goods", + "horse", + "hgv", + "hgv_articulated", + "hov", + "inline_skates", + "mofa", + "moped", + "motorcycle", + "motor_vehicle", + "motorcar", + "motorhome", + "psv", + "ski", + "snowmobile", + "taxi", + "tourist_bus", + "trailer", + "vehicle", + "wheelchair", +] as const; +const PROTECTED_KEYS = new Set([ + "area", + "bridge", + "covered", + "layer", + "level", + "restriction", + "tunnel", + "type", +]); +const ROUTING_KEYS = new Set([ + ...ACCESS_KEYS, + "barrier", + "crossing", + "highway", + "junction", + "kerb", + "maxspeed", + "oneway", +]); + +type EntityRelationContext = { + nodes: Set<number>; + ways: Set<number>; + restrictionNodes: Set<number>; + restrictionWays: Set<number>; +}; + +type DiscoveryContext = { + base: Osm; + patch: Osm; + options: ResolvedOsmConflationOptions; + baseWaysByNode: Map<number, OsmWay[]>; + patchWaysByNode: Map<number, OsmWay[]>; + baseRelations: EntityRelationContext; + patchRelations: EntityRelationContext; +}; + +// Trusted merge orchestrators keep untouched Osm objects and canonical discovery +// in the same module instance. This weak registry lets that internal path reuse an +// expensive discovery without weakening the public generation boundary, which +// still recomputes candidates before it accepts caller-provided review data. +const trustedDiscoveries = new WeakMap<OsmConflationDiscovery, { base: Osm; patch: Osm }>(); +const trustedCandidateCollections = new WeakSet<readonly OsmConflationCandidate[]>(); +const trustedCandidateIds = new WeakMap<readonly OsmConflationCandidate[], ReadonlySet<string>>(); + +function resolvedOptions(options: OsmConflationOptions): ResolvedOsmConflationOptions { + if (!Array.isArray(options.propertyKeys)) { + throw Error("Conflation propertyKeys must be an array"); + } + if (options.propertyKeys.some((key) => typeof key !== "string" || key.length === 0)) { + throw Error("Conflation propertyKeys must contain only non-empty strings"); + } + if (typeof options.attachNetwork !== "boolean") { + throw Error("Conflation attachNetwork must be a boolean"); + } + if (options.automatic != null && !["high-confidence", "none"].includes(options.automatic)) { + throw Error("Conflation automatic must be high-confidence or none"); + } + const maxDistanceMeters = options.maxDistanceMeters ?? DEFAULT_MAX_DISTANCE_METERS; + if (!Number.isFinite(maxDistanceMeters) || maxDistanceMeters <= 0) { + throw Error("Conflation maxDistanceMeters must be a positive finite number"); + } + const propertyKeys = [...new Set(options.propertyKeys)].toSorted(); + if (propertyKeys.length === 0 && !options.attachNetwork) { + throw Error("Conflation requires at least one property key or network attachment"); + } + return { + propertyKeys, + attachNetwork: options.attachNetwork, + maxDistanceMeters, + automatic: options.automatic ?? "high-confidence", + }; +} + +function candidateId(entityType: "node" | "way", sourceId: number, targetId: number | null) { + return `${entityType}:${sourceId}->${targetId ?? "none"}`; +} + +function uniqueReasons(reasons: readonly OsmConflationReasonCode[]) { + return [...new Set(reasons)].toSorted(); +} + +function roundEvidence(value: number) { + return Number(value.toFixed(6)); +} + +function waysByNode(osm: Osm) { + const result = new Map<number, OsmWay[]>(); + for (const way of osm.ways) { + for (const ref of new Set(way.refs)) { + const ways = result.get(ref) ?? []; + ways.push(way); + result.set(ref, ways); + } + } + return result; +} + +function relationContext(osm: Osm): EntityRelationContext { + const context: EntityRelationContext = { + nodes: new Set(), + ways: new Set(), + restrictionNodes: new Set(), + restrictionWays: new Set(), + }; + for (const relation of osm.relations) { + const restriction = relation.tags?.["type"] === "restriction"; + for (const member of relation.members) { + if (member.type === "node") { + context.nodes.add(member.ref); + if (restriction) context.restrictionNodes.add(member.ref); + } else if (member.type === "way") { + context.ways.add(member.ref); + if (restriction) context.restrictionWays.add(member.ref); + } + } + } + return context; +} + +function isAreaWay(way: OsmWay) { + if (String(way.tags?.["area"] ?? "") === "yes") return true; + if (way.refs.length < 4 || way.refs[0] !== way.refs.at(-1)) return false; + return ["building", "landuse", "natural", "boundary"].some((key) => way.tags?.[key] != null); +} + +function wayRoutingFamily(way: OsmWay): OsmConflationRoutingFamily { + const highway = String(way.tags?.["highway"] ?? ""); + if (!highway || isAreaWay(way)) return "non-routable"; + if ( + BICYCLE_HIGHWAYS.has(highway) || + (highway === "path" && !["no", "private"].includes(String(way.tags?.["bicycle"] ?? ""))) + ) { + return "bicycle-shared"; + } + if (PEDESTRIAN_HIGHWAYS.has(highway)) return "pedestrian"; + // Unknown highway values stay in the motor family. Treating a potentially + // drivable way as non-routable would make an unsafe attachment look harmless. + if (!NON_MOTOR_HIGHWAYS.has(highway)) return "motor-road"; + return "non-routable"; +} + +function routingFamilies(ways: readonly OsmWay[]) { + const families = new Set(ways.map(wayRoutingFamily)); + if (families.size > 1) families.delete("non-routable"); + return [...families].toSorted() as OsmConflationRoutingFamily[]; +} + +function familyCompatible(a: OsmConflationRoutingFamily, b: OsmConflationRoutingFamily) { + if (a === b) return true; + return ( + (a === "pedestrian" && b === "bicycle-shared") || (a === "bicycle-shared" && b === "pedestrian") + ); +} + +function accessSignature(tags: OsmTags | undefined) { + return Object.keys(tags ?? {}) + .filter((key) => + ACCESS_KEYS.some((accessKey) => key === accessKey || key.startsWith(`${accessKey}:`)), + ) + .toSorted() + .map((key) => `${key}=${String(tags?.[key] ?? "")}`) + .join("|"); +} + +// These signatures intentionally compare both presence and value. Rewriting a +// patch reference must not strand node-level routing semantics on the discarded node. +function barrierSignature(tags: OsmTags | undefined) { + return Object.keys(tags ?? {}) + .filter((key) => key === "barrier" || key.startsWith("barrier:")) + .toSorted() + .map((key) => `${key}=${String(tags?.[key] ?? "")}`) + .join("|"); +} + +function nodeRoutingSignature(tags: OsmTags | undefined) { + return Object.keys(tags ?? {}) + .filter( + (key) => + isRoutingProperty(key) && + !ACCESS_KEYS.some((accessKey) => key === accessKey || key.startsWith(`${accessKey}:`)) && + key !== "barrier" && + !key.startsWith("barrier:"), + ) + .toSorted() + .map((key) => `${key}=${String(tags?.[key] ?? "")}`) + .join("|"); +} + +function wayContextsCompatible(source: OsmWay, target: OsmWay) { + return ( + familyCompatible(wayRoutingFamily(source), wayRoutingFamily(target)) && + wayGradeAccessCompatible(source, target) + ); +} + +function wayGradeAccessCompatible(source: OsmWay, target: OsmWay) { + return ( + routingGradeSignature(source.tags) === routingGradeSignature(target.tags) && + accessSignature(source.tags) === accessSignature(target.tags) + ); +} + +function normalizedOneway(way: OsmWay) { + const value = String(way.tags?.["oneway"] ?? "").toLowerCase(); + if (["yes", "true", "1"].includes(value)) return "forward"; + if (["-1", "reverse"].includes(value)) return "reverse"; + if (String(way.tags?.["junction"] ?? "") === "roundabout" && value !== "no") { + return "forward"; + } + return "both"; +} + +function reversedOneway(value: ReturnType<typeof normalizedOneway>) { + return value === "forward" ? "reverse" : value === "reverse" ? "forward" : value; +} + +function wayRoutingSemanticsCompatible(source: OsmWay, target: OsmWay, targetReversed: boolean) { + const targetOneway = normalizedOneway(target); + if (normalizedOneway(source) !== (targetReversed ? reversedOneway(targetOneway) : targetOneway)) { + return false; + } + const routingKeys = new Set( + [...Object.keys(source.tags ?? {}), ...Object.keys(target.tags ?? {})].filter( + (key) => isRoutingProperty(key) && key !== "oneway", + ), + ); + if ( + targetReversed && + // Reversed geometry is safe only when no remaining routing tag has a direction + // whose meaning would also need to be inverted or swapped. + [...routingKeys].some( + (key) => + key.startsWith("oneway:") || + key.split(":").some((part) => ["backward", "forward", "left", "right"].includes(part)), + ) + ) { + return false; + } + return [...routingKeys].every( + (key) => String(source.tags?.[key] ?? "") === String(target.tags?.[key] ?? ""), + ); +} + +function isProtectedProperty(key: string) { + return PROTECTED_KEYS.has(key) || key.startsWith("restriction:"); +} + +function isRoutingProperty(key: string) { + return [...ROUTING_KEYS].some( + (routingKey) => key === routingKey || key.startsWith(`${routingKey}:`), + ); +} + +function selectedTagDiff( + source: OsmEntity, + target: OsmEntity, + propertyKeys: readonly string[], +): OsmConflationTagDiff[] { + const result: OsmConflationTagDiff[] = []; + for (const key of propertyKeys) { + const patchValue = source.tags?.[key]; + if (patchValue == null || target.tags?.[key] === patchValue) continue; + result.push({ + key, + patchValue, + baseValue: target.tags?.[key], + protected: isProtectedProperty(key), + routing: isRoutingProperty(key), + }); + } + return result; +} + +function propertyAssessment( + tagDiff: readonly OsmConflationTagDiff[], + options: ResolvedOsmConflationOptions, +): OsmConflationActionAssessment { + if (tagDiff.length === 0) { + return { status: "blocked", reasons: ["no-transferable-properties"] }; + } + const transferable = tagDiff.filter((diff) => !diff.protected); + if (transferable.length === 0) return { status: "blocked", reasons: ["protected-tag"] }; + + const reasons: OsmConflationReasonCode[] = []; + if (transferable.some((diff) => diff.routing)) reasons.push("routing-property"); + if (tagDiff.some((diff) => diff.protected)) reasons.push("protected-tag"); + if (reasons.length > 0 || options.automatic === "none") { + return { status: "review", reasons: uniqueReasons(reasons) }; + } + return { status: "automatic", reasons: [] }; +} + +function nodePropertyAssessment( + context: DiscoveryContext, + patchWays: readonly OsmWay[], + baseWays: readonly OsmWay[], + tagDiff: readonly OsmConflationTagDiff[], +) { + const assessment = propertyAssessment(tagDiff, context.options); + if (assessment.status === "blocked") return assessment; + + const patchAreaOnly = patchWays.length > 0 && patchWays.every(isAreaWay); + const baseAreaOnly = baseWays.length > 0 && baseWays.every(isAreaWay); + const patchRoutable = patchWays.filter((way) => wayRoutingFamily(way) !== "non-routable"); + const baseRoutable = baseWays.filter((way) => wayRoutingFamily(way) !== "non-routable"); + const reasons = [...assessment.reasons]; + let hardConflict = false; + if (patchAreaOnly !== baseAreaOnly && (patchAreaOnly || baseAreaOnly)) { + reasons.push("non-routing-target"); + hardConflict = true; + } + if (patchRoutable.length > 0 && baseRoutable.length > 0) { + const patchFamilies = routingFamilies(patchRoutable); + const baseFamilies = routingFamilies(baseRoutable); + if ( + !patchFamilies.every((family) => + baseFamilies.some((baseFamily) => familyCompatible(family, baseFamily)), + ) + ) { + reasons.push("routing-family-conflict"); + } + if ( + !patchRoutable.every((source) => + baseRoutable.some((target) => source.tags?.["highway"] === target.tags?.["highway"]), + ) + ) { + reasons.push("routing-family-conflict"); + } + if ( + !patchRoutable.every((source) => + baseRoutable.some((target) => wayGradeAccessCompatible(source, target)), + ) + ) { + reasons.push("grade-conflict"); + hardConflict = true; + } + } else if ( + (patchRoutable.length > 0 && baseWays.length > 0) || + (baseRoutable.length > 0 && patchWays.length > 0) + ) { + reasons.push("non-routing-target"); + hardConflict = true; + } + assessment.reasons = uniqueReasons(reasons); + if (hardConflict) assessment.status = "blocked"; + else if (assessment.reasons.length > 0 && assessment.status === "automatic") { + assessment.status = "review"; + } + return assessment; +} + +function lineLength(coordinates: readonly LonLat[]) { + let total = 0; + for (let index = 1; index < coordinates.length; index++) { + total += haversineDistance(coordinates[index - 1]!, coordinates[index]!); + } + return total; +} + +function interpolate(a: LonLat, b: LonLat, parameter: number): LonLat { + return [a[0] + (b[0] - a[0]) * parameter, a[1] + (b[1] - a[1]) * parameter]; +} + +function sampleLine(coordinates: readonly LonLat[]) { + if (coordinates.length <= 1) return [...coordinates]; + const result: LonLat[] = [coordinates[0]!]; + for (let index = 1; index < coordinates.length; index++) { + const start = coordinates[index - 1]!; + const end = coordinates[index]!; + const length = haversineDistance(start, end); + const samples = Math.floor(length / SAMPLE_INTERVAL_METERS); + for (let sample = 1; sample <= samples; sample++) { + const distance = sample * SAMPLE_INTERVAL_METERS; + if (distance >= length) break; + result.push(interpolate(start, end, distance / length)); + } + result.push(end); + } + return result; +} + +function pointSegmentDistance(point: LonLat, start: LonLat, end: LonLat) { + const latitudeRadians = (point[1] * Math.PI) / 180; + const xScale = 111_320 * Math.cos(latitudeRadians); + const yScale = 110_574; + const startX = (start[0] - point[0]) * xScale; + const startY = (start[1] - point[1]) * yScale; + const endX = (end[0] - point[0]) * xScale; + const endY = (end[1] - point[1]) * yScale; + const dx = endX - startX; + const dy = endY - startY; + const denominator = dx * dx + dy * dy; + const parameter = + denominator === 0 ? 0 : Math.max(0, Math.min(1, -(startX * dx + startY * dy) / denominator)); + return Math.hypot(startX + parameter * dx, startY + parameter * dy); +} + +function pointLineDistance(point: LonLat, line: readonly LonLat[]) { + let minimum = Number.POSITIVE_INFINITY; + for (let index = 1; index < line.length; index++) { + minimum = Math.min(minimum, pointSegmentDistance(point, line[index - 1]!, line[index]!)); + } + return minimum; +} + +function symmetricLineDistance(a: readonly LonLat[], b: readonly LonLat[]) { + let maximum = 0; + for (const point of sampleLine(a)) maximum = Math.max(maximum, pointLineDistance(point, b)); + for (const point of sampleLine(b)) maximum = Math.max(maximum, pointLineDistance(point, a)); + return maximum; +} + +function wayCoordinates(osm: Osm, way: OsmWay) { + const index = osm.ways.ids.getIndexFromId(way.id); + return index < 0 ? [] : osm.ways.getResolvedCoordinates(index); +} + +function lineBbox( + coordinates: readonly LonLat[], + paddingMeters: number, +): [number, number, number, number] { + let minLon = Number.POSITIVE_INFINITY; + let minLat = Number.POSITIVE_INFINITY; + let maxLon = Number.NEGATIVE_INFINITY; + let maxLat = Number.NEGATIVE_INFINITY; + for (const [lon, lat] of coordinates) { + minLon = Math.min(minLon, lon); + minLat = Math.min(minLat, lat); + maxLon = Math.max(maxLon, lon); + maxLat = Math.max(maxLat, lat); + } + const middleLat = (minLat + maxLat) / 2; + const latPadding = paddingMeters / 110_574; + const lonPadding = + paddingMeters / (111_320 * Math.max(0.01, Math.cos((middleLat * Math.PI) / 180))); + return [minLon - lonPadding, minLat - latPadding, maxLon + lonPadding, maxLat + latPadding]; +} + +function bearing(from: LonLat, to: LonLat) { + const latitude1 = (from[1] * Math.PI) / 180; + const latitude2 = (to[1] * Math.PI) / 180; + const deltaLongitude = ((to[0] - from[0]) * Math.PI) / 180; + const y = Math.sin(deltaLongitude) * Math.cos(latitude2); + const x = + Math.cos(latitude1) * Math.sin(latitude2) - + Math.sin(latitude1) * Math.cos(latitude2) * Math.cos(deltaLongitude); + return ((Math.atan2(y, x) * 180) / Math.PI + 360) % 360; +} + +function undirectedBearingDifference(a: number, b: number) { + const directed = Math.abs(a - b) % 360; + return Math.min(directed, 360 - directed, Math.abs(180 - directed)); +} + +function nodeSegments(osm: Osm, nodeId: number, ways: readonly OsmWay[]) { + const node = osm.nodes.getById(nodeId); + if (!node) return []; + const segments: { bearing: number; way: OsmWay }[] = []; + for (const way of ways) { + for (let index = 0; index < way.refs.length; index++) { + if (way.refs[index] !== nodeId) continue; + for (const neighborIndex of [index - 1, index + 1]) { + const neighborId = way.refs[neighborIndex]; + if (neighborId == null || neighborId === nodeId) continue; + const neighbor = osm.nodes.getById(neighborId); + if (!neighbor) continue; + segments.push({ + bearing: bearing([node.lon, node.lat], [neighbor.lon, neighbor.lat]), + way, + }); + } + } + } + return segments; +} + +function nodeAttachmentAssessment( + context: DiscoveryContext, + source: OsmNode, + target: OsmNode, + patchWays: readonly OsmWay[], + baseWays: readonly OsmWay[], +): { assessment: OsmConflationActionAssessment; evidence: Partial<OsmConflationEvidence> } { + if (!context.options.attachNetwork) + return { assessment: { status: "blocked", reasons: [] }, evidence: {} }; + const sourceWays = patchWays.filter( + (way) => !context.base.ways.ids.has(way.id) && wayRoutingFamily(way) !== "non-routable", + ); + const targetWays = baseWays.filter((way) => wayRoutingFamily(way) !== "non-routable"); + if (sourceWays.length === 0 || targetWays.length === 0) { + return { + assessment: { status: "blocked", reasons: ["non-routing-target"] }, + evidence: { patchWayIds: sourceWays.map((way) => way.id).toSorted((a, b) => a - b) }, + }; + } + + // Hard reasons describe invariants a manual decision cannot override. Review + // reasons are plausible matches whose routing intent still needs a person. + const hardReasons: OsmConflationReasonCode[] = []; + const reviewReasons: OsmConflationReasonCode[] = []; + if (routingGradeSignature(source.tags) !== routingGradeSignature(target.tags)) { + hardReasons.push("grade-conflict"); + } + if (accessSignature(source.tags) !== accessSignature(target.tags)) { + hardReasons.push("routing-family-conflict"); + } + const sourceBarrier = barrierSignature(source.tags); + const targetBarrier = barrierSignature(target.tags); + if (sourceBarrier !== targetBarrier) hardReasons.push("routing-family-conflict"); + else if (sourceBarrier !== "") reviewReasons.push("node-context-conflict"); + if (nodeRoutingSignature(source.tags) !== nodeRoutingSignature(target.tags)) { + hardReasons.push("routing-family-conflict"); + } + if ( + ["layer", "level", "bridge", "tunnel", "covered"].some( + (key) => source.tags?.[key] != null || target.tags?.[key] != null, + ) + ) { + reviewReasons.push("node-context-conflict"); + } + const restrictionMember = + context.patchRelations.restrictionNodes.has(source.id) || + context.baseRelations.restrictionNodes.has(target.id) || + sourceWays.some((way) => context.patchRelations.restrictionWays.has(way.id)) || + targetWays.some((way) => context.baseRelations.restrictionWays.has(way.id)); + const relationMember = + context.patchRelations.nodes.has(source.id) || + context.baseRelations.nodes.has(target.id) || + sourceWays.some((way) => context.patchRelations.ways.has(way.id)) || + targetWays.some((way) => context.baseRelations.ways.has(way.id)); + if (restrictionMember) hardReasons.push("relation-member"); + else if (relationMember) reviewReasons.push("relation-member"); + + const sourceFamilies = routingFamilies(sourceWays); + const targetFamilies = routingFamilies(targetWays); + if ( + !sourceFamilies.every((family) => + targetFamilies.some((targetFamily) => familyCompatible(family, targetFamily)), + ) + ) { + reviewReasons.push("routing-family-conflict"); + } + if (sourceFamilies.includes("motor-road")) reviewReasons.push("drivable-network"); + if ( + !sourceWays.every((sourceWay) => + targetWays.some((targetWay) => sourceWay.tags?.["highway"] === targetWay.tags?.["highway"]), + ) + ) { + reviewReasons.push("routing-family-conflict"); + } + + const gradeCompatible = sourceWays.every((sourceWay) => + targetWays.some( + (targetWay) => + routingGradeSignature(sourceWay.tags) === routingGradeSignature(targetWay.tags) && + accessSignature(sourceWay.tags) === accessSignature(targetWay.tags), + ), + ); + if (!gradeCompatible) hardReasons.push("grade-conflict"); + + const sourceSegments = nodeSegments(context.patch, source.id, sourceWays); + const targetSegments = nodeSegments(context.base, target.id, targetWays); + let maximumMinimumBearingDifference = 0; + // Every imported incident segment needs at least one compatible base segment. + // Taking the worst best-match prevents one aligned arm from hiding another. + for (const sourceSegment of sourceSegments) { + const compatibleTargets = targetSegments.filter((targetSegment) => + wayContextsCompatible(sourceSegment.way, targetSegment.way), + ); + const minimum = compatibleTargets.reduce( + (value, targetSegment) => + Math.min(value, undirectedBearingDifference(sourceSegment.bearing, targetSegment.bearing)), + Number.POSITIVE_INFINITY, + ); + maximumMinimumBearingDifference = Math.max(maximumMinimumBearingDifference, minimum); + } + if ( + sourceSegments.length === 0 || + !Number.isFinite(maximumMinimumBearingDifference) || + maximumMinimumBearingDifference > MAX_BEARING_DIFFERENCE_DEGREES + ) { + reviewReasons.push("bearing-mismatch"); + } + + for (const way of sourceWays) { + const replacedRefs = way.refs.map((ref) => (ref === source.id ? target.id : ref)); + const adjacentDuplicate = replacedRefs.some( + (ref, index) => index > 0 && ref === replacedRefs[index - 1], + ); + if (adjacentDuplicate || new Set(replacedRefs).size < 2) hardReasons.push("would-collapse-way"); + } + + const reasons = uniqueReasons([...hardReasons, ...reviewReasons]); + const status = + hardReasons.length > 0 + ? "blocked" + : reviewReasons.length > 0 || context.options.automatic === "none" + ? "review" + : "automatic"; + return { + assessment: { status, reasons }, + evidence: { + patchWayIds: sourceWays.map((way) => way.id).toSorted((a, b) => a - b), + bearingDifferenceDegrees: Number.isFinite(maximumMinimumBearingDifference) + ? roundEvidence(maximumMinimumBearingDifference) + : undefined, + }, + }; +} + +function overallAssessment( + property: OsmConflationActionAssessment, + attachment: OsmConflationActionAssessment | null, + options: ResolvedOsmConflationOptions, +) { + const enabled = [ + ...(options.propertyKeys.length > 0 ? [property] : []), + ...(options.attachNetwork && attachment ? [attachment] : []), + ]; + const reasons = uniqueReasons(enabled.flatMap((assessment) => assessment.reasons)); + if (enabled.some((assessment) => assessment.status === "review")) { + return { status: "review" as const, reasons }; + } + if (enabled.some((assessment) => assessment.status === "automatic")) { + return { status: "automatic" as const, reasons }; + } + return { status: "blocked" as const, reasons }; +} + +function addReviewReason(candidate: OsmConflationCandidate, reason: OsmConflationReasonCode) { + for (const assessment of [candidate.propertyTransfer, candidate.networkAttachment]) { + if (!assessment || assessment.status === "blocked" || assessment.status === "unmatched") { + continue; + } + if (assessment.status === "automatic") assessment.status = "review"; + assessment.reasons = uniqueReasons([...assessment.reasons, reason]); + } + candidate.reasons = uniqueReasons([...candidate.reasons, reason]); + if (candidate.status === "automatic") candidate.status = "review"; +} + +function discoverNodeCandidates(context: DiscoveryContext) { + const candidates: OsmConflationCandidate[] = []; + for (const source of context.patch.nodes.sorted()) { + // Same-ID entities belong to ordinary merge semantics; fuzzy matching must not + // reinterpret an authoritative patch update. + if (context.base.nodes.ids.has(source.id)) continue; + const patchWays = context.patchWaysByNode.get(source.id) ?? []; + const eligible = + context.options.propertyKeys.some((key) => source.tags?.[key] != null) || + (context.options.attachNetwork && + patchWays.some((way) => !context.base.ways.ids.has(way.id))); + if (!eligible) continue; + + const nearby = context.base.nodes + .findIndexesWithinRadius(source.lon, source.lat, context.options.maxDistanceMeters / 1_000) + .map((index) => context.base.nodes.getByIndex(index)); + // A base ID also present in the patch is mutable under direct merge, so it is + // not an immutable target for a different imported entity. + const targets = nearby.filter((target) => !context.patch.nodes.ids.has(target.id)); + if (targets.length === 0) { + candidates.push({ + id: candidateId("node", source.id, null), + entityType: "node", + sourceId: source.id, + targetId: null, + status: "unmatched", + reasons: [], + propertyTransfer: { status: "unmatched", reasons: [] }, + networkAttachment: context.options.attachNetwork + ? { status: "unmatched", reasons: [] } + : null, + evidence: { + distanceMeters: Number.POSITIVE_INFINITY, + sourceRoutingFamilies: routingFamilies(patchWays), + targetRoutingFamilies: [], + tagDiff: [], + }, + }); + continue; + } + + for (const target of targets.toSorted((a, b) => a.id - b.id)) { + const baseWays = context.baseWaysByNode.get(target.id) ?? []; + const tagDiff = selectedTagDiff(source, target, context.options.propertyKeys); + const property = nodePropertyAssessment(context, patchWays, baseWays, tagDiff); + const attachment = nodeAttachmentAssessment(context, source, target, patchWays, baseWays); + if (targets.length > 1) { + if (property.status === "automatic") property.status = "review"; + if (attachment.assessment.status === "automatic") attachment.assessment.status = "review"; + property.reasons = uniqueReasons([...property.reasons, "multiple-targets"]); + attachment.assessment.reasons = uniqueReasons([ + ...attachment.assessment.reasons, + "multiple-targets", + ]); + } + const overall = overallAssessment(property, attachment.assessment, context.options); + const distanceMeters = haversineDistance([source.lon, source.lat], [target.lon, target.lat]); + candidates.push({ + id: candidateId("node", source.id, target.id), + entityType: "node", + sourceId: source.id, + targetId: target.id, + status: overall.status, + reasons: overall.reasons, + propertyTransfer: property, + networkAttachment: context.options.attachNetwork ? attachment.assessment : null, + evidence: { + distanceMeters: roundEvidence(distanceMeters), + sourceRoutingFamilies: routingFamilies(patchWays), + targetRoutingFamilies: routingFamilies(baseWays), + tagDiff, + ...attachment.evidence, + }, + }); + } + } + return candidates; +} + +function endpointDistances(source: readonly LonLat[], target: readonly LonLat[]) { + const forward: [number, number] = [ + haversineDistance(source[0]!, target[0]!), + haversineDistance(source.at(-1)!, target.at(-1)!), + ]; + const reverse: [number, number] = [ + haversineDistance(source[0]!, target.at(-1)!), + haversineDistance(source.at(-1)!, target[0]!), + ]; + return Math.max(...forward) <= Math.max(...reverse) + ? { distances: forward, reversed: false } + : { distances: reverse, reversed: true }; +} + +function discoverWayCandidates(context: DiscoveryContext) { + const candidates: OsmConflationCandidate[] = []; + if (context.options.propertyKeys.length === 0) return candidates; + for (const source of context.patch.ways.sorted()) { + if (context.base.ways.ids.has(source.id)) continue; + if (!context.options.propertyKeys.some((key) => source.tags?.[key] != null)) continue; + const sourceCoordinates = wayCoordinates(context.patch, source); + if (sourceCoordinates.length < 2) continue; + const nearbyIndexes = context.base.ways.intersects( + lineBbox(sourceCoordinates, context.options.maxDistanceMeters), + ); + const matches: { + target: OsmWay; + reasons: OsmConflationReasonCode[]; + evidence: Pick< + OsmConflationEvidence, + | "distanceMeters" + | "endpointDistancesMeters" + | "lengthDifferenceRatio" + | "maxGeometryDistanceMeters" + >; + }[] = []; + for (const index of nearbyIndexes) { + const target = context.base.ways.getByIndex(index); + if (context.patch.ways.ids.has(target.id)) continue; + const targetCoordinates = wayCoordinates(context.base, target); + if (targetCoordinates.length < 2) continue; + const endpoints = endpointDistances(sourceCoordinates, targetCoordinates); + if (Math.max(...endpoints.distances) > context.options.maxDistanceMeters) continue; + const sourceLength = lineLength(sourceCoordinates); + const targetLength = lineLength(targetCoordinates); + const maximumLength = Math.max(sourceLength, targetLength); + const lengthDifferenceRatio = + maximumLength === 0 ? 0 : Math.abs(sourceLength - targetLength) / maximumLength; + const maxGeometryDistanceMeters = symmetricLineDistance(sourceCoordinates, targetCoordinates); + if (maxGeometryDistanceMeters > context.options.maxDistanceMeters) continue; + const reasons: OsmConflationReasonCode[] = []; + // Keep geometrically plausible conflicts as blocked candidate rows. Users need + // to see why a nearby way was rejected instead of seeing it as merely unmatched. + if (isAreaWay(source) !== isAreaWay(target)) reasons.push("geometry-mismatch"); + if (lengthDifferenceRatio > MAX_LENGTH_DIFFERENCE_RATIO) reasons.push("length-mismatch"); + if (routingGradeSignature(source.tags) !== routingGradeSignature(target.tags)) { + reasons.push("grade-conflict"); + } + if ( + !familyCompatible(wayRoutingFamily(source), wayRoutingFamily(target)) || + accessSignature(source.tags) !== accessSignature(target.tags) || + !wayRoutingSemanticsCompatible(source, target, endpoints.reversed) + ) { + reasons.push("routing-family-conflict"); + } + matches.push({ + target, + reasons: uniqueReasons(reasons), + evidence: { + distanceMeters: roundEvidence(maxGeometryDistanceMeters), + endpointDistancesMeters: endpoints.distances.map(roundEvidence) as [number, number], + lengthDifferenceRatio: roundEvidence(lengthDifferenceRatio), + maxGeometryDistanceMeters: roundEvidence(maxGeometryDistanceMeters), + }, + }); + } + + if (matches.length === 0) { + // Multiple nearby base ways may represent a segmented equivalent. This version + // deliberately reports that case instead of guessing a one-to-many mapping. + const reasons: OsmConflationReasonCode[] = + nearbyIndexes.length > 1 ? ["unsupported-way-chain"] : []; + candidates.push({ + id: candidateId("way", source.id, null), + entityType: "way", + sourceId: source.id, + targetId: null, + status: "unmatched", + reasons, + propertyTransfer: { status: "unmatched", reasons }, + networkAttachment: null, + evidence: { + distanceMeters: Number.POSITIVE_INFINITY, + sourceRoutingFamilies: [wayRoutingFamily(source)], + targetRoutingFamilies: [], + tagDiff: [], + }, + }); + continue; + } + + for (const match of matches.toSorted((a, b) => a.target.id - b.target.id)) { + const tagDiff = selectedTagDiff(source, match.target, context.options.propertyKeys); + const property = propertyAssessment(tagDiff, context.options); + if (match.reasons.length > 0) { + property.status = "blocked"; + property.reasons = uniqueReasons([...property.reasons, ...match.reasons]); + } + if (matches.length > 1 && property.status === "automatic") property.status = "review"; + if (matches.length > 1) + property.reasons = uniqueReasons([...property.reasons, "multiple-targets"]); + const sourceRelation = context.patchRelations.ways.has(source.id); + const targetRelation = context.baseRelations.ways.has(match.target.id); + const restriction = + context.patchRelations.restrictionWays.has(source.id) || + context.baseRelations.restrictionWays.has(match.target.id); + if (sourceRelation || targetRelation) { + property.reasons = uniqueReasons([...property.reasons, "relation-member"]); + property.status = restriction ? "blocked" : "review"; + } + candidates.push({ + id: candidateId("way", source.id, match.target.id), + entityType: "way", + sourceId: source.id, + targetId: match.target.id, + status: property.status, + reasons: property.reasons, + propertyTransfer: property, + networkAttachment: null, + evidence: { + ...match.evidence, + sourceRoutingFamilies: [wayRoutingFamily(source)], + targetRoutingFamilies: [wayRoutingFamily(match.target)], + tagDiff, + }, + }); + } + } + return candidates; +} + +function applyManyToOneClassification(candidates: OsmConflationCandidate[]) { + // Candidate discovery is local to each source. Enforce the batch-wide one-to-one + // invariant only after all otherwise plausible pairs are known. + const sourcesByTarget = new Map<string, Set<number>>(); + for (const candidate of candidates) { + if (candidate.targetId == null) continue; + const key = `${candidate.entityType}:${candidate.targetId}`; + const sources = sourcesByTarget.get(key) ?? new Set(); + sources.add(candidate.sourceId); + sourcesByTarget.set(key, sources); + } + for (const candidate of candidates) { + if (candidate.targetId == null) continue; + if ((sourcesByTarget.get(`${candidate.entityType}:${candidate.targetId}`)?.size ?? 0) <= 1) + continue; + addReviewReason(candidate, "many-to-one"); + } +} + +/** Discover fuzzy candidates strictly between untouched patch and immutable base inputs. */ +export function discoverConflationCandidates( + base: Osm, + patch: Osm, + options: OsmConflationOptions, +): OsmConflationDiscovery { + const resolved = resolvedOptions(options); + const context: DiscoveryContext = { + base, + patch, + options: resolved, + baseWaysByNode: waysByNode(base), + patchWaysByNode: waysByNode(patch), + baseRelations: relationContext(base), + patchRelations: relationContext(patch), + }; + const candidates = [ + ...discoverNodeCandidates(context), + ...discoverWayCandidates(context), + ].toSorted( + (a, b) => + a.entityType.localeCompare(b.entityType) || + a.sourceId - b.sourceId || + (a.targetId ?? Number.POSITIVE_INFINITY) - (b.targetId ?? Number.POSITIVE_INFINITY), + ); + applyManyToOneClassification(candidates); + return { + baseOsmId: base.id, + patchOsmId: patch.id, + options: resolved, + candidates, + summary: summarizeConflationCandidates(candidates), + }; +} + +/** + * Discover canonical candidates for an in-process merge orchestrator. + * + * @internal This capability must stay inside a same-call merge path. Unlike the + * public generation functions, its companion generators trust the object + * identity registered here instead of rediscovering candidates from scratch. + */ +export function discoverConflationCandidatesForTrustedMerge( + base: Osm, + patch: Osm, + options: OsmConflationOptions, +) { + const discovery = discoverConflationCandidates(base, patch, options); + trustedDiscoveries.set(discovery, { base, patch }); + trustedCandidateCollections.add(discovery.candidates); + return discovery; +} + +function decisionMap(decisions: readonly OsmConflationDecision[]) { + return new Map(decisions.map((decision) => [decision.candidateId, decision])); +} + +function validatedDecisionMap( + candidates: readonly OsmConflationCandidate[], + decisions: readonly OsmConflationDecision[], +) { + if (!Array.isArray(decisions)) throw Error("Conflation decisions must be an array"); + if (decisions.length === 0) return new Map<string, OsmConflationDecision>(); + let candidateIds = trustedCandidateIds.get(candidates); + if (!candidateIds) { + candidateIds = new Set(candidates.map((candidate) => candidate.id)); + // General callers may mutate their candidate arrays between validations. + // Cache IDs only for canonical collections retained by a trusted merge path. + if (trustedCandidateCollections.has(candidates)) { + trustedCandidateIds.set(candidates, candidateIds); + } + } + const result = new Map<string, OsmConflationDecision>(); + for (const decision of decisions) { + if (decision == null || typeof decision !== "object") { + throw Error("Conflation decision must be an object"); + } + if (typeof decision.candidateId !== "string" || !candidateIds.has(decision.candidateId)) { + throw Error(`Unknown conflation candidate: ${String(decision.candidateId)}`); + } + if (result.has(decision.candidateId)) { + throw Error(`Duplicate conflation decision for ${decision.candidateId}`); + } + if (decision.action !== "accept" && decision.action !== "reject") { + throw Error(`Invalid conflation decision action for ${decision.candidateId}`); + } + if ( + decision.transferProperties !== undefined && + typeof decision.transferProperties !== "boolean" + ) { + throw Error(`Conflation transferProperties must be a boolean for ${decision.candidateId}`); + } + if (decision.attachNetwork !== undefined && typeof decision.attachNetwork !== "boolean") { + throw Error(`Conflation attachNetwork must be a boolean for ${decision.candidateId}`); + } + result.set(decision.candidateId, decision); + } + return result; +} + +/** Validate review decisions against canonical candidates without mutating either input. */ +export function validateConflationDecisions( + candidates: readonly OsmConflationCandidate[], + decisions: readonly OsmConflationDecision[], +) { + validatedDecisionMap(candidates, decisions); +} + +/** Return a candidate's effective status without rerunning spatial discovery. */ +export function conflationEffectiveStatus( + candidate: OsmConflationCandidate, + decisions: readonly OsmConflationDecision[] = [], +): OsmConflationEffectiveStatus { + const action = decisionMap(decisions).get(candidate.id)?.action; + if (action === "accept") return "accepted"; + if (action === "reject") return "rejected"; + return candidate.status; +} + +/** Recompute review counts after lightweight decisions without rerunning discovery. */ +export function summarizeConflationCandidates( + candidates: readonly OsmConflationCandidate[], + decisions: readonly OsmConflationDecision[] = [], +): OsmConflationSummary { + const decisionsById = validatedDecisionMap(candidates, decisions); + const summary: OsmConflationSummary = { + total: candidates.length, + accepted: 0, + automatic: 0, + review: 0, + blocked: 0, + unmatched: 0, + rejected: 0, + }; + for (const candidate of candidates) { + const action = decisionsById.get(candidate.id)?.action; + const status = + action === "accept" ? "accepted" : action === "reject" ? "rejected" : candidate.status; + summary[status]++; + } + return summary; +} + +/** Filter candidate rows deterministically, including effective rejected status. */ +export function filterConflationCandidates( + candidates: readonly OsmConflationCandidate[], + filter: OsmConflationCandidateFilter, + decisions: readonly OsmConflationDecision[] = [], +) { + const decisionsById = decisionMap(decisions); + return candidates.filter((candidate) => { + if (filter.entityType != null && candidate.entityType !== filter.entityType) return false; + const action = decisionsById.get(candidate.id)?.action; + const status = + action === "accept" ? "accepted" : action === "reject" ? "rejected" : candidate.status; + if (filter.status != null && status !== filter.status) { + return false; + } + if (filter.reason != null && !candidate.reasons.includes(filter.reason)) return false; + if (filter.sourceId != null && candidate.sourceId !== filter.sourceId) return false; + if ("targetId" in filter && candidate.targetId !== filter.targetId) return false; + return true; + }); +} + +const BULK_AMBIGUITY_REASONS = new Set<OsmConflationReasonCode>([ + "many-to-one", + "multiple-targets", + "unsupported-way-chain", +]); + +function bulkActionAssessment( + candidate: OsmConflationCandidate, + action: OsmConflationBulkDecisionRequest["action"], +) { + if (action === "transfer-properties") return candidate.propertyTransfer; + if (action === "attach-network") return candidate.networkAttachment; + return null; +} + +function bulkActionEligible( + candidate: OsmConflationCandidate, + action: OsmConflationBulkDecisionRequest["action"], +) { + if (action === "reject") return true; + if (candidate.status === "blocked" || candidate.status === "unmatched") return false; + if (candidate.targetId == null) return false; + if (candidate.reasons.some((reason) => BULK_AMBIGUITY_REASONS.has(reason))) return false; + const assessment = bulkActionAssessment(candidate, action); + if (!assessment || assessment.status === "blocked" || assessment.status === "unmatched") { + return false; + } + return action !== "transfer-properties" || candidate.evidence.tagDiff.length > 0; +} + +function bulkAcceptDecision( + candidate: OsmConflationCandidate, + current: OsmConflationDecision | undefined, + action: Exclude<OsmConflationBulkDecisionRequest["action"], "reject">, +): OsmConflationDecision { + const preserveCurrentActions = current?.action !== "reject"; + const transferProperties = + action === "transfer-properties" || + (preserveCurrentActions && acceptedAction(candidate, "propertyTransfer", current)); + const attachNetwork = + action === "attach-network" || + (preserveCurrentActions && acceptedAction(candidate, "networkAttachment", current)); + return { + candidateId: candidate.id, + action: "accept", + transferProperties, + attachNetwork, + }; +} + +function decisionsHaveSameEffect( + candidate: OsmConflationCandidate, + current: OsmConflationDecision | undefined, + next: OsmConflationDecision, +) { + if (!current || current.action !== next.action) return false; + if (current.action === "reject") return true; + return ( + acceptedAction(candidate, "propertyTransfer", current) === + acceptedAction(candidate, "propertyTransfer", next) && + acceptedAction(candidate, "networkAttachment", current) === + acceptedAction(candidate, "networkAttachment", next) + ); +} + +/** Build one atomic decision update for every candidate matching a filter. */ +export function buildConflationBulkDecisionResult( + candidates: readonly OsmConflationCandidate[], + decisions: readonly OsmConflationDecision[], + request: OsmConflationBulkDecisionRequest, +): OsmConflationBulkDecisionResult { + if (request == null || typeof request !== "object") { + throw Error("Conflation bulk decision request must be an object"); + } + if (!new Set(["transfer-properties", "attach-network", "reject"]).has(request.action)) { + throw Error(`Invalid conflation bulk action: ${String(request.action)}`); + } + if (request.filter == null || typeof request.filter !== "object") { + throw Error("Conflation bulk decision filter must be an object"); + } + + const currentById = validatedDecisionMap(candidates, decisions); + const nextById = new Map(currentById); + const filtered = filterConflationCandidates(candidates, request.filter, decisions); + let eligibleCandidates = 0; + let changedCandidates = 0; + let automaticCandidates = 0; + let reviewCandidates = 0; + let overriddenDecisions = 0; + + for (const candidate of filtered) { + if (!bulkActionEligible(candidate, request.action)) continue; + eligibleCandidates++; + if (candidate.status === "automatic") automaticCandidates++; + if (candidate.status === "review") reviewCandidates++; + + const current = currentById.get(candidate.id); + const next = + request.action === "reject" + ? ({ candidateId: candidate.id, action: "reject" } as const) + : bulkAcceptDecision(candidate, current, request.action); + if (decisionsHaveSameEffect(candidate, current, next)) continue; + changedCandidates++; + if (current) overriddenDecisions++; + nextById.set(candidate.id, next); + } + + const nextDecisions = [...nextById.values()].toSorted((a, b) => + a.candidateId.localeCompare(b.candidateId), + ); + validateConflationDecisions(candidates, nextDecisions); + const preview = { + action: request.action, + filteredCandidates: filtered.length, + eligibleCandidates, + changedCandidates, + skippedCandidates: filtered.length - eligibleCandidates, + automaticCandidates, + reviewCandidates, + overriddenDecisions, + }; + return { + decisions: nextDecisions, + preview, + summary: summarizeConflationCandidates(candidates, nextDecisions), + }; +} + +function currentEntity<T extends "node" | "way">(changeset: OsmChangeset, type: T, id: number) { + const change = changeset.changes(type)[id]; + if (change?.changeType === "delete") return null; + return change?.entity ?? changeset.getEntity(type, id) ?? null; +} + +function currentRelations(changeset: OsmChangeset) { + const relations = new Map<number, OsmRelation>(); + for (const relation of changeset.osm.relations) { + const change = changeset.relationChanges[relation.id]; + if (change?.changeType !== "delete") relations.set(relation.id, change?.entity ?? relation); + } + for (const change of Object.values(changeset.relationChanges)) { + if (change.changeType === "delete") relations.delete(change.entity.id); + else relations.set(change.entity.id, change.entity); + } + return relations.values(); +} + +function currentWays(changeset: OsmChangeset) { + const ways = new Map<number, OsmWay>(); + for (const way of changeset.osm.ways) { + const change = changeset.wayChanges[way.id]; + if (change?.changeType !== "delete") ways.set(way.id, change?.entity ?? way); + } + for (const change of Object.values(changeset.wayChanges)) { + if (change.changeType === "delete") ways.delete(change.entity.id); + else ways.set(change.entity.id, change.entity); + } + return ways.values(); +} + +function removeCurrentEntity(changeset: OsmChangeset, entity: OsmEntity) { + const type = "lon" in entity ? "node" : "refs" in entity ? "way" : "relation"; + const change = changeset.changes(type)[entity.id]; + if (change?.changeType === "create") delete changeset.changes(type)[entity.id]; + else changeset.delete(entity); +} + +function acceptedAction( + candidate: OsmConflationCandidate, + action: "propertyTransfer" | "networkAttachment", + decision: OsmConflationDecision | undefined, +) { + if (decision?.action === "reject") return false; + const assessment = candidate[action]; + // Manual review can select among reviewable actions, but it cannot override a + // blocked invariant or manufacture a match for an unmatched candidate. + if (!assessment || assessment.status === "blocked" || assessment.status === "unmatched") + return false; + const selected = + action === "propertyTransfer" ? decision?.transferProperties : decision?.attachNetwork; + if (decision?.action === "accept") return selected ?? true; + return assessment.status === "automatic"; +} + +function transferSelectedProperties( + changeset: OsmChangeset, + candidate: OsmConflationCandidate, + source: OsmEntity, +) { + if (candidate.targetId == null) return; + const type = candidate.entityType; + changeset.modify(type, candidate.targetId, (target) => { + const tags = { ...target.tags }; + for (const diff of candidate.evidence.tagDiff) { + if (diff.protected) continue; + tags[diff.key] = source.tags![diff.key]!; + } + return { ...target, tags }; + }); +} + +function validateAcceptedMappings( + candidates: readonly OsmConflationCandidate[], + decisions: ReadonlyMap<string, OsmConflationDecision>, +) { + const sourceActions = new Set<string>(); + const attachmentTargets = new Set<number>(); + const wayTargets = new Set<number>(); + for (const candidate of candidates) { + const decision = decisions.get(candidate.id); + const transfer = acceptedAction(candidate, "propertyTransfer", decision); + const attach = acceptedAction(candidate, "networkAttachment", decision); + if (!transfer && !attach) continue; + const sourceKey = `${candidate.entityType}:${candidate.sourceId}`; + if (sourceActions.has(sourceKey)) { + throw Error(`Conflation accepted multiple targets for ${sourceKey}`); + } + sourceActions.add(sourceKey); + if (candidate.targetId == null) + throw Error(`Conflation accepted unmatched candidate ${candidate.id}`); + if (attach) { + if (attachmentTargets.has(candidate.targetId)) { + throw Error(`Conflation accepted multiple node attachments to ${candidate.targetId}`); + } + attachmentTargets.add(candidate.targetId); + } + if (candidate.entityType === "way" && transfer) { + if (wayTargets.has(candidate.targetId)) { + throw Error(`Conflation accepted multiple ways for target ${candidate.targetId}`); + } + wayTargets.add(candidate.targetId); + } + } +} + +function cleanupUnreferencedPatchNodes( + changeset: OsmChangeset, + patch: Osm, + originalBase: Osm, + cleanupCandidateIds: ReadonlySet<number>, +) { + // Cleanup is intentionally limited to nodes from a suppressed matched patch way. + // Removing every orphan patch node would violate direct merge preservation. + const referenced = new Set<number>(); + for (const way of currentWays(changeset)) for (const ref of way.refs) referenced.add(ref); + for (const relation of currentRelations(changeset)) { + for (const member of relation.members) if (member.type === "node") referenced.add(member.ref); + } + for (const nodeId of cleanupCandidateIds) { + const node = patch.nodes.getById(nodeId); + if (!node) continue; + if (originalBase.nodes.ids.has(node.id) || node.tags != null || referenced.has(node.id)) + continue; + const current = currentEntity(changeset, "node", node.id); + if (current) removeCurrentEntity(changeset, current); + } +} + +function applyDiscoveredConflation( + changeset: OsmChangeset, + patch: Osm, + discovery: OsmConflationDiscovery, + decisions: readonly OsmConflationDecision[], + originalBase: Osm, +) { + if (patch.id !== discovery.patchOsmId) { + throw Error(`Conflation discovery patch ${discovery.patchOsmId} does not match ${patch.id}`); + } + const decisionsById = validatedDecisionMap(discovery.candidates, decisions); + validateAcceptedMappings(discovery.candidates, decisionsById); + + const attachments = new Map<number, number>(); + const patchWayIds = new Set<number>(); + const cleanupCandidateNodeIds = new Set<number>(); + for (const candidate of discovery.candidates) { + const decision = decisionsById.get(candidate.id); + if (!acceptedAction(candidate, "networkAttachment", decision) || candidate.targetId == null) { + continue; + } + attachments.set(candidate.sourceId, candidate.targetId); + for (const wayId of candidate.evidence.patchWayIds ?? []) patchWayIds.add(wayId); + } + for (const wayId of patchWayIds) { + // Only patch-created ways are listed in attachment evidence. Base way refs are + // never rewritten, even when the nearby patch node is accepted. + const way = currentEntity(changeset, "way", wayId); + if (!way) continue; + const refs = way.refs.map((ref) => attachments.get(ref) ?? ref); + if (refs.some((ref, index) => index > 0 && ref === refs[index - 1])) { + throw Error(`Conflation attachment would create duplicate adjacent refs in way ${wayId}`); + } + if (way.tags?.["highway"] != null && new Set(refs).size < 2) { + throw Error(`Conflation attachment would collapse highway way ${wayId}`); + } + changeset.modify("way", wayId, (current) => ({ ...current, refs })); + } + + for (const candidate of discovery.candidates) { + const decision = decisionsById.get(candidate.id); + if (!acceptedAction(candidate, "propertyTransfer", decision) || candidate.targetId == null) { + continue; + } + const source = + candidate.entityType === "node" + ? patch.nodes.getById(candidate.sourceId) + : patch.ways.getById(candidate.sourceId); + if (!source) + throw Error(`Conflation source ${candidate.entityType} ${candidate.sourceId} is missing`); + transferSelectedProperties(changeset, candidate, source); + if ( + candidate.entityType !== "way" || + candidate.reasons.includes("relation-member") || + candidate.reasons.includes("protected-tag") + ) { + continue; + } + const current = currentEntity(changeset, "way", candidate.sourceId); + if (current) { + // An equivalent one-to-one patch way is suppressed after property transfer. + // Its nodes become cleanup candidates, not unconditional deletions. + for (const ref of current.refs) cleanupCandidateNodeIds.add(ref); + removeCurrentEntity(changeset, current); + } + } + cleanupUnreferencedPatchNodes(changeset, patch, originalBase, cleanupCandidateNodeIds); +} + +function generateConflationApplicationArtifacts( + baseline: Osm, + patch: Osm, + canonicalDiscovery: OsmConflationDiscovery, + originalBase: Osm, + decisions: readonly OsmConflationDecision[] = [], +) { + if (patch.id !== canonicalDiscovery.patchOsmId) { + throw Error( + `Conflation discovery patch ${canonicalDiscovery.patchOsmId} does not match ${patch.id}`, + ); + } + if (originalBase.id !== canonicalDiscovery.baseOsmId) { + throw Error( + `Conflation discovery base ${canonicalDiscovery.baseOsmId} does not match ${originalBase.id}`, + ); + } + const changeset = new OsmChangeset(baseline); + applyDiscoveredConflation(changeset, patch, canonicalDiscovery, decisions, originalBase); + const result = applyChangesetToOsm(changeset); + assertConflationPreservesBaseTopology(originalBase, baseline, result); + return { changeset, result }; +} + +/** Generate fuzzy-only changes over an already applied ordinary direct/exact merge baseline. */ +export function generateConflationApplicationChangeset( + baseline: Osm, + patch: Osm, + discovery: OsmConflationDiscovery, + originalBase: Osm, + decisions: readonly OsmConflationDecision[] = [], +) { + // Reject review data from another merge session before rediscovery. The + // candidate evidence is deliberately untrusted, but its input IDs are still + // part of the public API's stale-session guard. + if (patch.id !== discovery.patchOsmId) { + throw Error(`Conflation discovery patch ${discovery.patchOsmId} does not match ${patch.id}`); + } + if (originalBase.id !== discovery.baseOsmId) { + throw Error( + `Conflation discovery base ${discovery.baseOsmId} does not match ${originalBase.id}`, + ); + } + // Recompute from untouched entities before applying. Candidate records returned + // to callers are review data, not trusted instructions for mutating topology. + const canonicalDiscovery = discoverConflationCandidates(originalBase, patch, discovery.options); + return generateConflationApplicationArtifacts( + baseline, + patch, + canonicalDiscovery, + originalBase, + decisions, + ).changeset; +} + +function validateCumulativeConflationOptions( + base: Osm, + patch: Osm, + options: Partial<OsmMergeOptions>, + discovery: OsmConflationDiscovery, +) { + if (!options.conflation) throw Error("generateConflationChangeset requires conflation options"); + if (!options.directMerge) + throw Error("Fuzzy conflation requires directMerge to preserve unmatched patch entities"); + if (options.createIntersections) { + throw Error( + "generateConflationChangeset cannot create intersections in the cumulative changeset", + ); + } + if (discovery.baseOsmId !== base.id || discovery.patchOsmId !== patch.id) { + throw Error("Conflation discovery does not match the untouched merge inputs"); + } + const expectedOptions = resolvedOptions(options.conflation); + if ( + discovery.options.attachNetwork !== expectedOptions.attachNetwork || + discovery.options.automatic !== expectedOptions.automatic || + discovery.options.maxDistanceMeters !== expectedOptions.maxDistanceMeters || + discovery.options.propertyKeys.length !== expectedOptions.propertyKeys.length || + discovery.options.propertyKeys.some((key, index) => key !== expectedOptions.propertyKeys[index]) + ) { + throw Error("Conflation discovery options do not match generation options"); + } +} + +function generateCumulativeConflationArtifacts( + base: Osm, + patch: Osm, + options: Partial<OsmMergeOptions>, + decisions: readonly OsmConflationDecision[], + canonicalDiscovery: OsmConflationDiscovery, + onProgress?: (progress: ProgressEvent) => void, +) { + validateCumulativeConflationOptions(base, patch, options, canonicalDiscovery); + const ordinaryOptions = { + directMerge: true, + deduplicateNodes: options.deduplicateNodes ?? false, + deduplicateWays: options.deduplicateWays ?? false, + createIntersections: false, + }; + // Applying does not consume a changeset. Build the ordinary changes once, use + // them to materialize the comparison baseline, then add fuzzy changes to that + // same cumulative changeset. + const changeset = onProgress + ? generateChangeset(base, patch, ordinaryOptions, onProgress) + : generateChangeset(base, patch, ordinaryOptions); + const ordinaryBaseline = applyChangesetToOsm(changeset); + applyDiscoveredConflation(changeset, patch, canonicalDiscovery, decisions, base); + const result = applyChangesetToOsm(changeset); + assertConflationPreservesBaseTopology(base, ordinaryBaseline, result); + return { changeset, ordinaryBaseline, result }; +} + +/** + * Generate a cumulative direct/exact/fuzzy changeset from untouched inputs. + * Intersection creation remains a later stage because newly created ways are not indexed yet. + */ +export function generateConflationChangeset( + base: Osm, + patch: Osm, + options: Partial<OsmMergeOptions>, + decisions: readonly OsmConflationDecision[] = options.conflation?.decisions ?? [], + discovery?: OsmConflationDiscovery, +) { + if (!options.conflation) throw Error("generateConflationChangeset requires conflation options"); + if (!options.directMerge) + throw Error("Fuzzy conflation requires directMerge to preserve unmatched patch entities"); + if (options.createIntersections) { + throw Error( + "generateConflationChangeset cannot create intersections in the cumulative changeset", + ); + } + // Generation never trusts possibly stale or caller-mutated candidate evidence. + // Stable decisions are replayed against a fresh discovery from untouched inputs. + const canonicalDiscovery = discoverConflationCandidates(base, patch, options.conflation); + const suppliedDiscovery = discovery ?? canonicalDiscovery; + // Validate the supplied review snapshot even though the fresh canonical + // discovery remains the only source of mutation instructions. + validateCumulativeConflationOptions(base, patch, options, suppliedDiscovery); + return generateCumulativeConflationArtifacts(base, patch, options, decisions, canonicalDiscovery) + .changeset; +} + +/** + * Generate cumulative artifacts from a canonical same-process discovery. + * + * @internal Public generation must use {@link generateConflationChangeset}, which + * deliberately rediscovers candidates before applying caller-supplied decisions. + */ +export function generateConflationArtifactsFromTrustedDiscovery( + base: Osm, + patch: Osm, + options: Partial<OsmMergeOptions>, + decisions: readonly OsmConflationDecision[], + discovery: OsmConflationDiscovery, + onProgress: (progress: ProgressEvent) => void, +) { + const inputs = trustedDiscoveries.get(discovery); + if (inputs?.base !== base || inputs.patch !== patch) { + throw Error("Conflation discovery is not owned by this trusted merge session"); + } + return generateCumulativeConflationArtifacts( + base, + patch, + options, + decisions, + discovery, + onProgress, + ); +} + +/** + * Generate fuzzy-only artifacts from a canonical same-process discovery. + * + * @internal Used for the automatic network-attachment CAR safety projection. + */ +export function generateConflationApplicationArtifactsFromTrustedDiscovery( + baseline: Osm, + patch: Osm, + discovery: OsmConflationDiscovery, + originalBase: Osm, + decisions: readonly OsmConflationDecision[], +) { + const inputs = trustedDiscoveries.get(discovery); + if (inputs?.base !== originalBase || inputs.patch !== patch) { + throw Error("Conflation discovery is not owned by this trusted merge session"); + } + return generateConflationApplicationArtifacts( + baseline, + patch, + discovery, + originalBase, + decisions, + ); +} diff --git a/packages/change/src/generate-changeset.ts b/packages/change/src/generate-changeset.ts index 4f1a4a1a..cdc806a4 100644 --- a/packages/change/src/generate-changeset.ts +++ b/packages/change/src/generate-changeset.ts @@ -25,6 +25,9 @@ import type { OsmMergeOptions } from "./types.ts"; * @param options - Options controlling which operations to run. * @param onProgress - Callback for progress updates (throttled for way operations). * @returns The populated OsmChangeset ready for application or inspection. + * @throws When direct merge and intersection creation are requested together. Newly created + * patch ways are not spatially indexed until the direct changeset is applied; use `merge()` or + * apply the direct changes before generating an intersection-only changeset. * * @example * ```ts @@ -42,6 +45,12 @@ export function generateChangeset( options: Partial<OsmMergeOptions> = {}, onProgress: (progress: ProgressEvent) => void = logProgress, ) { + if (options.directMerge && options.createIntersections) { + throw Error( + "generateChangeset cannot combine directMerge with createIntersections because new patch ways are not indexed; use merge() or apply direct changes before generating intersections", + ); + } + const patchId = patch.id; const baseId = base.id; @@ -55,38 +64,39 @@ export function generateChangeset( changeset.generateDirectChanges(patch); } + if (options.deduplicateNodes) { + log(`Reconciling nodes from ${patchId} with ${baseId}...`); + changeset.deduplicateNodes(patch.nodes); + log( + `Node deduplication results: ${changeset.deduplicatedNodes} de-duplicated nodes, ${changeset.deduplicatedNodesReplaced} nodes replaced`, + ); + } + if (options.deduplicateWays) { let checkedWays = 0; let dedpulicatedWays = 0; - log(`Deduplicating ways from ${patchId}...`); + log(`Reconciling ways from ${patchId} with ${baseId}...`); for (const wayStats of changeset.deduplicateWaysGenerator(patch.ways)) { checkedWays++; dedpulicatedWays += wayStats; logEverySecond( - `Deduplicating ways: ${checkedWays.toLocaleString()} ways checked, ${dedpulicatedWays.toLocaleString()} ways deduplicated`, + `Way reconciliation: ${checkedWays.toLocaleString()} ways checked, ${dedpulicatedWays.toLocaleString()} ways reconciled`, ); } } - if (options.deduplicateNodes) { - log(`Deduplicating nodes from ${patchId}...`); - changeset.deduplicateNodes(patch.nodes); - log( - `Node deduplication results: ${changeset.deduplicatedNodes} de-duplicated nodes, ${changeset.deduplicatedNodesReplaced} nodes replaced`, - ); - } - if (options.createIntersections) { let checkedWays = 0; log(`Creating intersections from ${patchId}...`); + const progressMessage = () => + `Intersection creation progress: ${checkedWays.toLocaleString()} of ${patch.ways.size.toLocaleString()} ways checked`; // This will check if the osm dataset has the way before trying to create intersections for it. for (const _wayStats of changeset.createIntersectionsForWaysGenerator(patch.ways)) { checkedWays++; - logEverySecond( - `Intersection creation progress: ${checkedWays.toLocaleString()} ways checked`, - ); + logEverySecond(progressMessage()); } + if (checkedWays > 0) log(progressMessage()); } return changeset; diff --git a/packages/change/src/index.ts b/packages/change/src/index.ts index e70489cc..2522d68f 100644 --- a/packages/change/src/index.ts +++ b/packages/change/src/index.ts @@ -18,8 +18,8 @@ * * // Manual changeset workflow * const changeset = new OsmChangeset(baseOsm) - * changeset.deduplicateNodes(baseOsm.nodes) * changeset.generateDirectChanges(patchOsm) + * changeset.deduplicateNodes(patchOsm.nodes) * const merged = applyChangesToOsm(changeset) * * // Or use the high-level merge function @@ -34,6 +34,16 @@ export * from "./apply-changeset.ts"; export * from "./changeset.ts"; +export { + buildConflationBulkDecisionResult, + conflationEffectiveStatus, + discoverConflationCandidates, + filterConflationCandidates, + generateConflationApplicationChangeset, + generateConflationChangeset, + summarizeConflationCandidates, + validateConflationDecisions, +} from "./conflation.ts"; export * from "./generate-changeset.ts"; export * from "./merge.ts"; export * from "./osc.ts"; diff --git a/packages/change/src/integrity.ts b/packages/change/src/integrity.ts new file mode 100644 index 00000000..e5e5c06a --- /dev/null +++ b/packages/change/src/integrity.ts @@ -0,0 +1,329 @@ +import type { Osm } from "@osmix/core"; +import type { OsmRelation, OsmWay } from "@osmix/types"; + +import { routingGradeSignature } from "./utils.ts"; + +type IntegrityIssue = { + key: string; + description: string; +}; + +type IncidentHighway = { + way: OsmWay; + gradeSignature: string; + interior: boolean; + endpoint: boolean; +}; + +// Finalized Osm indexes are immutable. Keep their ordered analysis by object +// identity so adjacent merge stages do not rescan the same million-entity +// dataset. Never key this cache by the user-facing OSM ID: an ID may be reused +// for a newly merged dataset with different contents. +const routingIntegrityIssuesByOsm = new WeakMap<Osm, readonly IntegrityIssue[]>(); + +const SURFACE_GRADE_SIGNATURE = "layer=0|level=|bridge=no|tunnel=no|covered=no"; + +function sharesNode(a: OsmWay, b: OsmWay) { + const aRefs = new Set(a.refs); + return b.refs.some((ref) => aRefs.has(ref)); +} + +/** + * A bridge or tunnel can legitimately terminate at a portal node on a surface network. + * When an interior way also touches that portal (for example a crossing footway), the + * same-grade endpoint continuation proves that the interior way is connected to the + * surface side, not spliced into the grade-separated segment. + */ +function hasSameGradeEndpointContinuation( + ways: readonly IncidentHighway[], + left: IncidentHighway, + right: IncidentHighway, +) { + if (left.interior === right.interior) return false; + + const interiorWay = left.interior ? left : right; + const interiorSignature = interiorWay.gradeSignature; + // A continuation only proves a normal portal when the interior way is on the + // default surface level. It must not legitimize a new surface endpoint spliced + // into the middle of a tunnel or bridge. + if (interiorSignature !== SURFACE_GRADE_SIGNATURE) return false; + return ways.some( + (candidate) => + candidate.way.id !== left.way.id && + candidate.way.id !== right.way.id && + candidate.endpoint && + candidate.gradeSignature === interiorSignature, + ); +} + +function isAbsoluteIntegrityIssue(issue: IntegrityIssue) { + return ( + /^way:[^:]+:missing-node:/.test(issue.key) || + /^way:[^:]+:degenerate-highway$/.test(issue.key) || + /^relation:[^:]+:missing-/.test(issue.key) + ); +} + +function restrictionIssues(osm: Osm, relation: OsmRelation): IntegrityIssue[] { + if (relation.tags?.["type"] !== "restriction") return []; + + const issues: IntegrityIssue[] = []; + const fromWays = relation.members + .filter((member) => member.type === "way" && member.role === "from") + .map((member) => osm.ways.getById(member.ref)) + .filter((way): way is OsmWay => way != null); + const toWays = relation.members + .filter((member) => member.type === "way" && member.role === "to") + .map((member) => osm.ways.getById(member.ref)) + .filter((way): way is OsmWay => way != null); + const viaNodes = relation.members.filter( + (member) => member.type === "node" && member.role === "via", + ); + const viaWays = relation.members + .filter((member) => member.type === "way" && member.role === "via") + .map((member) => osm.ways.getById(member.ref)) + .filter((way): way is OsmWay => way != null); + + if (fromWays.length === 0) { + issues.push({ + key: `restriction:${relation.id}:missing-from`, + description: `restriction ${relation.id} has no existing from way`, + }); + } + if (toWays.length === 0) { + issues.push({ + key: `restriction:${relation.id}:missing-to`, + description: `restriction ${relation.id} has no existing to way`, + }); + } + if (viaNodes.length === 0 && viaWays.length === 0) { + issues.push({ + key: `restriction:${relation.id}:missing-via`, + description: `restriction ${relation.id} has no existing via member`, + }); + } + + for (const viaNode of viaNodes) { + const belongsToFrom = fromWays.some((way) => way.refs.includes(viaNode.ref)); + const belongsToTo = toWays.some((way) => way.refs.includes(viaNode.ref)); + if (!belongsToFrom || !belongsToTo) { + issues.push({ + key: `restriction:${relation.id}:detached-via-node:${viaNode.ref}`, + description: `restriction ${relation.id} via node ${viaNode.ref} is detached from its from/to ways`, + }); + } + } + + if (viaWays.length > 0 && fromWays.length > 0 && toWays.length > 0) { + const connectedFrom = fromWays.some((way) => sharesNode(way, viaWays[0]!)); + const connectedTo = toWays.some((way) => sharesNode(viaWays.at(-1)!, way)); + const connectedChain = viaWays.every( + (way, index) => index === 0 || sharesNode(viaWays[index - 1]!, way), + ); + if (!connectedFrom || !connectedChain || !connectedTo) { + issues.push({ + key: `restriction:${relation.id}:detached-via-way-chain`, + description: `restriction ${relation.id} has a disconnected via-way chain`, + }); + } + } + + return issues; +} + +function collectRoutingIntegrityIssues(osm: Osm): readonly IntegrityIssue[] { + const cachedIssues = routingIntegrityIssuesByOsm.get(osm); + if (cachedIssues) return cachedIssues; + + const issues: IntegrityIssue[] = []; + const highwayWaysByNode = new Map<number, IncidentHighway[]>(); + + for (const way of osm.ways) { + for (const ref of way.refs) { + if (osm.nodes.ids.has(ref)) continue; + issues.push({ + key: `way:${way.id}:missing-node:${ref}`, + description: `way ${way.id} references missing node ${ref}`, + }); + } + const distinctRefs = new Set(way.refs); + if (way.tags?.["highway"] != null && distinctRefs.size < 2) { + issues.push({ + key: `way:${way.id}:degenerate-highway`, + description: `highway way ${way.id} has fewer than two distinct nodes`, + }); + } + if (way.tags?.["highway"] != null) { + const gradeSignature = routingGradeSignature(way.tags); + const interiorRefs = new Set(way.refs.slice(1, -1)); + const endpointRefs = new Set([way.refs[0], way.refs.at(-1)]); + for (const ref of distinctRefs) { + const incidentWays = highwayWaysByNode.get(ref) ?? []; + incidentWays.push({ + way, + gradeSignature, + interior: interiorRefs.has(ref), + endpoint: endpointRefs.has(ref), + }); + highwayWaysByNode.set(ref, incidentWays); + } + } + } + + for (const [nodeId, ways] of highwayWaysByNode) { + for (let leftIndex = 0; leftIndex < ways.length; leftIndex++) { + for (let rightIndex = leftIndex + 1; rightIndex < ways.length; rightIndex++) { + const left = ways[leftIndex]!; + const right = ways[rightIndex]!; + if (left.gradeSignature === right.gradeSignature) continue; + if (!left.interior && !right.interior) continue; + if (hasSameGradeEndpointContinuation(ways, left, right)) continue; + const [firstWayId, secondWayId] = [left.way.id, right.way.id].toSorted((a, b) => a - b); + issues.push({ + key: `node:${nodeId}:incompatible-grade:${firstWayId}:${secondWayId}`, + description: `node ${nodeId} newly connects grade-separated highways ${firstWayId} and ${secondWayId}`, + }); + } + } + } + + for (const relation of osm.relations) { + for (const member of relation.members) { + const exists = + member.type === "node" + ? osm.nodes.ids.has(member.ref) + : member.type === "way" + ? osm.ways.ids.has(member.ref) + : osm.relations.ids.has(member.ref); + if (exists) continue; + issues.push({ + key: `relation:${relation.id}:missing-${member.type}:${member.ref}`, + description: `relation ${relation.id} references missing ${member.type} ${member.ref}`, + }); + } + issues.push(...restrictionIssues(osm, relation)); + } + + if (osm.isReady()) routingIntegrityIssuesByOsm.set(osm, issues); + return issues; +} + +export function routingIntegrityIssueKeys(osm: Osm) { + return new Set(collectRoutingIntegrityIssues(osm).map((issue) => issue.key)); +} + +/** Reuse analysis only when two finalized wrappers reference identical entity buffers. */ +export function reuseRoutingIntegrityAnalysis(source: Osm, target: Osm) { + const issues = collectRoutingIntegrityIssues(source); + if (target.isReady()) routingIntegrityIssuesByOsm.set(target, issues); +} + +/** + * Combine inherited issues from both inputs while treating same-ID patch entities as + * modifications that must remain valid when their base counterpart was valid. + */ +export function inheritedRoutingIntegrityIssueKeys( + base: Osm, + patch: Osm, + baseKeys: ReadonlySet<string> = routingIntegrityIssueKeys(base), +) { + const keys = new Set(baseKeys); + for (const issue of collectRoutingIntegrityIssues(patch)) { + // Missing references and degenerate highways in a patch are never inherited: + // accepting them would allow malformed input to pass through unchanged. + if (isAbsoluteIntegrityIssue(issue)) continue; + const [kind, idText] = issue.key.split(":"); + // Restriction topology must be evaluated in the merged entity context. A patch + // relation may legitimately reference base ways, so its patch-only issue is not + // evidence of a pre-existing defect and must never suppress merged validation. + if (kind === "restriction") continue; + const id = Number(idText); + const collidesWithBase = + kind === "node" + ? base.nodes.ids.has(id) + : kind === "way" + ? base.ways.ids.has(id) + : kind === "relation" || kind === "restriction" + ? base.relations.ids.has(id) + : false; + if (!collidesWithBase) keys.add(issue.key); + } + return keys; +} + +/** Throw when a merge introduces routing-integrity issues not present in the base dataset. */ +export function assertNoNewRoutingIntegrityIssues(baselineKeys: ReadonlySet<string>, merged: Osm) { + const newIssues = collectRoutingIntegrityIssues(merged).filter( + (issue) => !baselineKeys.has(issue.key), + ); + if (newIssues.length === 0) return; + + const descriptions = newIssues.slice(0, 10).map((issue) => issue.description); + const omitted = newIssues.length - descriptions.length; + const suffix = omitted > 0 ? `; and ${omitted} more` : ""; + throw Error(`Merge introduced routing-integrity problems: ${descriptions.join("; ")}${suffix}`); +} + +/** + * Ensure fuzzy conflation did not rewrite geometry or relation topology that already existed in + * the base. Same-ID patch updates are compared at the ordinary-merge baseline, not the raw base. + */ +export function assertConflationPreservesBaseTopology( + originalBase: Osm, + ordinaryBaseline: Osm, + conflated: Osm, +) { + const violations: string[] = []; + for (const original of originalBase.nodes) { + const baseline = ordinaryBaseline.nodes.getById(original.id); + const result = conflated.nodes.getById(original.id); + if (!baseline || !result) { + violations.push(`base node ${original.id} was removed`); + continue; + } + if (baseline.lon !== result.lon || baseline.lat !== result.lat) { + violations.push(`base node ${original.id} coordinates changed`); + } + } + for (const original of originalBase.ways) { + const baseline = ordinaryBaseline.ways.getById(original.id); + const result = conflated.ways.getById(original.id); + if (!baseline || !result) { + violations.push(`base way ${original.id} was removed`); + continue; + } + if ( + baseline.refs.length !== result.refs.length || + baseline.refs.some((ref, index) => ref !== result.refs[index]) + ) { + violations.push(`base way ${original.id} references changed`); + } + } + for (const original of originalBase.relations) { + const baseline = ordinaryBaseline.relations.getById(original.id); + const result = conflated.relations.getById(original.id); + if (!baseline || !result) { + violations.push(`base relation ${original.id} was removed`); + continue; + } + if ( + baseline.members.length !== result.members.length || + baseline.members.some((member, index) => { + const resultMember = result.members[index]; + return ( + !resultMember || + member.type !== resultMember.type || + member.ref !== resultMember.ref || + member.role !== resultMember.role + ); + }) + ) { + violations.push(`base relation ${original.id} members changed`); + } + } + if (violations.length === 0) return; + const descriptions = violations.slice(0, 10); + const omitted = violations.length - descriptions.length; + const suffix = omitted > 0 ? `; and ${omitted} more` : ""; + throw Error(`Conflation changed protected base topology: ${descriptions.join("; ")}${suffix}`); +} diff --git a/packages/change/src/internal/conflation.ts b/packages/change/src/internal/conflation.ts new file mode 100644 index 00000000..222e8037 --- /dev/null +++ b/packages/change/src/internal/conflation.ts @@ -0,0 +1,15 @@ +/** + * Same-process conflation capabilities for trusted merge orchestrators. + * + * This module is deliberately absent from the package entry point. Its callers + * own both untouched inputs and the canonical discovery object for the lifetime + * of one merge. General callers must use the defensive public generators, which + * rediscover candidates before changing topology. + * + * @internal + */ +export { + discoverConflationCandidatesForTrustedMerge, + generateConflationApplicationArtifactsFromTrustedDiscovery, + generateConflationArtifactsFromTrustedDiscovery, +} from "../conflation.ts"; diff --git a/packages/change/src/merge.ts b/packages/change/src/merge.ts index 7720ccd0..867c53ab 100644 --- a/packages/change/src/merge.ts +++ b/packages/change/src/merge.ts @@ -1,8 +1,8 @@ /** * High-level merge pipeline for OSM datasets. * - * Orchestrates a complete merge workflow including deduplication of nodes and ways - * in both datasets, direct change generation, and optional intersection creation. + * Orchestrates direct change generation, conservative cross-dataset reconciliation, + * and optional intersection creation. * * @module */ @@ -11,7 +11,11 @@ import type { Osm } from "@osmix/core"; import { logProgress, type ProgressEvent, progressEvent } from "@osmix/shared/progress"; import { applyChangesetToOsm } from "./apply-changeset.ts"; -import { OsmChangeset } from "./changeset.ts"; +import { generateChangeset } from "./generate-changeset.ts"; +import { + discoverConflationCandidatesForTrustedMerge, + generateConflationApplicationArtifactsFromTrustedDiscovery, +} from "./internal/conflation.ts"; import type { OsmMergeOptions } from "./types.ts"; import { changeStatsSummary } from "./utils.ts"; @@ -19,10 +23,10 @@ import { changeStatsSummary } from "./utils.ts"; * Run a full merge pipeline on two OSM datasets. * * Executes a multi-stage merge process: - * 1. Deduplicates nodes and ways in both base and patch datasets - * 2. Optionally generates direct changes from patch to base (`directMerge`) - * 3. Optionally deduplicates nodes/ways in the final merged dataset - * 4. Optionally creates intersection nodes where ways cross + * 1. Optionally generates direct changes from patch to base (`directMerge`) + * 2. Optionally reconciles coincident patch nodes/ways with base entities + * 3. Optionally creates intersection nodes where ways cross + * 4. Verifies that the merge introduced no new routing-integrity problems * * @param base - The base OSM dataset to merge into. * @param patch - The patch OSM dataset to merge from. @@ -47,61 +51,59 @@ export async function merge( onProgress: (progress: ProgressEvent) => void = logProgress, ) { const log = (msg: string) => onProgress(progressEvent(msg)); - // De-duplicate nodes and ways in original datasets - log("Deduplicating ways in base OSM..."); - let changeset = new OsmChangeset(base); - changeset.deduplicateWays(base.ways); - log(changeStatsSummary(changeset.stats)); - let modifiedBase = applyChangesetToOsm(changeset); + let modifiedBase = base; - log("Deduplicating nodes in base OSM..."); - changeset = new OsmChangeset(modifiedBase); - changeset.deduplicateNodes(modifiedBase.nodes); - log(changeStatsSummary(changeset.stats)); - modifiedBase = applyChangesetToOsm(changeset); - - log("Deduplicating ways in patch OSM..."); - changeset = new OsmChangeset(patch); - changeset.deduplicateWays(patch.ways); - log(changeStatsSummary(changeset.stats)); - let modifiedPatch = applyChangesetToOsm(changeset); - - log("Deduplicating nodes in patch OSM..."); - changeset = new OsmChangeset(modifiedPatch); - changeset.deduplicateNodes(modifiedPatch.nodes); - log(changeStatsSummary(changeset.stats)); - modifiedPatch = applyChangesetToOsm(changeset); - - // Generate direct changes - if (options.directMerge) { - log("Generating direct changes from patch OSM to base OSM..."); - changeset = new OsmChangeset(modifiedBase); - changeset.generateDirectChanges(modifiedPatch); + // Generate direct changes and reconcile against the original, immutable base in + // one changeset. This keeps patch entities out of the base candidate pool. + if (options.directMerge || options.deduplicateNodes || options.deduplicateWays) { + const changeset = generateChangeset( + base, + patch, + { + directMerge: options.directMerge ?? false, + deduplicateNodes: options.deduplicateNodes ?? false, + deduplicateWays: options.deduplicateWays ?? false, + createIntersections: false, + }, + onProgress, + ); log(changeStatsSummary(changeset.stats)); modifiedBase = applyChangesetToOsm(changeset); } - // De-duplicate nodes and ways in final dataset - if (options.deduplicateWays) { - log("Deduplicating ways in final dataset..."); - changeset = new OsmChangeset(modifiedBase); - changeset.deduplicateWays(modifiedPatch.ways); - log(changeStatsSummary(changeset.stats)); - modifiedBase = applyChangesetToOsm(changeset); - } - if (options.deduplicateNodes) { - log("Deduplicating nodes in final dataset..."); - changeset = new OsmChangeset(modifiedBase); - changeset.deduplicateNodes(modifiedPatch.nodes); - log(changeStatsSummary(changeset.stats)); - modifiedBase = applyChangesetToOsm(changeset); + if (options.conflation) { + if (!options.directMerge) { + throw Error("Fuzzy conflation requires directMerge to preserve unmatched patch entities"); + } + log(`Discovering imported-data matches from ${patch.id} against ${base.id}...`); + // Fuzzy discovery always sees untouched inputs. The ordinary result is only the + // application baseline, preventing transitive matches through imported entities. + const discovery = discoverConflationCandidatesForTrustedMerge(base, patch, options.conflation); + const ordinaryBaseline = modifiedBase; + const conflation = generateConflationApplicationArtifactsFromTrustedDiscovery( + ordinaryBaseline, + patch, + discovery, + base, + options.conflation.decisions ?? [], + ); + // Generation already materialized and validated this exact result. Installing + // it directly avoids a second full decode, index build, and integrity pass. + modifiedBase = conflation.result; + log( + `Conflation candidates: ${discovery.summary.automatic.toLocaleString()} automatic, ${discovery.summary.review.toLocaleString()} review, ${discovery.summary.blocked.toLocaleString()} blocked, ${discovery.summary.unmatched.toLocaleString()} unmatched`, + ); } - // Create intersections + // Intersections run after conflation so accepted patch attachments participate in + // crossing insertion, while candidate discovery remains based on untouched inputs. if (options.createIntersections) { - log("Creating intersections in final dataset..."); - changeset = new OsmChangeset(modifiedBase); - changeset.createIntersectionsForWays(modifiedPatch.ways); + const changeset = generateChangeset( + modifiedBase, + patch, + { createIntersections: true }, + onProgress, + ); log(changeStatsSummary(changeset.stats)); modifiedBase = applyChangesetToOsm(changeset); } diff --git a/packages/change/src/sweepline-intersections.ts b/packages/change/src/sweepline-intersections.ts index f5bfba57..d1a1c907 100644 --- a/packages/change/src/sweepline-intersections.ts +++ b/packages/change/src/sweepline-intersections.ts @@ -253,37 +253,45 @@ function processFeature( for (let i = 0; i < coords.length; i++) { for (let ii = 0; ii < coords[i]!.length; ii++) { - const ring = coords[i]![ii]!; - let currentP = ring[0]!; - let nextP: Position | null = null; - ringId = ringId + 1; - for (let iii = 0; iii < ring.length - 1; iii++) { - nextP = ring[iii + 1]!; - - const e1 = new Event(currentP, featureId, ringId, eventId); - const e2 = new Event(nextP, featureId, ringId, eventId + 1); - - e1.otherEvent = e2; - e2.otherEvent = e1; - - if (checkWhichEventIsLeft(e1, e2) > 0) { - e2.isLeftEndpoint = true; - e1.isLeftEndpoint = false; - } else { - e1.isLeftEndpoint = true; - e2.isLeftEndpoint = false; - } - eventQueue.push(e1); - eventQueue.push(e2); - - currentP = nextP; - eventId = eventId + 1; - } + fillLineEventQueue(coords[i]![ii]!, eventQueue); } } featureId = featureId + 1; } +/** + * Add one line to the same event queue used by the GeoJSON-compatible entry point. + * Keeping this as the single event-construction path lets the merge hot path avoid + * temporary GeoJSON wrappers without changing the robust intersection kernel. + */ +function fillLineEventQueue(line: readonly Position[], eventQueue: TinyQueue<Event>): void { + let currentP = line[0]!; + let nextP: Position | null = null; + ringId = ringId + 1; + for (let index = 0; index < line.length - 1; index++) { + nextP = line[index + 1]!; + + const e1 = new Event(currentP, featureId, ringId, eventId); + const e2 = new Event(nextP, featureId, ringId, eventId + 1); + + e1.otherEvent = e2; + e2.otherEvent = e1; + + if (checkWhichEventIsLeft(e1, e2) > 0) { + e2.isLeftEndpoint = true; + e1.isLeftEndpoint = false; + } else { + e1.isLeftEndpoint = true; + e2.isLeftEndpoint = false; + } + eventQueue.push(e1); + eventQueue.push(e2); + + currentP = nextP; + eventId = eventId + 1; + } +} + class Segment { leftSweepEvent: Event; rightSweepEvent: Event; @@ -695,3 +703,22 @@ export default function sweeplineIntersections( ): [number, number][] { return sweeplineIntersectionsRuntime(geojson, ignoreSelfIntersections); } + +/** + * Check two lines with the same event ordering and robust predicates as the + * GeoJSON-compatible runtime, but without allocating wrapper features. + * + * This intentionally remains internal to `@osmix/change`: callers that need + * general GeoJSON support should use the default entry point above. + */ +export function sweeplineLineIntersections( + lineA: readonly Point[], + lineB: readonly Point[], +): [number, number][] { + const eventQueue = new TinyQueue<Event>([], checkWhichEventIsLeft); + fillLineEventQueue(lineA, eventQueue); + featureId++; + fillLineEventQueue(lineB, eventQueue); + featureId++; + return runCheck(eventQueue, true); +} diff --git a/packages/change/src/types.ts b/packages/change/src/types.ts index eb23318b..67a01d70 100644 --- a/packages/change/src/types.ts +++ b/packages/change/src/types.ts @@ -51,6 +51,172 @@ export interface OsmMergeOptions { deduplicateNodes: boolean; deduplicateWays: boolean; createIntersections: boolean; + + /** Optional, explicitly configured cross-dataset proximity conflation. */ + conflation?: OsmConflationOptions; +} + +/** Entity kinds supported by fuzzy conflation. */ +export type OsmConflationEntityType = "node" | "way"; + +/** Whether high-confidence candidates should be accepted without a review decision. */ +export type OsmConflationAutomatic = "high-confidence" | "none"; + +/** Intrinsic classification of a discovered source/target match. */ +export type OsmConflationStatus = "automatic" | "review" | "blocked" | "unmatched"; + +/** Candidate status after applying an optional user decision. */ +export type OsmConflationEffectiveStatus = OsmConflationStatus | "accepted" | "rejected"; + +/** Stable, machine-readable explanations for a conflation classification. */ +export type OsmConflationReasonCode = + | "bearing-mismatch" + | "drivable-network" + | "exact-match" + | "geometry-mismatch" + | "grade-conflict" + | "length-mismatch" + | "many-to-one" + | "multiple-targets" + | "no-transferable-properties" + | "node-context-conflict" + | "non-routing-target" + | "protected-tag" + | "relation-member" + | "routing-family-conflict" + | "routing-property" + | "same-id" + | "unsupported-way-chain" + | "would-collapse-way"; + +/** A selected patch tag and the value it would replace on the base entity. */ +export interface OsmConflationTagDiff { + key: string; + patchValue: string | number; + baseValue?: string | number; + protected: boolean; + routing: boolean; +} + +/** Serializable matching evidence used by the UI and deterministic tests. */ +export interface OsmConflationEvidence { + distanceMeters: number; + sourceRoutingFamilies: OsmConflationRoutingFamily[]; + targetRoutingFamilies: OsmConflationRoutingFamily[]; + tagDiff: OsmConflationTagDiff[]; + patchWayIds?: number[]; + bearingDifferenceDegrees?: number; + endpointDistancesMeters?: [number, number]; + lengthDifferenceRatio?: number; + maxGeometryDistanceMeters?: number; +} + +/** Normalized routing contexts used to compare imported and base geometry. */ +export type OsmConflationRoutingFamily = + | "bicycle-shared" + | "motor-road" + | "non-routable" + | "pedestrian"; + +/** Classification for one independently selectable conflation action. */ +export interface OsmConflationActionAssessment { + status: OsmConflationStatus; + reasons: OsmConflationReasonCode[]; +} + +/** One stable source/target candidate. Ambiguous sources have one row per target. */ +export interface OsmConflationCandidate { + id: string; + entityType: OsmConflationEntityType; + sourceId: number; + targetId: number | null; + status: OsmConflationStatus; + reasons: OsmConflationReasonCode[]; + propertyTransfer: OsmConflationActionAssessment; + networkAttachment: OsmConflationActionAssessment | null; + evidence: OsmConflationEvidence; +} + +/** Explicit fuzzy-conflation configuration. Property transfer is disabled by an empty key list. */ +export interface OsmConflationOptions { + propertyKeys: string[]; + attachNetwork: boolean; + maxDistanceMeters?: number; + automatic?: OsmConflationAutomatic; + decisions?: OsmConflationDecision[]; +} + +/** Fully defaulted options captured with a deterministic discovery result. */ +export interface ResolvedOsmConflationOptions { + propertyKeys: string[]; + attachNetwork: boolean; + maxDistanceMeters: number; + automatic: OsmConflationAutomatic; +} + +/** A user's explicit choice for a discovered source/target pair. */ +export interface OsmConflationDecision { + candidateId: string; + action: "accept" | "reject"; + transferProperties?: boolean; + attachNetwork?: boolean; +} + +/** A filter-wide review operation performed atomically in the conflation worker. */ +export type OsmConflationBulkAction = "transfer-properties" | "attach-network" | "reject"; + +/** Stable input for applying one bulk decision to all candidates matching a filter. */ +export interface OsmConflationBulkDecisionRequest { + action: OsmConflationBulkAction; + filter: OsmConflationCandidateFilter; +} + +/** Counts shown before confirming a filter-wide decision. */ +export interface OsmConflationBulkDecisionPreview { + action: OsmConflationBulkAction; + filteredCandidates: number; + eligibleCandidates: number; + changedCandidates: number; + skippedCandidates: number; + automaticCandidates: number; + reviewCandidates: number; + overriddenDecisions: number; +} + +/** Atomic result returned after a filter-wide decision is applied. */ +export interface OsmConflationBulkDecisionResult { + decisions: OsmConflationDecision[]; + preview: OsmConflationBulkDecisionPreview; + summary: OsmConflationSummary; +} + +/** Counts used to present discovery and review progress. */ +export interface OsmConflationSummary { + total: number; + accepted: number; + automatic: number; + review: number; + blocked: number; + unmatched: number; + rejected: number; +} + +/** Deterministic discovery result produced only from untouched inputs. */ +export interface OsmConflationDiscovery { + baseOsmId: string; + patchOsmId: string; + options: ResolvedOsmConflationOptions; + candidates: OsmConflationCandidate[]; + summary: OsmConflationSummary; +} + +/** Serializable filters used by paged worker APIs. */ +export interface OsmConflationCandidateFilter { + entityType?: OsmConflationEntityType; + status?: OsmConflationEffectiveStatus; + reason?: OsmConflationReasonCode; + sourceId?: number; + targetId?: number | null; } /** diff --git a/packages/change/src/utils.ts b/packages/change/src/utils.ts index 1b973b33..50795e2d 100644 --- a/packages/change/src/utils.ts +++ b/packages/change/src/utils.ts @@ -13,7 +13,7 @@ import { haversineDistance } from "@osmix/geo/haversine-distance"; import type { OsmEntity, OsmRelation, OsmTags, OsmWay } from "@osmix/types"; -import sweeplineIntersections from "./sweepline-intersections.ts"; +import { sweeplineLineIntersections } from "./sweepline-intersections.ts"; import type { OsmChangesetStats } from "./types.ts"; const XML_ATTRIBUTE_ESCAPES: Record<string, string> = { @@ -106,11 +106,30 @@ const isFootish = (t: OsmTags) => ["footway", "path", "cycleway", "bridleway", "steps"].includes(String(t["highway"])); const isPolygonish = (t: OsmTags) => !!(t["building"] || t["landuse"] || t["natural"]); +function normalizedGradeValue(value: number | string | undefined, defaultValue: string) { + const normalized = String(value ?? ""); + if (normalized === "" || normalized === "0" || normalized === "false" || normalized === "no") { + return defaultValue; + } + return normalized; +} + +/** Normalize the routing-relevant vertical context of a way for safe comparisons. */ +export function routingGradeSignature(tags?: OsmTags) { + return [ + `layer=${String(tags?.["layer"] ?? "0")}`, + `level=${String(tags?.["level"] ?? "")}`, + `bridge=${normalizedGradeValue(tags?.["bridge"], "no")}`, + `tunnel=${normalizedGradeValue(tags?.["tunnel"], "no")}`, + `covered=${normalizedGradeValue(tags?.["covered"], "no")}`, + ].join("|"); +} + /** * Determine if two ways should be connected based on their tags. * Connection logic: * - Never connect if either is an area (building, landuse, etc). - * - Never connect if separated by bridge/tunnel/layer. + * - Never connect if layer, level, bridge, tunnel, or covered context differs. * - Connect highway-highway, highway-footway, footway-footway. */ export function waysShouldConnect(tagsA?: OsmTags, tagsB?: OsmTags) { @@ -118,9 +137,7 @@ export function waysShouldConnect(tagsA?: OsmTags, tagsB?: OsmTags) { const b = tagsB || {}; if (isPolygonish(a) || isPolygonish(b)) return false; - const isSeparated = !!(a["bridge"] || a["tunnel"] || b["bridge"] || b["tunnel"]); - const diffLayer = (a["layer"] ?? "0") !== (b["layer"] ?? "0"); - if (isSeparated || diffLayer) return false; + if (routingGradeSignature(a) !== routingGradeSignature(b)) return false; if (isHighway(a) && isHighway(b)) return true; if (isHighway(a) && isFootish(b)) return true; @@ -133,8 +150,13 @@ export function waysShouldConnect(tagsA?: OsmTags, tagsB?: OsmTags) { /** * Determine if a way is a candidate for connecting to another way */ +export function areWayTagsIntersectionCandidate(tags?: OsmTags) { + return !!tags && (isHighway(tags) || isFootish(tags)) && !isPolygonish(tags); +} + +/** Determine if a complete way is a candidate for connecting to another way. */ export function isWayIntersectionCandidate(way: OsmWay) { - return way.tags && (isHighway(way.tags) || isFootish(way.tags)) && !isPolygonish(way.tags); + return areWayTagsIntersectionCandidate(way.tags); } /** @@ -172,30 +194,7 @@ export function waysIntersect( wayA: [number, number][], wayB: [number, number][], ): [number, number][] { - const intersections = sweeplineIntersections( - { - type: "FeatureCollection", - features: [ - { - type: "Feature", - geometry: { - type: "LineString", - coordinates: wayA, - }, - properties: {}, - }, - { - type: "Feature", - geometry: { - type: "LineString", - coordinates: wayB, - }, - properties: {}, - }, - ], - }, - true, - ); + const intersections = sweeplineLineIntersections(wayA, wayB); const uniqueFeatures: [number, number][] = []; const seen = new Set<string>(); diff --git a/packages/change/test/apply-changeset.test.ts b/packages/change/test/apply-changeset.test.ts index 931b0b2f..f0311d5b 100644 --- a/packages/change/test/apply-changeset.test.ts +++ b/packages/change/test/apply-changeset.test.ts @@ -67,6 +67,33 @@ function serializeEntities(osm: OsmType) { } describe("applyChangesetToOsm", () => { + it("reuses immutable buffers for an empty changeset while returning a fresh wrapper", () => { + const base = createBaseOsm(); + base.buildSpatialIndexes(); + const changeset = new OsmChangeset(base); + + const result = applyChangesetToOsm(changeset, "empty-result"); + + expect(result).not.toBe(base); + expect(result.id).toBe("empty-result"); + expect(result.contentHash()).toBe(base.contentHash()); + expect(result.hasSpatialIndexes()).toBe(true); + expect(serializeEntities(result)).toEqual(serializeEntities(base)); + expect(result.nodes.transferables().ids).toBe(base.nodes.transferables().ids); + expect(result.ways.transferables().refs).toBe(base.ways.transferables().refs); + }); + + it("builds missing spatial indexes when an empty base is not fully indexed", () => { + const base = createBaseOsm(); + const changeset = new OsmChangeset(base); + + const result = applyChangesetToOsm(changeset); + + expect(base.hasSpatialIndexes()).toBe(false); + expect(result.hasSpatialIndexes()).toBe(true); + expect(serializeEntities(result)).toEqual(serializeEntities(base)); + }); + it("preserves changeset records and supports applying the same object twice", () => { const base = createBaseOsm(); const changeset = createChangeset(base); @@ -162,4 +189,18 @@ describe("applyChangesetToOsm", () => { expect(JSON.stringify(way)).toBe(before); }); + + it("preserves the invalid-stage error for a change whose ID is absent from the base", () => { + const base = createBaseOsm(); + const changeset = new OsmChangeset(base); + changeset.nodeChanges[999] = { + changeType: "modify", + entity: { id: 999, lon: -120, lat: 46 }, + osmId: base.id, + }; + + expect(() => applyChangesetToOsm(changeset)).toThrow( + "Changeset still contains node changes in incorrect stage.", + ); + }); }); diff --git a/packages/change/test/conflation.test.ts b/packages/change/test/conflation.test.ts new file mode 100644 index 00000000..2c594c34 --- /dev/null +++ b/packages/change/test/conflation.test.ts @@ -0,0 +1,1264 @@ +import { Osm } from "@osmix/core"; +import type { OsmNode, OsmRelation, OsmWay } from "@osmix/types"; +import { describe, expect, it, vi } from "vitest"; + +import { applyChangesetToOsm } from "../src/apply-changeset.ts"; +import { + buildConflationBulkDecisionResult, + discoverConflationCandidates, + filterConflationCandidates, + generateConflationApplicationChangeset, + generateConflationChangeset, + summarizeConflationCandidates, + validateConflationDecisions, +} from "../src/conflation.ts"; +import { generateChangeset } from "../src/generate-changeset.ts"; +import * as publicChangeApi from "../src/index.ts"; +import { merge } from "../src/merge.ts"; +import type { + OsmConflationCandidate, + OsmConflationDecision, + OsmConflationOptions, +} from "../src/types.ts"; + +function createOsm( + id: string, + nodes: OsmNode[], + ways: OsmWay[] = [], + relations: OsmRelation[] = [], +) { + const osm = new Osm({ id }); + for (const node of nodes) osm.nodes.addNode(node); + for (const way of ways) osm.ways.addWay(way); + for (const relation of relations) osm.relations.addRelation(relation); + osm.buildIndexes(); + osm.buildSpatialIndexes(); + return osm; +} + +const silent = () => {}; + +const attachmentOptions: OsmConflationOptions = { + propertyKeys: [], + attachNetwork: true, +}; + +describe("safe fuzzy conflation discovery", () => { + it("keeps trusted generation capabilities out of the public package API", () => { + expect(publicChangeApi).not.toHaveProperty("discoverConflationCandidatesForTrustedMerge"); + expect(publicChangeApi).not.toHaveProperty("generateConflationArtifactsFromTrustedDiscovery"); + expect(publicChangeApi).not.toHaveProperty( + "generateConflationApplicationArtifactsFromTrustedDiscovery", + ); + }); + + it("automatically attaches a unique aligned imported sidewalk without moving the base", async () => { + const base = createOsm( + "base", + [ + { id: 1, lon: 0, lat: 0 }, + { id: 2, lon: -0.001, lat: 0 }, + ], + [{ id: 10, refs: [2, 1], tags: { highway: "footway" } }], + ); + const patch = createOsm( + "patch", + [ + { id: 101, lon: 0.000005, lat: 0 }, + { id: 102, lon: 0.001, lat: 0 }, + ], + [{ id: 20, refs: [101, 102], tags: { highway: "footway" } }], + ); + + const discovery = discoverConflationCandidates(base, patch, attachmentOptions); + const match = discovery.candidates.find((candidate) => candidate.sourceId === 101); + expect(match).toMatchObject({ + id: "node:101->1", + status: "automatic", + networkAttachment: { status: "automatic" }, + }); + expect(match?.evidence.distanceMeters).toBeGreaterThan(0.5); + expect(match?.evidence.distanceMeters).toBeLessThan(0.6); + + const buildIndexes = vi.spyOn(Osm.prototype, "buildIndexes"); + let result!: Osm; + try { + result = await merge( + base, + patch, + { directMerge: true, conflation: attachmentOptions }, + silent, + ); + expect(buildIndexes).toHaveBeenCalledTimes(2); + } finally { + buildIndexes.mockRestore(); + } + expect(result.nodes.getById(1)).toMatchObject({ lon: 0, lat: 0 }); + expect(result.nodes.ids.has(101)).toBe(true); + expect(result.ways.getById(10)?.refs).toEqual([2, 1]); + expect(result.ways.getById(20)?.refs).toEqual([1, 102]); + const cumulative = applyChangesetToOsm( + generateConflationChangeset(base, patch, { + directMerge: true, + conflation: attachmentOptions, + }), + ); + expect([...cumulative.nodes].map((node) => node.id)).toEqual( + [...result.nodes].map((node) => node.id), + ); + expect(cumulative.ways.getById(20)?.refs).toEqual(result.ways.getById(20)?.refs); + }); + + it("blocks an area-only school boundary vertex near a routing node", () => { + const base = createOsm( + "base", + [ + { id: 1, lon: 0, lat: 0 }, + { id: 2, lon: -0.001, lat: 0 }, + ], + [{ id: 10, refs: [2, 1], tags: { highway: "footway" } }], + ); + const patch = createOsm( + "patch", + [ + { id: 101, lon: 0.000005, lat: 0 }, + { id: 102, lon: 0.001, lat: 0 }, + { id: 103, lon: 0.001, lat: 0.001 }, + ], + [ + { + id: 20, + refs: [101, 102, 103, 101], + tags: { boundary: "school", area: "yes" }, + }, + ], + ); + + const match = discoverConflationCandidates(base, patch, attachmentOptions).candidates.find( + (candidate) => candidate.sourceId === 101, + ); + expect(match?.status).toBe("blocked"); + expect(match?.reasons).toContain("non-routing-target"); + }); + + it("does not automatically transfer properties between a footway and school boundary", () => { + const base = createOsm( + "base", + [ + { id: 1, lon: 0, lat: 0 }, + { id: 2, lon: -0.001, lat: 0 }, + ], + [{ id: 10, refs: [2, 1], tags: { highway: "footway" } }], + ); + const patch = createOsm( + "patch", + [ + { id: 101, lon: 0.000005, lat: 0, tags: { name: "School boundary" } }, + { id: 102, lon: 0.001, lat: 0 }, + { id: 103, lon: 0.001, lat: 0.001 }, + ], + [ + { + id: 20, + refs: [101, 102, 103, 101], + tags: { boundary: "school", area: "yes" }, + }, + ], + ); + const candidate = discoverConflationCandidates(base, patch, { + propertyKeys: ["name"], + attachNetwork: false, + }).candidates.find((item) => item.sourceId === 101); + expect(candidate?.propertyTransfer.status).toBe("blocked"); + expect(candidate?.propertyTransfer.reasons).toContain("non-routing-target"); + }); + + it("classifies multiple targets and many-to-one matches for review", () => { + const base = createOsm("base", [ + { id: 1, lon: -0.000003, lat: 0, tags: { name: "A" } }, + { id: 2, lon: 0.000003, lat: 0, tags: { name: "B" } }, + { id: 3, lon: 0.001, lat: 0, tags: { name: "C" } }, + ]); + const patch = createOsm("patch", [ + { id: 101, lon: 0, lat: 0, tags: { name: "Imported A" } }, + { id: 102, lon: 0.001005, lat: 0, tags: { name: "Imported C 1" } }, + { id: 103, lon: 0.000995, lat: 0, tags: { name: "Imported C 2" } }, + ]); + + const discovery = discoverConflationCandidates(base, patch, { + propertyKeys: ["name"], + attachNetwork: false, + }); + const ambiguous = discovery.candidates.filter((candidate) => candidate.sourceId === 101); + expect(ambiguous).toHaveLength(2); + expect(ambiguous.every((candidate) => candidate.status === "review")).toBe(true); + expect(ambiguous.every((candidate) => candidate.reasons.includes("multiple-targets"))).toBe( + true, + ); + const manyToOne = discovery.candidates.filter((candidate) => candidate.targetId === 3); + expect(manyToOne).toHaveLength(2); + expect(manyToOne.every((candidate) => candidate.reasons.includes("many-to-one"))).toBe(true); + }); + + it("keeps decision summaries and filters lightweight", () => { + const base = createOsm("base", [{ id: 1, lon: 0, lat: 0, tags: { name: "Base" } }]); + const patch = createOsm("patch", [{ id: 101, lon: 0.000005, lat: 0, tags: { name: "Patch" } }]); + const discovery = discoverConflationCandidates(base, patch, { + propertyKeys: ["name"], + attachNetwork: false, + }); + const decisions = [{ candidateId: "node:101->1", action: "reject" as const }]; + expect(summarizeConflationCandidates(discovery.candidates, decisions)).toMatchObject({ + total: 1, + automatic: 0, + rejected: 1, + }); + expect( + filterConflationCandidates(discovery.candidates, { status: "rejected" }, decisions), + ).toHaveLength(1); + + const accepted = [{ candidateId: "node:101->1", action: "accept" as const }]; + expect(summarizeConflationCandidates(discovery.candidates, accepted)).toMatchObject({ + total: 1, + accepted: 1, + automatic: 0, + }); + expect( + filterConflationCandidates(discovery.candidates, { status: "accepted" }, accepted), + ).toHaveLength(1); + }); + + it("builds filter-wide action-specific decisions and skips ambiguous candidates", () => { + const automatic: OsmConflationCandidate = { + id: "node:101->1", + entityType: "node", + sourceId: 101, + targetId: 1, + status: "automatic", + reasons: [], + propertyTransfer: { status: "automatic", reasons: [] }, + networkAttachment: { status: "automatic", reasons: [] }, + evidence: { + distanceMeters: 0.5, + sourceRoutingFamilies: ["pedestrian"], + targetRoutingFamilies: ["pedestrian"], + tagDiff: [{ key: "name", patchValue: "Imported", protected: false, routing: false }], + }, + }; + const review: OsmConflationCandidate = { + ...structuredClone(automatic), + id: "node:102->2", + sourceId: 102, + targetId: 2, + status: "review", + reasons: ["routing-property"], + propertyTransfer: { status: "review", reasons: ["routing-property"] }, + }; + const ambiguous: OsmConflationCandidate = { + ...structuredClone(review), + id: "node:103->3", + sourceId: 103, + targetId: 3, + reasons: ["multiple-targets"], + propertyTransfer: { status: "review", reasons: ["multiple-targets"] }, + networkAttachment: { status: "review", reasons: ["multiple-targets"] }, + }; + const blocked: OsmConflationCandidate = { + ...structuredClone(automatic), + id: "node:104->4", + sourceId: 104, + targetId: 4, + status: "blocked", + reasons: ["grade-conflict"], + propertyTransfer: { status: "blocked", reasons: ["grade-conflict"] }, + networkAttachment: { status: "blocked", reasons: ["grade-conflict"] }, + }; + const candidates = [automatic, review, ambiguous, blocked]; + const initialDecisions: OsmConflationDecision[] = [ + { candidateId: review.id, action: "reject" }, + ]; + + const propertyResult = buildConflationBulkDecisionResult(candidates, initialDecisions, { + action: "transfer-properties", + filter: { entityType: "node" }, + }); + expect(propertyResult.preview).toEqual({ + action: "transfer-properties", + filteredCandidates: 4, + eligibleCandidates: 2, + changedCandidates: 2, + skippedCandidates: 2, + automaticCandidates: 1, + reviewCandidates: 1, + overriddenDecisions: 1, + }); + expect(propertyResult.decisions).toEqual([ + { + candidateId: automatic.id, + action: "accept", + transferProperties: true, + attachNetwork: true, + }, + { + candidateId: review.id, + action: "accept", + transferProperties: true, + attachNetwork: false, + }, + ]); + expect(propertyResult.summary).toMatchObject({ accepted: 2, blocked: 1, review: 1 }); + + const networkResult = buildConflationBulkDecisionResult(candidates, propertyResult.decisions, { + action: "attach-network", + filter: { status: "accepted" }, + }); + expect(networkResult.preview).toMatchObject({ + filteredCandidates: 2, + eligibleCandidates: 2, + changedCandidates: 1, + skippedCandidates: 0, + overriddenDecisions: 1, + }); + expect(networkResult.decisions.find((decision) => decision.candidateId === review.id)).toEqual({ + candidateId: review.id, + action: "accept", + transferProperties: true, + attachNetwork: true, + }); + + const rejectResult = buildConflationBulkDecisionResult(candidates, networkResult.decisions, { + action: "reject", + filter: { status: "accepted" }, + }); + expect(rejectResult.preview).toMatchObject({ + filteredCandidates: 2, + eligibleCandidates: 2, + changedCandidates: 2, + skippedCandidates: 0, + overriddenDecisions: 2, + }); + expect(rejectResult.summary).toMatchObject({ rejected: 2, blocked: 1, review: 1 }); + + const scopedResult = buildConflationBulkDecisionResult( + candidates, + [...networkResult.decisions, { candidateId: blocked.id, action: "reject" }], + { action: "reject", filter: { sourceId: automatic.sourceId } }, + ); + expect(scopedResult.decisions).toContainEqual({ + candidateId: blocked.id, + action: "reject", + }); + }); + + it("validates required configuration fields for untyped callers", () => { + const base = createOsm("base", []); + const patch = createOsm("patch", []); + expect(() => + discoverConflationCandidates(base, patch, { + attachNetwork: false, + } as unknown as OsmConflationOptions), + ).toThrow("propertyKeys must be an array"); + expect(() => + discoverConflationCandidates(base, patch, { + propertyKeys: [""], + attachNetwork: false, + }), + ).toThrow("non-empty strings"); + expect(() => + discoverConflationCandidates(base, patch, { + propertyKeys: ["name"], + } as unknown as OsmConflationOptions), + ).toThrow("attachNetwork must be a boolean"); + }); + + it("rejects stale, duplicate, and malformed decisions at the generation boundary", () => { + const base = createOsm("base", [{ id: 1, lon: 0, lat: 0, tags: { name: "Base" } }]); + const patch = createOsm("patch", [{ id: 101, lon: 0.000005, lat: 0, tags: { name: "Patch" } }]); + const conflation = { propertyKeys: ["name"], attachNetwork: false }; + const discovery = discoverConflationCandidates(base, patch, conflation); + const generate = (decisions: readonly OsmConflationDecision[]) => + generateConflationApplicationChangeset(base, patch, discovery, base, decisions); + const validDecisions: OsmConflationDecision[] = [ + { candidateId: "node:101->1", action: "accept", transferProperties: true }, + ]; + const beforeValidation = structuredClone(validDecisions); + expect(() => validateConflationDecisions(discovery.candidates, validDecisions)).not.toThrow(); + expect(validDecisions).toEqual(beforeValidation); + + expect(() => generate([{ candidateId: "node:missing->1", action: "accept" }])).toThrow( + "Unknown conflation candidate: node:missing->1", + ); + expect(() => + generate([ + { candidateId: "node:101->1", action: "accept" }, + { candidateId: "node:101->1", action: "reject" }, + ]), + ).toThrow("Duplicate conflation decision for node:101->1"); + expect(() => + generate([ + { candidateId: "node:101->1", action: "approve" }, + ] as unknown as OsmConflationDecision[]), + ).toThrow("Invalid conflation decision action for node:101->1"); + expect(() => + generate([ + { candidateId: "node:101->1", action: "accept", transferProperties: "yes" }, + ] as unknown as OsmConflationDecision[]), + ).toThrow("transferProperties must be a boolean for node:101->1"); + expect(() => + generate([ + { candidateId: "node:101->1", action: "accept", attachNetwork: null }, + ] as unknown as OsmConflationDecision[]), + ).toThrow("attachNetwork must be a boolean for node:101->1"); + expect(() => generate({} as unknown as OsmConflationDecision[])).toThrow( + "Conflation decisions must be an array", + ); + + expect(() => + generateConflationChangeset(base, patch, { + directMerge: true, + conflation: { + ...conflation, + decisions: [{ candidateId: "node:stale->1", action: "reject" }], + }, + }), + ).toThrow("Unknown conflation candidate: node:stale->1"); + }); + + it("rejects a fuzzy-only discovery from another merge session", () => { + const base = createOsm("base", [{ id: 1, lon: 0, lat: 0 }]); + const patch = createOsm("patch", [{ id: 101, lon: 0.000005, lat: 0 }]); + const otherBase = createOsm("other-base", [{ id: 2, lon: 0, lat: 0 }]); + const otherPatch = createOsm("other-patch", [{ id: 102, lon: 0.000005, lat: 0 }]); + const discovery = discoverConflationCandidates(otherBase, otherPatch, { + propertyKeys: ["name"], + attachNetwork: false, + }); + + expect(() => generateConflationApplicationChangeset(base, patch, discovery, base)).toThrow( + "Conflation discovery patch other-patch does not match patch", + ); + }); + + it("rejects unknown decisions through the high-level merge API", async () => { + const base = createOsm("base", [{ id: 1, lon: 0, lat: 0, tags: { name: "Base" } }]); + const patch = createOsm("patch", [{ id: 101, lon: 0.000005, lat: 0, tags: { name: "Patch" } }]); + await expect( + merge( + base, + patch, + { + directMerge: true, + conflation: { + propertyKeys: ["name"], + attachNetwork: false, + decisions: [{ candidateId: "node:stale->1", action: "reject" }], + }, + }, + silent, + ), + ).rejects.toThrow("Unknown conflation candidate: node:stale->1"); + }); + + it("recomputes canonical candidates instead of trusting caller-mutated discovery data", () => { + const base = createOsm( + "base", + [ + { id: 1, lon: 0, lat: 0 }, + { id: 2, lon: -0.001, lat: 0 }, + ], + [ + { + id: 10, + refs: [2, 1], + tags: { highway: "footway", layer: "-1", tunnel: "yes" }, + }, + ], + ); + const patch = createOsm( + "patch", + [ + { id: 101, lon: 0.000005, lat: 0 }, + { id: 102, lon: 0.001, lat: 0 }, + ], + [{ id: 20, refs: [101, 102], tags: { highway: "footway" } }], + ); + const discovery = discoverConflationCandidates(base, patch, attachmentOptions); + const forged = { + ...discovery, + candidates: discovery.candidates.map((candidate) => + candidate.sourceId === 101 + ? { + ...candidate, + targetId: 2, + status: "automatic" as const, + reasons: [], + networkAttachment: { status: "automatic" as const, reasons: [] }, + evidence: { ...candidate.evidence, patchWayIds: [20] }, + } + : candidate, + ), + }; + + const cumulative = applyChangesetToOsm( + generateConflationChangeset( + base, + patch, + { directMerge: true, conflation: attachmentOptions }, + [], + forged, + ), + ); + expect(cumulative.ways.getById(20)?.refs).toEqual([101, 102]); + + const direct = applyChangesetToOsm(generateChangeset(base, patch, { directMerge: true })); + const fuzzyOnly = applyChangesetToOsm( + generateConflationApplicationChangeset(direct, patch, forged, base), + ); + expect(fuzzyOnly.ways.getById(20)?.refs).toEqual([101, 102]); + }); +}); + +describe("safe fuzzy property transfer", () => { + it("overwrites only selected properties and retains the imported point geometry", async () => { + const base = createOsm("base", [ + { id: 1, lon: 0, lat: 0, tags: { amenity: "cafe", name: "Old" } }, + ]); + const patch = createOsm("patch", [ + { + id: 101, + lon: 0.000005, + lat: 0, + tags: { amenity: "school", name: "Imported", source: "survey" }, + }, + ]); + + const result = await merge( + base, + patch, + { + directMerge: true, + conflation: { propertyKeys: ["name", "missing"], attachNetwork: false }, + }, + silent, + ); + expect(result.nodes.getById(1)?.tags).toEqual({ amenity: "cafe", name: "Imported" }); + expect(result.nodes.getById(101)?.tags).toEqual({ + amenity: "school", + name: "Imported", + source: "survey", + }); + }); + + it("blocks structural tags and requires review for routing tags", () => { + const base = createOsm("base", [{ id: 1, lon: 0, lat: 0 }]); + const patch = createOsm("patch", [ + { + id: 101, + lon: 0.000005, + lat: 0, + tags: { highway: "crossing", layer: "1" }, + }, + ]); + const protectedMatch = discoverConflationCandidates(base, patch, { + propertyKeys: ["layer"], + attachNetwork: false, + }).candidates[0]; + expect(protectedMatch?.propertyTransfer).toEqual({ + status: "blocked", + reasons: ["protected-tag"], + }); + + const routingMatch = discoverConflationCandidates(base, patch, { + propertyKeys: ["highway"], + attachNetwork: false, + }).candidates[0]; + expect(routingMatch?.propertyTransfer).toEqual({ + status: "review", + reasons: ["routing-property"], + }); + }); + + it("requires review for conditional and namespaced modal routing properties", () => { + const base = createOsm("base", [{ id: 1, lon: 0, lat: 0 }]); + const patch = createOsm("patch", [ + { + id: 101, + lon: 0.000005, + lat: 0, + tags: { + "foot:conditional": "no @ (snow)", + "kerb:left": "lowered", + "motorcycle:conditional": "no @ (wet)", + "maxspeed:hgv:conditional": "30 @ (weight>7.5)", + }, + }, + ]); + const candidate = discoverConflationCandidates(base, patch, { + propertyKeys: [ + "foot:conditional", + "kerb:left", + "motorcycle:conditional", + "maxspeed:hgv:conditional", + ], + attachNetwork: false, + }).candidates[0]; + + expect(candidate?.propertyTransfer).toEqual({ + status: "review", + reasons: ["routing-property"], + }); + expect(candidate?.evidence.tagDiff.every((diff) => diff.routing)).toBe(true); + }); + + it("applies an explicitly reviewed routing property but never a protected property", async () => { + const base = createOsm("base", [{ id: 1, lon: 0, lat: 0 }]); + const patch = createOsm("patch", [ + { + id: 101, + lon: 0.000005, + lat: 0, + tags: { highway: "crossing", layer: "1" }, + }, + ]); + const conflation: OsmConflationOptions = { + propertyKeys: ["highway", "layer"], + attachNetwork: false, + decisions: [{ candidateId: "node:101->1", action: "accept" }], + }; + const result = await merge(base, patch, { directMerge: true, conflation }, silent); + expect(result.nodes.getById(1)?.tags).toEqual({ highway: "crossing" }); + }); + + it("matches reversed one-to-one ways and removes only redundant imported geometry", async () => { + const base = createOsm( + "base", + [ + { id: 1, lon: 0, lat: 0 }, + { id: 2, lon: 0.001, lat: 0 }, + ], + [{ id: 10, refs: [1, 2], tags: { highway: "footway", name: "Old" } }], + ); + const patch = createOsm( + "patch", + [ + { id: 101, lon: 0.001, lat: 0.000004 }, + { id: 102, lon: 0, lat: 0.000004 }, + { id: 999, lon: 0.01, lat: 0.01 }, + ], + [{ id: 20, refs: [101, 102], tags: { highway: "footway", name: "Imported" } }], + ); + const options = { propertyKeys: ["name"], attachNetwork: false }; + const candidate = discoverConflationCandidates(base, patch, options).candidates.find( + (item) => item.entityType === "way", + ); + expect(candidate).toMatchObject({ sourceId: 20, targetId: 10, status: "automatic" }); + + const result = await merge(base, patch, { directMerge: true, conflation: options }, silent); + expect(result.ways.getById(10)?.refs).toEqual([1, 2]); + expect(result.ways.getById(10)?.tags?.["name"]).toBe("Imported"); + expect(result.ways.ids.has(20)).toBe(false); + expect(result.nodes.ids.has(101)).toBe(false); + expect(result.nodes.ids.has(102)).toBe(false); + expect(result.nodes.ids.has(999)).toBe(true); + }); + + it("allows selected patch-wins properties on exact node and way geometry", async () => { + const base = createOsm( + "base", + [ + { id: 1, lon: 0, lat: 0, tags: { ref: "base" } }, + { id: 2, lon: 0.001, lat: 0 }, + ], + [{ id: 10, refs: [1, 2], tags: { highway: "footway", surface: "gravel" } }], + ); + const patch = createOsm( + "patch", + [ + { id: 101, lon: 0, lat: 0, tags: { ref: "patch" } }, + { id: 201, lon: 0, lat: 0 }, + { id: 202, lon: 0.001, lat: 0 }, + ], + [{ id: 20, refs: [201, 202], tags: { highway: "footway", surface: "paved" } }], + ); + const conflation = { propertyKeys: ["ref", "surface"], attachNetwork: false }; + const discovery = discoverConflationCandidates(base, patch, conflation); + expect(discovery.candidates).toEqual( + expect.arrayContaining([ + expect.objectContaining({ id: "node:101->1", status: "automatic" }), + expect.objectContaining({ id: "way:20->10", status: "automatic" }), + ]), + ); + const result = await merge( + base, + patch, + { + directMerge: true, + deduplicateNodes: true, + deduplicateWays: true, + conflation, + }, + silent, + ); + expect(result.nodes.getById(1)?.tags?.["ref"]).toBe("patch"); + expect(result.ways.getById(10)?.tags?.["surface"]).toBe("paved"); + expect(result.ways.ids.has(20)).toBe(false); + }); + + it("keeps same-ID patch updates authoritative over nearby fuzzy sources", async () => { + const base = createOsm("base", [{ id: 1, lon: 0, lat: 0, tags: { name: "Base" } }]); + const patch = createOsm("patch", [ + { id: 1, lon: 0, lat: 0, tags: { name: "Same-ID authoritative" } }, + { id: 101, lon: 0.000005, lat: 0, tags: { name: "Nearby fuzzy" } }, + ]); + const conflation = { propertyKeys: ["name"], attachNetwork: false }; + const discovery = discoverConflationCandidates(base, patch, conflation); + expect(discovery.candidates.find((candidate) => candidate.sourceId === 101)).toMatchObject({ + status: "unmatched", + targetId: null, + }); + const result = await merge(base, patch, { directMerge: true, conflation }, silent); + expect(result.nodes.getById(1)?.tags?.["name"]).toBe("Same-ID authoritative"); + }); + + it("does not suppress a geometrically reversed way with incompatible oneway semantics", async () => { + const base = createOsm( + "base", + [ + { id: 1, lon: 0, lat: 0 }, + { id: 2, lon: 0.001, lat: 0 }, + ], + [ + { + id: 10, + refs: [1, 2], + tags: { highway: "residential", oneway: "yes", name: "Base" }, + }, + ], + ); + const patch = createOsm( + "patch", + [ + { id: 101, lon: 0.001, lat: 0.000004 }, + { id: 102, lon: 0, lat: 0.000004 }, + ], + [ + { + id: 20, + refs: [101, 102], + tags: { highway: "residential", oneway: "yes", name: "Imported" }, + }, + ], + ); + const conflation = { propertyKeys: ["name"], attachNetwork: false }; + const result = await merge(base, patch, { directMerge: true, conflation }, silent); + expect(result.ways.getById(10)?.tags?.["name"]).toBe("Base"); + expect(result.ways.ids.has(20)).toBe(true); + }); + + it("does not suppress an equivalent way with a conditional access conflict", async () => { + const base = createOsm( + "base", + [ + { id: 1, lon: 0, lat: 0 }, + { id: 2, lon: 0.001, lat: 0 }, + ], + [ + { + id: 10, + refs: [1, 2], + tags: { highway: "footway", name: "Base", "wheelchair:conditional": "yes @ (dry)" }, + }, + ], + ); + const patch = createOsm( + "patch", + [ + { id: 101, lon: 0, lat: 0.000004 }, + { id: 102, lon: 0.001, lat: 0.000004 }, + ], + [ + { + id: 20, + refs: [101, 102], + tags: { highway: "footway", name: "Imported", "wheelchair:conditional": "no @ (wet)" }, + }, + ], + ); + const conflation = { propertyKeys: ["name"], attachNetwork: false }; + const candidate = discoverConflationCandidates(base, patch, conflation).candidates.find( + (item) => item.entityType === "way", + ); + + expect(candidate).toMatchObject({ + targetId: 10, + status: "blocked", + reasons: ["routing-family-conflict"], + }); + const result = await merge(base, patch, { directMerge: true, conflation }, silent); + expect(result.ways.getById(10)?.tags?.["name"]).toBe("Base"); + expect(result.ways.ids.has(20)).toBe(true); + }); + + it("does not suppress reversed geometry with directional routing tags", async () => { + const base = createOsm( + "base", + [ + { id: 1, lon: 0, lat: 0 }, + { id: 2, lon: 0.001, lat: 0 }, + ], + [ + { + id: 10, + refs: [1, 2], + tags: { highway: "footway", "kerb:left": "lowered", name: "Base" }, + }, + ], + ); + const patch = createOsm( + "patch", + [ + { id: 101, lon: 0.001, lat: 0.000004 }, + { id: 102, lon: 0, lat: 0.000004 }, + ], + [ + { + id: 20, + refs: [101, 102], + tags: { highway: "footway", "kerb:left": "lowered", name: "Imported" }, + }, + ], + ); + const conflation = { propertyKeys: ["name"], attachNetwork: false }; + const candidate = discoverConflationCandidates(base, patch, conflation).candidates.find( + (item) => item.entityType === "way", + ); + + expect(candidate).toMatchObject({ + targetId: 10, + status: "blocked", + reasons: ["routing-family-conflict"], + }); + const result = await merge(base, patch, { directMerge: true, conflation }, silent); + expect(result.ways.getById(10)?.tags?.["name"]).toBe("Base"); + expect(result.ways.ids.has(20)).toBe(true); + }); + + it("blocks a sub-meter way match whose true relative length differs by over five percent", async () => { + const base = createOsm( + "base", + [ + { id: 1, lon: 0, lat: 0 }, + { id: 2, lon: 0.000000898, lat: 0 }, + ], + [{ id: 10, refs: [1, 2], tags: { highway: "footway", name: "Base" } }], + ); + const patch = createOsm( + "patch", + [ + { id: 101, lon: 0, lat: 0.000004 }, + { id: 102, lon: 0.000001257, lat: 0.000004 }, + ], + [{ id: 20, refs: [101, 102], tags: { highway: "footway", name: "Imported" } }], + ); + const conflation = { propertyKeys: ["name"], attachNetwork: false }; + const candidate = discoverConflationCandidates(base, patch, conflation).candidates.find( + (item) => item.entityType === "way", + ); + + expect(candidate).toMatchObject({ targetId: 10, status: "blocked" }); + expect(candidate?.reasons).toContain("length-mismatch"); + expect(candidate?.evidence.lengthDifferenceRatio).toBeGreaterThan(0.25); + const result = await merge(base, patch, { directMerge: true, conflation }, silent); + expect(result.ways.getById(10)?.tags?.["name"]).toBe("Base"); + expect(result.ways.ids.has(20)).toBe(true); + }); + + it("reports a geometrically plausible grade-conflicting way instead of hiding it as unmatched", async () => { + const base = createOsm( + "base", + [ + { id: 1, lon: 0, lat: 0 }, + { id: 2, lon: 0.001, lat: 0 }, + ], + [ + { + id: 10, + refs: [1, 2], + tags: { highway: "footway", tunnel: "yes", layer: "-1", name: "Tunnel" }, + }, + ], + ); + const patch = createOsm( + "patch", + [ + { id: 101, lon: 0, lat: 0.000004 }, + { id: 102, lon: 0.001, lat: 0.000004 }, + ], + [{ id: 20, refs: [101, 102], tags: { highway: "footway", name: "Surface" } }], + ); + const conflation = { propertyKeys: ["name"], attachNetwork: false }; + const candidate = discoverConflationCandidates(base, patch, conflation).candidates.find( + (item) => item.entityType === "way", + ); + + expect(candidate).toMatchObject({ targetId: 10, status: "blocked" }); + expect(candidate?.reasons).toContain("grade-conflict"); + const result = await merge(base, patch, { directMerge: true, conflation }, silent); + expect(result.ways.ids.has(20)).toBe(true); + }); +}); + +describe("safe fuzzy topology gates", () => { + it("requires review before attaching drivable living-street geometry", async () => { + const base = createOsm( + "base", + [ + { id: 1, lon: 0, lat: 0 }, + { id: 2, lon: -0.001, lat: 0 }, + ], + [{ id: 10, refs: [2, 1], tags: { highway: "living_street" } }], + ); + const patch = createOsm( + "patch", + [ + { id: 101, lon: 0.000005, lat: 0 }, + { id: 102, lon: 0.001, lat: 0 }, + ], + [{ id: 20, refs: [101, 102], tags: { highway: "living_street" } }], + ); + const candidate = discoverConflationCandidates(base, patch, attachmentOptions).candidates.find( + (item) => item.sourceId === 101, + ); + expect(candidate).toMatchObject({ + status: "review", + networkAttachment: { + status: "review", + reasons: ["drivable-network"], + }, + evidence: { + sourceRoutingFamilies: ["motor-road"], + targetRoutingFamilies: ["motor-road"], + }, + }); + + const result = await merge( + base, + patch, + { directMerge: true, conflation: attachmentOptions }, + silent, + ); + expect(result.ways.getById(20)?.refs).toEqual([101, 102]); + expect(result.nodes.ids.has(101)).toBe(true); + }); + + it("blocks conflicting node grade and access context before network attachment", () => { + const base = createOsm( + "base", + [ + { id: 1, lon: 0, lat: 0 }, + { id: 2, lon: -0.001, lat: 0 }, + ], + [{ id: 10, refs: [2, 1], tags: { highway: "footway" } }], + ); + const patch = createOsm( + "patch", + [ + { id: 101, lon: 0.000005, lat: 0, tags: { layer: "-1", access: "private" } }, + { id: 102, lon: 0.001, lat: 0 }, + ], + [{ id: 20, refs: [101, 102], tags: { highway: "footway" } }], + ); + const candidate = discoverConflationCandidates(base, patch, attachmentOptions).candidates.find( + (item) => item.sourceId === 101, + ); + expect(candidate?.networkAttachment?.status).toBe("blocked"); + expect(candidate?.networkAttachment?.reasons).toEqual( + expect.arrayContaining(["grade-conflict", "routing-family-conflict"]), + ); + }); + + it("never auto-attaches barrier or floor nodes even when their contexts agree", () => { + const base = createOsm( + "base", + [ + { id: 1, lon: 0, lat: 0, tags: { barrier: "gate", level: "1" } }, + { id: 2, lon: -0.001, lat: 0 }, + ], + [{ id: 10, refs: [2, 1], tags: { highway: "footway" } }], + ); + const patch = createOsm( + "patch", + [ + { id: 101, lon: 0.000005, lat: 0, tags: { barrier: "gate", level: "1" } }, + { id: 102, lon: 0.001, lat: 0 }, + ], + [{ id: 20, refs: [101, 102], tags: { highway: "footway" } }], + ); + const candidate = discoverConflationCandidates(base, patch, attachmentOptions).candidates.find( + (item) => item.sourceId === 101, + ); + expect(candidate?.networkAttachment).toMatchObject({ + status: "review", + reasons: ["node-context-conflict"], + }); + }); + + it("blocks incompatible crossing and kerb node context but permits exact context", () => { + const base = createOsm( + "base", + [ + { + id: 1, + lon: 0, + lat: 0, + tags: { highway: "crossing", crossing: "marked", kerb: "lowered" }, + }, + { id: 2, lon: -0.001, lat: 0 }, + ], + [{ id: 10, refs: [2, 1], tags: { highway: "footway" } }], + ); + const patch = createOsm( + "patch", + [ + { + id: 101, + lon: 0.000005, + lat: 0, + tags: { highway: "crossing", crossing: "unmarked", kerb: "raised" }, + }, + { id: 102, lon: 0.001, lat: 0 }, + ], + [{ id: 20, refs: [101, 102], tags: { highway: "footway" } }], + ); + const conflict = discoverConflationCandidates(base, patch, attachmentOptions).candidates.find( + (item) => item.sourceId === 101, + ); + expect(conflict?.networkAttachment).toMatchObject({ + status: "blocked", + reasons: ["routing-family-conflict"], + }); + + const exactPatch = createOsm( + "exact-patch", + [ + { + id: 101, + lon: 0.000005, + lat: 0, + tags: { highway: "crossing", crossing: "marked", kerb: "lowered" }, + }, + { id: 102, lon: 0.001, lat: 0 }, + ], + [{ id: 20, refs: [101, 102], tags: { highway: "footway" } }], + ); + const exact = discoverConflationCandidates(base, exactPatch, attachmentOptions).candidates.find( + (item) => item.sourceId === 101, + ); + expect(exact?.networkAttachment).toEqual({ status: "automatic", reasons: [] }); + }); + + it("blocks grade conflicts and reviews perpendicular attachments", () => { + const base = createOsm( + "base", + [ + { id: 1, lon: 0, lat: 0 }, + { id: 2, lon: 0, lat: -0.001 }, + ], + [ + { + id: 10, + refs: [2, 1], + tags: { highway: "footway", tunnel: "yes", layer: "-1" }, + }, + ], + ); + const patch = createOsm( + "patch", + [ + { id: 101, lon: 0.000005, lat: 0 }, + { id: 102, lon: 0.001, lat: 0 }, + ], + [{ id: 20, refs: [101, 102], tags: { highway: "footway" } }], + ); + const gradeConflict = discoverConflationCandidates( + base, + patch, + attachmentOptions, + ).candidates.find((candidate) => candidate.sourceId === 101); + expect(gradeConflict?.status).toBe("blocked"); + expect(gradeConflict?.reasons).toContain("grade-conflict"); + + const surfaceBase = createOsm( + "surface", + [ + { id: 1, lon: 0, lat: 0 }, + { id: 2, lon: 0, lat: -0.001 }, + ], + [{ id: 10, refs: [2, 1], tags: { highway: "footway" } }], + ); + const perpendicular = discoverConflationCandidates( + surfaceBase, + patch, + attachmentOptions, + ).candidates.find((candidate) => candidate.sourceId === 101); + expect(perpendicular?.status).toBe("review"); + expect(perpendicular?.reasons).toContain("bearing-mismatch"); + }); + + it("blocks patch-way collapse and relation-member attachment", () => { + const base = createOsm( + "base", + [ + { id: 1, lon: 0, lat: 0 }, + { id: 2, lon: -0.001, lat: 0 }, + ], + [{ id: 10, refs: [2, 1], tags: { highway: "footway" } }], + ); + const collapsePatch = createOsm( + "collapse", + [{ id: 101, lon: 0.000005, lat: 0 }], + [{ id: 20, refs: [101, 1], tags: { highway: "footway" } }], + ); + const collapse = discoverConflationCandidates( + base, + collapsePatch, + attachmentOptions, + ).candidates.find((candidate) => candidate.sourceId === 101); + expect(collapse?.status).toBe("blocked"); + expect(collapse?.reasons).toContain("would-collapse-way"); + + const relationPatch = createOsm( + "relation", + [ + { id: 101, lon: 0.000005, lat: 0 }, + { id: 102, lon: 0.001, lat: 0 }, + ], + [{ id: 20, refs: [101, 102], tags: { highway: "footway" } }], + [ + { + id: 30, + members: [{ type: "node", ref: 101, role: "stop" }], + tags: { type: "route" }, + }, + ], + ); + const relation = discoverConflationCandidates( + base, + relationPatch, + attachmentOptions, + ).candidates.find((candidate) => candidate.sourceId === 101); + expect(relation?.status).toBe("review"); + expect(relation?.reasons).toContain("relation-member"); + + const restrictionPatch = createOsm( + "restriction", + [ + { id: 101, lon: 0.000005, lat: 0 }, + { id: 102, lon: 0.001, lat: 0 }, + ], + [{ id: 20, refs: [101, 102], tags: { highway: "footway" } }], + [ + { + id: 31, + members: [{ type: "node", ref: 101, role: "via" }], + tags: { type: "restriction", restriction: "no_left_turn" }, + }, + ], + ); + const restriction = discoverConflationCandidates( + base, + restrictionPatch, + attachmentOptions, + ).candidates.find((candidate) => candidate.sourceId === 101); + expect(restriction?.status).toBe("blocked"); + expect(restriction?.reasons).toContain("relation-member"); + }); + + it("reports one-to-many way chains as unsupported and leaves them in the direct merge", async () => { + const base = createOsm( + "base", + [ + { id: 1, lon: 0, lat: 0 }, + { id: 2, lon: 0.001, lat: 0 }, + { id: 3, lon: 0.002, lat: 0 }, + ], + [ + { id: 10, refs: [1, 2], tags: { highway: "footway" } }, + { id: 11, refs: [2, 3], tags: { highway: "footway" } }, + ], + ); + const patch = createOsm( + "patch", + [ + { id: 101, lon: 0, lat: 0.000004 }, + { id: 102, lon: 0.001, lat: 0.000004 }, + { id: 103, lon: 0.002, lat: 0.000004 }, + ], + [ + { + id: 20, + refs: [101, 102, 103], + tags: { highway: "footway", name: "Imported" }, + }, + ], + ); + const options = { propertyKeys: ["name"], attachNetwork: false }; + const unsupported = discoverConflationCandidates(base, patch, options).candidates.find( + (candidate) => candidate.entityType === "way", + ); + expect(unsupported).toMatchObject({ status: "unmatched", targetId: null }); + expect(unsupported?.reasons).toContain("unsupported-way-chain"); + + const result = await merge(base, patch, { directMerge: true, conflation: options }, silent); + expect(result.ways.getById(20)?.refs).toEqual([101, 102, 103]); + }); + + it("generates equivalent fuzzy-only and cumulative changesets from canonical discovery", () => { + const base = createOsm("base", [{ id: 1, lon: 0, lat: 0, tags: { name: "Old" } }]); + const patch = createOsm("patch", [ + { id: 101, lon: 0.000005, lat: 0, tags: { name: "Imported" } }, + ]); + const conflation = { propertyKeys: ["name"], attachNetwork: false }; + const discovery = discoverConflationCandidates(base, patch, conflation); + + const cumulative = applyChangesetToOsm( + generateConflationChangeset(base, patch, { directMerge: true, conflation }, [], discovery), + ); + const direct = applyChangesetToOsm(generateChangeset(base, patch, { directMerge: true })); + const fuzzyOnly = applyChangesetToOsm( + generateConflationApplicationChangeset(direct, patch, discovery, base), + ); + expect(fuzzyOnly.nodes.getById(1)?.tags).toEqual(cumulative.nodes.getById(1)?.tags); + expect(fuzzyOnly.nodes.getById(101)).toEqual(cumulative.nodes.getById(101)); + }); + + it("enforces the protected-base assertion inside the fuzzy-only generator", () => { + const originalBase = createOsm("base", [{ id: 1, lon: 0, lat: 0 }]); + const malformedBaseline = createOsm("base", []); + const patch = createOsm("patch", []); + const discovery = discoverConflationCandidates(originalBase, patch, { + propertyKeys: ["name"], + attachNetwork: false, + }); + + expect(() => + generateConflationApplicationChangeset(malformedBaseline, patch, discovery, originalBase), + ).toThrow("Conflation changed protected base topology"); + }); + + it("applies and validates the cumulative result before returning its changeset", () => { + const base = createOsm("base", []); + const patch = createOsm( + "patch", + [{ id: 101, lon: 0, lat: 0 }], + [{ id: 20, refs: [101, 999], tags: { highway: "footway", name: "Imported" } }], + ); + + expect(() => + generateConflationChangeset(base, patch, { + directMerge: true, + conflation: { propertyKeys: ["name"], attachNetwork: false }, + }), + ).toThrow("way 20 references missing node 999"); + }); +}); diff --git a/packages/change/test/intersections.test.ts b/packages/change/test/intersections.test.ts index 02eae06e..14906ebe 100644 --- a/packages/change/test/intersections.test.ts +++ b/packages/change/test/intersections.test.ts @@ -3,6 +3,8 @@ import { describe, expect, it } from "vitest"; import { applyChangesetToOsm } from "../src/apply-changeset.ts"; import { OsmChangeset } from "../src/changeset.ts"; +import { merge } from "../src/merge.ts"; +import { waysShouldConnect } from "../src/utils.ts"; function crossingWays() { const osm = new Osm({ id: "intersections" }); @@ -25,6 +27,27 @@ function crossingWays() { } describe("intersection geometry integrity", () => { + it("normalizes negative grade tags and compares the full vertical context", () => { + expect( + waysShouldConnect( + { bridge: "no", highway: "primary", tunnel: "false" }, + { highway: "secondary" }, + ), + ).toBe(true); + expect( + waysShouldConnect( + { highway: "primary", layer: "-1", tunnel: "yes" }, + { highway: "secondary", layer: "-1", tunnel: "yes" }, + ), + ).toBe(true); + expect( + waysShouldConnect({ covered: "yes", highway: "primary" }, { highway: "secondary" }), + ).toBe(false); + expect( + waysShouldConnect({ highway: "primary", level: "1" }, { highway: "secondary", level: "2" }), + ).toBe(false); + }); + it("resolves pending intersection nodes when a way is spliced more than once", () => { const osm = crossingWays(); const changeset = new OsmChangeset(osm); @@ -39,11 +62,262 @@ describe("intersection geometry integrity", () => { }); const result = applyChangesetToOsm(changeset); const horizontal = result.ways.getById(10); - expect(horizontal?.refs).toHaveLength(4); + expect(horizontal?.refs).toEqual([1, 7, 8, 2]); expect(horizontal?.refs.every((ref) => result.nodes.ids.has(ref))).toBe(true); expect(result.ways.getById(13)?.refs).toEqual([1]); }); + it("inserts multiple intersections in way order for reversed ways", () => { + const osm = crossingWays(); + const reversed = osm.ways.getById(10)!; + const changeset = new OsmChangeset(osm); + changeset.modify("way", reversed.id, (way) => ({ ...way, refs: [2, 1] })); + + changeset.createIntersectionsForWays(osm.ways); + + const result = applyChangesetToOsm(changeset); + expect(result.ways.getById(10)?.refs).toEqual([2, 8, 7, 1]); + }); + + it("rewrites via-node relation members when coincident way nodes are unified", () => { + const osm = new Osm({ id: "restriction-intersection" }); + for (const node of [ + { id: 1, lon: -1, lat: 0 }, + { id: 2, lon: 0, lat: 0 }, + { id: 3, lon: 1, lat: 0 }, + { id: 4, lon: 0, lat: -1 }, + { id: 5, lon: 0, lat: 0 }, + { id: 6, lon: 0, lat: 1 }, + ]) { + osm.nodes.addNode(node); + } + osm.ways.addWay({ id: 10, refs: [1, 2, 3], tags: { highway: "primary" } }); + osm.ways.addWay({ id: 20, refs: [4, 5, 6], tags: { highway: "primary" } }); + osm.relations.addRelation({ + id: 100, + tags: { type: "restriction", restriction: "no_left_turn" }, + members: [ + { type: "way", ref: 10, role: "from" }, + { type: "node", ref: 5, role: "via" }, + { type: "way", ref: 20, role: "to" }, + ], + }); + osm.buildIndexes(); + osm.buildSpatialIndexes(); + const changeset = new OsmChangeset(osm); + + changeset.createIntersectionsForWays(osm.ways); + + const result = applyChangesetToOsm(changeset); + expect(result.ways.getById(20)?.refs).toEqual([4, 2, 6]); + expect(result.relations.getById(100)?.members[1]).toEqual({ + type: "node", + ref: 2, + role: "via", + }); + }); + + it("preserves a routing-critical base endpoint when a patch endpoint is reused", () => { + const osm = new Osm({ id: "protected-endpoint" }); + for (const node of [ + { id: 1, lon: -1, lat: 0 }, + { id: 2, lon: 0, lat: 0, tags: { barrier: "gate", access: "private" } }, + { id: 5, lon: 0, lat: 0 }, + { id: 6, lon: 0, lat: 1 }, + ]) { + osm.nodes.addNode(node); + } + osm.ways.addWay({ id: 10, refs: [1, 2], tags: { highway: "service" } }); + osm.ways.addWay({ id: 20, refs: [5, 6], tags: { highway: "service" } }); + osm.buildIndexes(); + osm.buildSpatialIndexes(); + + const patch = new Osm({ id: "patch" }); + patch.nodes.addNode({ id: 5, lon: 0, lat: 0 }); + patch.nodes.addNode({ id: 6, lon: 0, lat: 1 }); + patch.ways.addWay({ id: 20, refs: [5, 6], tags: { highway: "service" } }); + patch.buildIndexes(); + const changeset = new OsmChangeset(osm); + + changeset.createIntersectionsForWays(patch.ways); + + const result = applyChangesetToOsm(changeset); + expect(result.ways.getById(10)?.refs).toEqual([1, 2]); + expect(result.ways.getById(20)?.refs).toEqual([2, 6]); + expect(result.nodes.getById(2)?.tags).toEqual({ + access: "private", + barrier: "gate", + crossing: "yes", + }); + }); + + it("preserves a shared base node ID when the patch endpoint adds routing tags", () => { + const osm = new Osm({ id: "shared-base-endpoint" }); + for (const node of [ + { id: 1, lon: -1, lat: 0 }, + { id: 2, lon: 0, lat: 0 }, + { id: 3, lon: 1, lat: 0 }, + { id: 5, lon: 0, lat: 0, tags: { barrier: "gate" } }, + { id: 6, lon: 0, lat: 1 }, + ]) { + osm.nodes.addNode(node); + } + osm.ways.addWay({ id: 10, refs: [1, 2], tags: { highway: "service" } }); + osm.ways.addWay({ id: 11, refs: [2, 3], tags: { highway: "service" } }); + osm.ways.addWay({ id: 20, refs: [5, 6], tags: { highway: "service" } }); + osm.buildIndexes(); + osm.buildSpatialIndexes(); + + const patch = new Osm({ id: "patch" }); + patch.nodes.addNode({ id: 5, lon: 0, lat: 0, tags: { barrier: "gate" } }); + patch.nodes.addNode({ id: 6, lon: 0, lat: 1 }); + patch.ways.addWay({ id: 20, refs: [5, 6], tags: { highway: "service" } }); + patch.buildIndexes(); + const changeset = new OsmChangeset(osm); + + changeset.createIntersectionsForWays(patch.ways); + + const result = applyChangesetToOsm(changeset); + expect(result.ways.getById(10)?.refs).toEqual([1, 2]); + expect(result.ways.getById(11)?.refs).toEqual([2, 3]); + expect(result.ways.getById(20)?.refs).toEqual([2, 6]); + expect(result.nodes.getById(2)?.tags).toEqual({ barrier: "gate", crossing: "yes" }); + }); + + it("creates a dedicated node when endpoint reuse would collapse a short patch way", async () => { + const base = new Osm({ id: "short-crossing-base" }); + for (const node of [ + { id: 1, lon: -120.5618765, lat: 46.5963651 }, + { id: 2, lon: -120.5618635, lat: 46.5963706 }, + { id: 3, lon: -120.5618787, lat: 46.5963832 }, + ]) { + base.nodes.addNode(node); + } + base.ways.addWay({ id: 10, refs: [1, 2, 3], tags: { highway: "footway" } }); + base.buildIndexes(); + base.buildSpatialIndexes(); + + const patch = new Osm({ id: "short-crossing-patch" }); + patch.nodes.addNode({ id: 101, lon: -120.561871, lat: 46.5963653 }); + patch.nodes.addNode({ id: 102, lon: -120.5618605, lat: 46.5963786 }); + patch.ways.addWay({ id: 20, refs: [101, 102], tags: { highway: "footway" } }); + patch.buildIndexes(); + + const progress: string[] = []; + const result = await merge( + base, + patch, + { createIntersections: true, directMerge: true }, + (event) => progress.push(event.detail.msg), + ); + + const baseWay = result.ways.getById(10)!; + const patchWay = result.ways.getById(20)!; + const generatedRefs = patchWay.refs.filter((ref) => ref > 102); + + expect(patchWay.refs).toContain(2); + expect(generatedRefs).toHaveLength(1); + expect(baseWay.refs).toContain(generatedRefs[0]!); + expect(new Set(patchWay.refs).size).toBe(patchWay.refs.length); + expect(progress).toContain("Intersection creation progress: 1 of 1 ways checked"); + }); + + it("reports every patch way after exact reconciliation removes an equivalent way", async () => { + const base = new Osm({ id: "progress-base" }); + base.nodes.addNode({ id: 1, lon: 0, lat: 0 }); + base.nodes.addNode({ id: 2, lon: 1, lat: 0 }); + base.ways.addWay({ id: 10, refs: [1, 2], tags: { highway: "service" } }); + base.buildIndexes(); + base.buildSpatialIndexes(); + + const patch = new Osm({ id: "progress-patch" }); + patch.ways.addWay({ id: 20, refs: [1, 2], tags: { highway: "service" } }); + patch.buildIndexes(); + + const progress: string[] = []; + const result = await merge( + base, + patch, + { + createIntersections: true, + deduplicateWays: true, + directMerge: true, + }, + (event) => progress.push(event.detail.msg), + ); + + expect(result.ways.ids.has(20)).toBe(false); + expect(progress).toContain("Intersection creation progress: 1 of 1 ways checked"); + }); + + it("declines endpoint reuse when node tags conflict", () => { + const osm = new Osm({ id: "conflicting-endpoint" }); + for (const node of [ + { id: 1, lon: -1, lat: 0 }, + { id: 2, lon: 0, lat: 0, tags: { barrier: "gate" } }, + { id: 5, lon: 0, lat: 0, tags: { barrier: "lift_gate" } }, + { id: 6, lon: 0, lat: 1 }, + ]) { + osm.nodes.addNode(node); + } + osm.ways.addWay({ id: 10, refs: [1, 2], tags: { highway: "service" } }); + osm.ways.addWay({ id: 20, refs: [5, 6], tags: { highway: "service" } }); + osm.buildIndexes(); + osm.buildSpatialIndexes(); + const changeset = new OsmChangeset(osm); + + changeset.createIntersectionsForWays(osm.ways); + + expect(changeset.stats.intersectionPointsFound).toBe(0); + const result = applyChangesetToOsm(changeset); + expect(result.ways.getById(10)?.refs).toEqual([1, 2]); + expect(result.ways.getById(20)?.refs).toEqual([5, 6]); + }); + + it("reuses one pending node when three ways cross at the same point", () => { + const osm = new Osm({ id: "three-way-intersection" }); + for (const node of [ + { id: 1, lon: -1, lat: 0 }, + { id: 2, lon: 1, lat: 0 }, + { id: 3, lon: 0, lat: -1 }, + { id: 4, lon: 0, lat: 1 }, + { id: 5, lon: -1, lat: -1 }, + { id: 6, lon: 1, lat: 1 }, + ]) { + osm.nodes.addNode(node); + } + osm.ways.addWay({ id: 10, refs: [1, 2], tags: { highway: "primary" } }); + osm.ways.addWay({ id: 20, refs: [3, 4], tags: { highway: "secondary" } }); + osm.ways.addWay({ id: 30, refs: [5, 6], tags: { highway: "residential" } }); + osm.buildIndexes(); + osm.buildSpatialIndexes(); + const changeset = new OsmChangeset(osm); + + changeset.createIntersectionsForWays(osm.ways); + + expect(changeset.stats.intersectionNodesCreated).toBe(1); + const result = applyChangesetToOsm(changeset); + const sharedRefs = [10, 20, 30].map( + (wayId) => result.ways.getById(wayId)!.refs.find((ref) => ref > 6)!, + ); + expect(new Set(sharedRefs)).toEqual(new Set([7])); + for (const wayId of [10, 20, 30]) { + const way = result.ways.getById(wayId)!; + const coordinates = way.refs.map((ref) => { + const node = result.nodes.getById(ref)!; + return [node.lon, node.lat]; + }); + expect( + coordinates.some( + (coordinate, index) => + index > 0 && + coordinate[0] === coordinates[index - 1]![0] && + coordinate[1] === coordinates[index - 1]![1], + ), + ).toBe(false); + } + }); + it("skips incomplete current and candidate ways without inventing geometry", () => { const osm = crossingWays(); const changeset = new OsmChangeset(osm); diff --git a/packages/change/test/merge.test.ts b/packages/change/test/merge.test.ts index d9d7507f..38953ddb 100644 --- a/packages/change/test/merge.test.ts +++ b/packages/change/test/merge.test.ts @@ -4,6 +4,7 @@ import { describe, expect, it } from "vitest"; import { applyChangesetToOsm } from "../src/apply-changeset"; import { OsmChangeset } from "../src/changeset"; +import { generateChangeset } from "../src/generate-changeset"; const sizes = (osm: Osm) => ({ nodes: osm.nodes.size, @@ -15,6 +16,7 @@ describe("merge osm", () => { it("should generate and apply osm changes", () => { const base = createMockBaseOsm(); const patch = createMockPatchOsm(); + base.buildSpatialIndexes(); expect(sizes(base)).toEqual({ nodes: 2, @@ -59,25 +61,28 @@ describe("merge osm", () => { }, }); - changeset = new OsmChangeset(directResult); - changeset.deduplicateWays(patch.ways); - changeset.deduplicateNodes(patch.nodes); + changeset = generateChangeset(base, patch, { + directMerge: true, + deduplicateNodes: true, + deduplicateWays: true, + }); const deduplicatedResult = applyChangesetToOsm(changeset, "deduplicated"); - // Node 0 is deleted because node 2 has more tags (version/tags logic) - expect(deduplicatedResult.nodes.ids.has(0)).toBe(false); + // The immutable base node survives and receives non-conflicting patch tags. + expect(deduplicatedResult.nodes.ids.has(0)).toBe(true); + expect(deduplicatedResult.nodes.ids.has(2)).toBe(false); expect(deduplicatedResult.ways.getById(1)).toEqual({ id: 1, - refs: [2, 1], // Node 0 replaced with node 2 + refs: [0, 1], tags: { highway: "primary", version: "2", }, }); + expect(deduplicatedResult.ways.getById(2)?.refs).toEqual([0, 3]); - // Node 2 is kept because it has tags - expect(deduplicatedResult.nodes.getById(2)).toEqual({ - id: 2, + expect(deduplicatedResult.nodes.getById(0)).toEqual({ + id: 0, lat: 46.60207, lon: -120.505898, tags: { diff --git a/packages/change/test/relation-dedup.test.ts b/packages/change/test/relation-dedup.test.ts index f914316d..7859fc3d 100644 --- a/packages/change/test/relation-dedup.test.ts +++ b/packages/change/test/relation-dedup.test.ts @@ -21,7 +21,7 @@ function createOsm(nodes: OsmNode[], ways: OsmWay[], relations: OsmRelation[]) { } describe("relation-safe deduplication", () => { - it("returns flattened node maps and rewrites node members before deletion", () => { + it("does not collapse merely nearby nodes or rewrite their relation members", () => { const nodes: OsmNode[] = [ { id: 1, lat: 0, lon: 0 }, { id: 2, lat: 0.000007, lon: 0 }, @@ -41,18 +41,14 @@ describe("relation-safe deduplication", () => { const replacements = changeset.deduplicateNodes(osm.nodes); - expect(replacements).toEqual( - new Map([ - [1, 3], - [2, 3], - ]), - ); + expect(replacements).toEqual(new Map()); const result = applyChangesetToOsm(changeset); - expect(result.nodes.ids.has(1)).toBe(false); - expect(result.nodes.ids.has(2)).toBe(false); - expect(result.ways.getById(10)?.refs).toEqual([3]); + expect(result.nodes.ids.has(1)).toBe(true); + expect(result.nodes.ids.has(2)).toBe(true); + expect(result.ways.getById(10)?.refs).toEqual([1, 3]); expect(result.relations.getById(20)?.members).toEqual([ - { type: "node", ref: 3, role: "stop" }, + { type: "node", ref: 1, role: "stop" }, + { type: "node", ref: 2, role: "stop" }, { type: "node", ref: 3, role: "platform" }, ]); }); diff --git a/packages/change/test/routing-integrity.test.ts b/packages/change/test/routing-integrity.test.ts new file mode 100644 index 00000000..4a724a07 --- /dev/null +++ b/packages/change/test/routing-integrity.test.ts @@ -0,0 +1,633 @@ +import { Osm } from "@osmix/core"; +import type { OsmNode, OsmRelation, OsmWay } from "@osmix/types"; +import { describe, expect, it } from "vitest"; + +import { applyChangesetToOsm } from "../src/apply-changeset.ts"; +import { generateChangeset } from "../src/generate-changeset.ts"; +import { merge } from "../src/merge.ts"; + +function createOsm( + id: string, + nodes: OsmNode[], + ways: OsmWay[] = [], + relations: OsmRelation[] = [], +) { + const osm = new Osm({ id }); + for (const node of nodes) osm.nodes.addNode(node); + for (const way of ways) osm.ways.addWay(way); + for (const relation of relations) osm.relations.addRelation(relation); + osm.buildIndexes(); + osm.buildSpatialIndexes(); + return osm; +} + +const silent = () => {}; + +describe("routing-safe merge reconciliation", () => { + it("keeps an empty-patch merge as an identity operation", async () => { + const base = createOsm( + "base", + [ + { id: 1, lon: 0, lat: 0 }, + { id: 2, lon: 0.000005, lat: 0 }, + ], + [{ id: 10, refs: [1, 2], tags: { highway: "residential" } }], + ); + const patch = createOsm("empty", []); + + const result = await merge( + base, + patch, + { directMerge: true, deduplicateNodes: true, deduplicateWays: true }, + silent, + ); + + expect([...result.nodes].map((node) => node.id)).toEqual([1, 2]); + expect(result.ways.getById(10)?.refs).toEqual([1, 2]); + }); + + it("does not reconcile nearby or grade-separated nodes", async () => { + const base = createOsm( + "base", + [ + { id: 1, lon: 0, lat: 0 }, + { id: 2, lon: 0.01, lat: 0 }, + ], + [ + { + id: 10, + refs: [1, 2], + tags: { highway: "secondary", layer: "-1", tunnel: "yes" }, + }, + ], + ); + const patch = createOsm( + "patch", + [ + { id: 101, lon: 0, lat: 0 }, + { id: 102, lon: 0.000005, lat: 0 }, + { id: 103, lon: 0.01, lat: 0.01 }, + ], + [ + { id: 20, refs: [101, 103], tags: { highway: "secondary" } }, + { id: 21, refs: [102, 103], tags: { highway: "residential" } }, + ], + ); + + const result = await merge(base, patch, { directMerge: true, deduplicateNodes: true }, silent); + + expect(result.nodes.ids.has(101)).toBe(true); + expect(result.nodes.ids.has(102)).toBe(true); + expect(result.ways.getById(20)?.refs).toEqual([101, 103]); + expect(result.ways.getById(21)?.refs).toEqual([102, 103]); + }); + + it("rejects conflicting node tags and preserves non-conflicting descriptive tags", async () => { + const base = createOsm( + "base", + [ + { id: 1, lon: 0, lat: 0, tags: { amenity: "cafe" } }, + { id: 2, lon: -1, lat: 0 }, + { id: 3, lon: 1, lat: 0 }, + ], + [{ id: 10, refs: [2, 1], tags: { highway: "residential" } }], + ); + const patch = createOsm( + "patch", + [ + { id: 101, lon: 0, lat: 0, tags: { amenity: "school" } }, + { id: 102, lon: 1, lat: 0, tags: { name: "Patch endpoint" } }, + { id: 103, lon: 0, lat: 1 }, + ], + [ + { id: 20, refs: [101, 103], tags: { highway: "residential" } }, + { id: 21, refs: [102, 103], tags: { highway: "residential" } }, + ], + ); + + const result = await merge(base, patch, { directMerge: true, deduplicateNodes: true }, silent); + + expect(result.nodes.ids.has(101)).toBe(true); + expect(result.ways.getById(20)?.refs).toEqual([101, 103]); + expect(result.nodes.ids.has(102)).toBe(false); + expect(result.nodes.getById(3)?.tags).toEqual({ name: "Patch endpoint" }); + expect(result.ways.getById(21)?.refs).toEqual([3, 103]); + }); + + it("rejects a candidate when any incident way has incompatible context", async () => { + const base = createOsm( + "base", + [ + { id: 1, lon: 0, lat: 0 }, + { id: 2, lon: -1, lat: 0 }, + { id: 3, lon: 1, lat: 0 }, + ], + [ + { id: 10, refs: [2, 1], tags: { highway: "primary" } }, + { + id: 11, + refs: [1, 3], + tags: { highway: "primary", layer: "-1", tunnel: "yes" }, + }, + ], + ); + const patch = createOsm( + "patch", + [ + { id: 101, lon: 0, lat: 0 }, + { id: 102, lon: 0, lat: 1 }, + ], + [{ id: 20, refs: [101, 102], tags: { highway: "secondary" } }], + ); + + const result = await merge(base, patch, { directMerge: true, deduplicateNodes: true }, silent); + + expect(result.nodes.ids.has(101)).toBe(true); + expect(result.ways.getById(20)?.refs).toEqual([101, 102]); + }); + + it("keeps same-ID patch nodes authoritative instead of deleting a base identity", async () => { + const base = createOsm( + "base", + [ + { id: 1, lon: 0, lat: 0 }, + { id: 2, lon: 1, lat: 0 }, + { id: 3, lon: 0, lat: 1 }, + ], + [{ id: 10, refs: [1, 3], tags: { highway: "residential" } }], + ); + const patch = createOsm("patch", [{ id: 1, lon: 1, lat: 0 }]); + + const result = await merge(base, patch, { directMerge: true, deduplicateNodes: true }, silent); + + expect(result.nodes.ids.has(1)).toBe(true); + expect(result.nodes.ids.has(2)).toBe(true); + expect(result.nodes.getById(1)).toMatchObject({ lon: 1, lat: 0 }); + expect(result.ways.getById(10)?.refs).toEqual([1, 3]); + }); + + it("does not collapse a routable patch way to one distinct base node", async () => { + const base = createOsm("base", [{ id: 1, lon: 0, lat: 0 }]); + const patch = createOsm( + "patch", + [ + { id: 101, lon: 0, lat: 0 }, + { id: 102, lon: 0, lat: 0 }, + ], + [{ id: 20, refs: [101, 102], tags: { highway: "service" } }], + ); + + const result = await merge(base, patch, { directMerge: true, deduplicateNodes: true }, silent); + + expect(result.nodes.ids.has(101)).toBe(true); + expect(result.nodes.ids.has(102)).toBe(true); + expect(result.ways.getById(20)?.refs).toEqual([101, 102]); + }); + + it("does not reconcile ways with conflicting routing tags", async () => { + const base = createOsm( + "base", + [ + { id: 1, lon: 0, lat: 0 }, + { id: 2, lon: 1, lat: 0 }, + ], + [{ id: 10, refs: [1, 2], tags: { highway: "residential" } }], + ); + const patch = createOsm( + "patch", + [], + [{ id: 20, refs: [1, 2], tags: { highway: "residential", oneway: "yes" } }], + ); + + const result = await merge(base, patch, { directMerge: true, deduplicateWays: true }, silent); + + expect(result.ways.ids.has(10)).toBe(true); + expect(result.ways.ids.has(20)).toBe(true); + }); + + it("does not reconcile ways with conditional access semantics", async () => { + const base = createOsm( + "base", + [ + { id: 1, lon: 0, lat: 0 }, + { id: 2, lon: 1, lat: 0 }, + ], + [{ id: 10, refs: [1, 2], tags: { highway: "residential" } }], + ); + const patch = createOsm( + "patch", + [], + [ + { + id: 20, + refs: [1, 2], + tags: { + "access:conditional": "no @ (Mo-Fr 07:00-09:00)", + highway: "residential", + name: "School Street", + }, + }, + ], + ); + + const result = await merge(base, patch, { directMerge: true, deduplicateWays: true }, silent); + + expect(result.ways.ids.has(10)).toBe(true); + expect(result.ways.ids.has(20)).toBe(true); + expect(result.ways.getById(10)?.tags).toEqual({ highway: "residential" }); + }); + + it("copies only non-conflicting descriptive tags when ways reconcile", async () => { + const base = createOsm( + "base", + [ + { id: 1, lon: 0, lat: 0 }, + { id: 2, lon: 1, lat: 0 }, + ], + [{ id: 10, refs: [1, 2], tags: { highway: "residential" } }], + ); + const patch = createOsm( + "patch", + [], + [{ id: 20, refs: [1, 2], tags: { highway: "residential", name: "Connector" } }], + ); + + const result = await merge(base, patch, { directMerge: true, deduplicateWays: true }, silent); + + expect(result.ways.ids.has(20)).toBe(false); + expect(result.ways.getById(10)?.tags).toEqual({ + highway: "residential", + name: "Connector", + }); + }); + + it("checks complete way semantics when exact-index hashes collide", async () => { + const base = createOsm( + "base", + [ + { id: 1, lon: 0, lat: 0 }, + { id: 2, lon: 1, lat: 0 }, + ], + [ + { + id: 10, + refs: [1, 2], + tags: { highway: "residential", surface: "1ugdp92ail" }, + }, + { + id: 11, + refs: [1, 2], + tags: { highway: "residential", surface: "c9n7431ir0" }, + }, + ], + ); + const patch = createOsm( + "patch", + [], + [ + { + id: 20, + refs: [1, 2], + tags: { + highway: "residential", + name: "Matching target", + surface: "c9n7431ir0", + }, + }, + ], + ); + + const result = await merge(base, patch, { directMerge: true, deduplicateWays: true }, silent); + + expect(result.ways.ids.has(20)).toBe(false); + expect(result.ways.getById(10)?.tags?.["name"]).toBeUndefined(); + expect(result.ways.getById(11)?.tags?.["name"]).toBe("Matching target"); + }); + + it("rejects patch dangling refs even when they already exist in the patch", async () => { + const base = createOsm("base", []); + const patch = createOsm( + "patch", + [{ id: 101, lon: 0, lat: 0 }], + [{ id: 20, refs: [101, 999], tags: { highway: "service" } }], + ); + + await expect(merge(base, patch, { directMerge: true }, silent)).rejects.toThrow( + "way 20 references missing node 999", + ); + }); + + it("rejects a new patch restriction that is detached in the merged network", async () => { + const base = createOsm( + "base", + [ + { id: 1, lon: 0, lat: 0 }, + { id: 2, lon: 1, lat: 0 }, + { id: 3, lon: 2, lat: 0 }, + { id: 4, lon: 3, lat: 0 }, + ], + [ + { id: 10, refs: [1, 2], tags: { highway: "primary" } }, + { id: 20, refs: [3, 4], tags: { highway: "primary" } }, + ], + ); + const patch = createOsm( + "patch", + [], + [], + [ + { + id: 100, + tags: { type: "restriction", restriction: "no_left_turn" }, + members: [ + { type: "way", ref: 10, role: "from" }, + { type: "node", ref: 2, role: "via" }, + { type: "way", ref: 20, role: "to" }, + ], + }, + ], + ); + + await expect(merge(base, patch, { directMerge: true }, silent)).rejects.toThrow( + "restriction 100 via node 2 is detached", + ); + }); + + it("rewrites pending restriction via-node members with reconciled patch nodes", async () => { + const base = createOsm( + "base", + [ + { id: 1, lon: 0, lat: 0 }, + { id: 2, lon: -1, lat: 0 }, + { id: 3, lon: 1, lat: 0 }, + ], + [{ id: 20, refs: [1, 3], tags: { highway: "primary" } }], + ); + const patch = createOsm( + "patch", + [ + { id: 101, lon: 0, lat: 0 }, + { id: 102, lon: -1, lat: 0 }, + ], + [{ id: 30, refs: [102, 101], tags: { highway: "primary" } }], + [ + { + id: 100, + tags: { type: "restriction", restriction: "no_left_turn" }, + members: [ + { type: "way", ref: 30, role: "from" }, + { type: "node", ref: 101, role: "via" }, + { type: "way", ref: 20, role: "to" }, + ], + }, + ], + ); + + const result = await merge(base, patch, { directMerge: true, deduplicateNodes: true }, silent); + + expect(result.ways.getById(30)?.refs).toEqual([2, 1]); + expect(result.relations.getById(100)?.members[1]).toEqual({ + type: "node", + ref: 1, + role: "via", + }); + }); + + it("rejects same-ID changes that detach a valid restriction via node", async () => { + const base = createOsm( + "base", + [ + { id: 1, lon: 0, lat: 0 }, + { id: 2, lon: 1, lat: 0 }, + { id: 3, lon: 2, lat: 0 }, + { id: 4, lon: 1, lat: 1 }, + ], + [ + { id: 10, refs: [1, 2], tags: { highway: "primary" } }, + { id: 20, refs: [2, 3], tags: { highway: "primary" } }, + ], + [ + { + id: 100, + tags: { type: "restriction", restriction: "no_left_turn" }, + members: [ + { type: "way", ref: 10, role: "from" }, + { type: "node", ref: 2, role: "via" }, + { type: "way", ref: 20, role: "to" }, + ], + }, + ], + ); + const patch = createOsm( + "patch", + [ + { id: 3, lon: 2, lat: 0 }, + { id: 4, lon: 1, lat: 1 }, + ], + [{ id: 20, refs: [4, 3], tags: { highway: "primary" } }], + ); + + await expect(merge(base, patch, { directMerge: true }, silent)).rejects.toThrow( + "restriction 100 via node 2 is detached", + ); + }); + + it("rejects newly connected highways with incompatible grade signatures", async () => { + const base = createOsm( + "base", + [ + { id: 1, lon: 0, lat: 0 }, + { id: 2, lon: -1, lat: 0 }, + { id: 3, lon: 1, lat: 0 }, + { id: 4, lon: 0, lat: 1 }, + ], + [ + { id: 10, refs: [2, 1, 4], tags: { highway: "primary" } }, + { id: 20, refs: [1, 3], tags: { highway: "primary" } }, + ], + ); + const patch = createOsm( + "patch", + [], + [ + { + id: 20, + refs: [1, 3], + tags: { highway: "primary", layer: "-1", tunnel: "yes" }, + }, + ], + ); + + await expect(merge(base, patch, { directMerge: true }, silent)).rejects.toThrow( + "node 1 newly connects grade-separated highways 10 and 20", + ); + }); + + it("allows a surface road endpoint to transition into a bridge endpoint", async () => { + const base = createOsm( + "base", + [ + { id: 1, lon: 0, lat: 0 }, + { id: 2, lon: 1, lat: 0 }, + ], + [{ id: 10, refs: [1, 2], tags: { highway: "primary" } }], + ); + const patch = createOsm( + "patch", + [ + { id: 2, lon: 1, lat: 0 }, + { id: 3, lon: 2, lat: 0 }, + ], + [ + { + id: 20, + refs: [2, 3], + tags: { highway: "primary", bridge: "yes", layer: "1" }, + }, + ], + ); + + const result = await merge(base, patch, { directMerge: true }, silent); + + expect(result.ways.getById(20)?.refs).toEqual([2, 3]); + }); + + it("allows an interior way at a bridge portal with a same-grade endpoint continuation", async () => { + const base = createOsm( + "base", + [ + { id: 1, lon: -1, lat: 0 }, + { id: 2, lon: 0, lat: 0 }, + { id: 3, lon: 1, lat: 0 }, + { id: 4, lon: 0, lat: 1 }, + ], + [ + { id: 10, refs: [1, 2, 3], tags: { highway: "footway" } }, + { id: 30, refs: [2, 4], tags: { highway: "primary" } }, + ], + ); + const patch = createOsm( + "patch", + [ + { id: 2, lon: 0, lat: 0 }, + { id: 5, lon: 0, lat: -1 }, + ], + [ + { + id: 20, + refs: [2, 5], + tags: { highway: "primary", bridge: "yes", layer: "1" }, + }, + ], + ); + + const result = await merge(base, patch, { directMerge: true }, silent); + + expect(result.ways.getById(10)?.refs).toEqual([1, 2, 3]); + expect(result.ways.getById(20)?.refs).toEqual([2, 5]); + expect(result.ways.getById(30)?.refs).toEqual([2, 4]); + }); + + it("rejects a surface endpoint spliced into an interior tunnel despite a tunnel continuation", async () => { + const base = createOsm( + "base", + [ + { id: 1, lon: -1, lat: 0 }, + { id: 2, lon: 0, lat: 0 }, + { id: 3, lon: 1, lat: 0 }, + { id: 4, lon: 0, lat: 1 }, + ], + [ + { + id: 10, + refs: [1, 2, 3], + tags: { highway: "primary", layer: "-1", tunnel: "yes" }, + }, + { + id: 30, + refs: [2, 4], + tags: { highway: "primary", layer: "-1", tunnel: "yes" }, + }, + ], + ); + const patch = createOsm( + "patch", + [ + { id: 2, lon: 0, lat: 0 }, + { id: 5, lon: 0, lat: -1 }, + ], + [{ id: 20, refs: [2, 5], tags: { highway: "primary" } }], + ); + + await expect(merge(base, patch, { directMerge: true }, silent)).rejects.toThrow( + "node 2 newly connects grade-separated highways 10 and 20", + ); + }); + + it("tolerates an inherited interior grade issue during an unrelated change", () => { + const base = createOsm( + "base", + [ + { id: 1, lon: -1, lat: 0 }, + { id: 2, lon: 0, lat: 0 }, + { id: 3, lon: 1, lat: 0 }, + { id: 4, lon: 0, lat: 1 }, + ], + [ + { id: 10, refs: [1, 2, 3], tags: { highway: "primary" } }, + { + id: 20, + refs: [2, 4], + tags: { highway: "primary", bridge: "yes", layer: "1" }, + }, + ], + ); + const changeset = generateChangeset( + base, + createOsm("patch", [{ id: 1, lon: -1, lat: 0, tags: { name: "Unrelated" } }]), + { directMerge: true }, + silent, + ); + + expect(() => applyChangesetToOsm(changeset)).not.toThrow(); + }); + + it("rejects direct merge plus intersections in one generated changeset", () => { + const base = createOsm("base", []); + const patch = createOsm("patch", []); + + expect(() => + generateChangeset(base, patch, { directMerge: true, createIntersections: true }, silent), + ).toThrow("generateChangeset cannot combine directMerge with createIntersections"); + }); + + it("keeps high-level and generated changeset reconciliation in parity", async () => { + const base = createOsm( + "base", + [ + { id: 1, lon: 0, lat: 0 }, + { id: 2, lon: 1, lat: 0 }, + ], + [{ id: 10, refs: [1, 2], tags: { highway: "residential" } }], + ); + const patch = createOsm( + "patch", + [ + { id: 101, lon: 1, lat: 0 }, + { id: 102, lon: 2, lat: 0 }, + ], + [{ id: 20, refs: [101, 102], tags: { highway: "residential" } }], + ); + const options = { directMerge: true, deduplicateNodes: true, deduplicateWays: true }; + + const highLevel = await merge(base, patch, options, silent); + const generated = applyChangesetToOsm(generateChangeset(base, patch, options, silent)); + + expect([...highLevel.nodes].map((node) => node.id)).toEqual( + [...generated.nodes].map((node) => node.id), + ); + expect([...highLevel.ways].map((way) => way.refs)).toEqual( + [...generated.ways].map((way) => way.refs), + ); + }); +}); diff --git a/packages/change/test/ways-intersect.test.ts b/packages/change/test/ways-intersect.test.ts index 8cd7b7d6..ae84081d 100644 --- a/packages/change/test/ways-intersect.test.ts +++ b/packages/change/test/ways-intersect.test.ts @@ -1,5 +1,8 @@ import { describe, expect, it } from "vitest"; +import sweeplineIntersections, { + sweeplineLineIntersections, +} from "../src/sweepline-intersections.ts"; import { waysIntersect } from "../src/utils.ts"; type Point = [number, number]; @@ -116,4 +119,27 @@ describe("waysIntersect", () => { it.each(cases)("matches pinned behavior for %s", (_name, wayA, wayB, expected) => { expect(waysIntersect(wayA, wayB)).toEqual(expected); }); + + it.each(cases)("keeps the direct line entry point equivalent for %s", (_name, wayA, wayB) => { + const wrapped = sweeplineIntersections( + { + type: "FeatureCollection", + features: [ + { + type: "Feature", + geometry: { type: "LineString", coordinates: wayA }, + properties: {}, + }, + { + type: "Feature", + geometry: { type: "LineString", coordinates: wayB }, + properties: {}, + }, + ], + }, + true, + ); + + expect(sweeplineLineIntersections(wayA, wayB)).toEqual(wrapped); + }); }); diff --git a/packages/core/src/entities.ts b/packages/core/src/entities.ts index 3fe140ad..71dc55ec 100644 --- a/packages/core/src/entities.ts +++ b/packages/core/src/entities.ts @@ -241,6 +241,13 @@ export abstract class Entities<T extends OsmEntity> { } } + /** @internal Iterate entities in canonical OSM file order. */ + *osmSorted(): Generator<T> { + for (const [id, index] of this.ids.osmSortedEntries()) { + yield this.getFullEntity(index, id, this.tags.getTags(index)); + } + } + /** * Search for entities with a specific tag key and optional value. */ diff --git a/packages/core/src/ids.ts b/packages/core/src/ids.ts index 44561e1f..14fa2ba7 100644 --- a/packages/core/src/ids.ts +++ b/packages/core/src/ids.ts @@ -242,18 +242,50 @@ export class Ids { /** @internal Iterate sorted IDs with their original storage positions. */ *sortedEntries(): Generator<readonly [id: number, index: number]> { for (let i = 0; i < this.idsSorted.length; i++) { - const id = this.idsSorted[i]; + yield this.sortedEntry(i); + } + } + + /** + * Iterate in canonical OSM file order: negative IDs first by increasing + * absolute value, followed by non-negative IDs in ascending order. + * + * @internal + */ + *osmSortedEntries(): Generator<readonly [id: number, index: number]> { + let firstNonNegative = 0; + while (firstNonNegative < this.idsSorted.length && this.idsSorted[firstNonNegative]! < 0) { + firstNonNegative++; + } + + // Numeric sorting places negative IDs in the opposite of canonical OSM + // order. Reverse ID groups while preserving duplicate insertion order. + let groupEnd = firstNonNegative; + while (groupEnd > 0) { + const id = this.idsSorted[groupEnd - 1]; assertValue(id, "Sorted ID is undefined"); - if (this.idsAreSorted) { - yield [id, i]; - } else { - const index = this.sortedIdPositionToIndex[i]; - assertValue(index, "Sorted position is undefined"); - yield [id, index]; + let groupStart = groupEnd - 1; + while (groupStart > 0 && this.idsSorted[groupStart - 1] === id) groupStart--; + for (let position = groupStart; position < groupEnd; position++) { + yield this.sortedEntry(position); } + groupEnd = groupStart; + } + + for (let position = firstNonNegative; position < this.idsSorted.length; position++) { + yield this.sortedEntry(position); } } + private sortedEntry(position: number): readonly [id: number, index: number] { + const id = this.idsSorted[position]; + assertValue(id, "Sorted ID is undefined"); + if (this.idsAreSorted) return [id, position]; + const index = this.sortedIdPositionToIndex[position]; + assertValue(index, "Sorted position is undefined"); + return [id, index]; + } + /** * Get transferable buffers for passing to another thread. * @returns Serializable representation of this index. diff --git a/packages/core/test/ids.test.ts b/packages/core/test/ids.test.ts index aa178ddd..60a1fae4 100644 --- a/packages/core/test/ids.test.ts +++ b/packages/core/test/ids.test.ts @@ -66,6 +66,18 @@ describe("Ids sorted entries", () => { ]); }); + it("orders negative IDs canonically for OSM serialization", () => { + expect(Array.from(buildIds([3, -3, -1, 2, -2, 1, -2]).osmSortedEntries())).toEqual([ + [-1, 2], + [-2, 4], + [-2, 6], + [-3, 1], + [1, 5], + [2, 3], + [3, 0], + ]); + }); + it("omits redundant sorted buffers for ascending IDs", () => { const transferables = buildIds([1, 2, 3]).transferables(); diff --git a/packages/geoparquet/test/monaco-parquet.test.ts b/packages/geoparquet/test/monaco-parquet.test.ts index df268efe..b74c06e8 100644 --- a/packages/geoparquet/test/monaco-parquet.test.ts +++ b/packages/geoparquet/test/monaco-parquet.test.ts @@ -1,7 +1,7 @@ import { readFile, stat } from "node:fs/promises"; import { getFixturePath } from "@osmix/test-utils/fixtures"; -import { describe, expect, it } from "vitest"; +import { beforeAll, describe, expect, it } from "vitest"; import { fromGeoParquet, GeoParquetOsmBuilder } from "../src"; @@ -22,11 +22,25 @@ import { fromGeoParquet, GeoParquetOsmBuilder } from "../src"; describe("@osmix/geoparquet: Monaco highways fixture", () => { const fixturePath = () => getFixturePath("monaco.parquet"); - const readFixture = async (): Promise<ArrayBuffer> => { - const buffer = await readFile(fixturePath()); - return buffer.buffer.slice(buffer.byteOffset, buffer.byteOffset + buffer.byteLength); + let fixturePromise: Promise<ArrayBuffer> | undefined; + let osmPromise: ReturnType<typeof fromGeoParquet> | undefined; + const readFixture = (): Promise<ArrayBuffer> => { + fixturePromise ??= readFile(fixturePath()).then((buffer) => + buffer.buffer.slice(buffer.byteOffset, buffer.byteOffset + buffer.byteLength), + ); + return fixturePromise; }; - const getOsm = async () => fromGeoParquet(await readFixture()); + const getOsm = () => { + osmPromise ??= readFixture().then((fixture) => fromGeoParquet(fixture)); + return osmPromise; + }; + + // Conversion is the expensive integration boundary and the resulting OSM is + // read-only in this suite. Build it once instead of repeating the same 5,308- + // feature conversion in every behavioral assertion. + beforeAll(async () => { + await getOsm(); + }); it("should load the monaco.parquet fixture", async () => { const { size } = await stat(fixturePath()); diff --git a/packages/load/src/entity-stream.ts b/packages/load/src/entity-stream.ts index 2e653d64..4eab22af 100644 --- a/packages/load/src/entity-stream.ts +++ b/packages/load/src/entity-stream.ts @@ -9,20 +9,20 @@ import type { OsmPbfHeaderBlock } from "@osmix/pbf"; import type { OsmEntity } from "@osmix/types"; function* getAllEntitiesSorted(osm: Osm): Generator<OsmEntity> { - for (const node of osm.nodes.sorted()) { + for (const node of osm.nodes.osmSorted()) { yield node; } - for (const way of osm.ways.sorted()) { + for (const way of osm.ways.osmSorted()) { yield way; } - for (const relation of osm.relations.sorted()) { + for (const relation of osm.relations.osmSorted()) { yield relation; } } /** * Convert the `Osm` index to a `ReadableStream` of header and entity objects. - * Header is emitted first, followed by all entities in sorted order. + * Header is emitted first, followed by all entities in canonical OSM order. * Stream can be piped through transform streams for further processing. */ export function createReadableEntityStreamFromOsm( diff --git a/packages/osmix/README.md b/packages/osmix/README.md index 0e169e3d..e85b2dbd 100644 --- a/packages/osmix/README.md +++ b/packages/osmix/README.md @@ -44,6 +44,98 @@ const rasterTile = await merged.getRasterTile([10561, 22891, 16]); console.log(rasterTile.byteLength); ``` +High-level merges leave the original inputs intact and reconcile only compatible patch entities with unique +base matches. Regenerate PBFs created by older releases from their original inputs if automatic within-file +deduplication may already have rewritten routing topology. + +### Profile merge performance + +The test-only merge profiler runs the reviewed merge stages in their production order and reports per-stage +wall time, CPU time, RSS, heap use, operation counts, and output fingerprints as JSON. The PBF fingerprint +normalizes the export header's current timestamp; all entity bytes still use the production serializer. Monaco +is checked into the repository and is the safe default: + +```sh +pnpm --filter osmix profile:merge -- --scenario monaco --runs 5 --output /tmp/monaco-merge.json +``` + +Two larger profiles use ignored local fixtures. They never download data and fail with the required path when +a fixture is absent: + +```sh +# Recommended Yakima property keys, 1-meter matching, network attachment, and all merge stages. +pnpm --filter osmix profile:merge -- --scenario yakima --runs 3 --output /tmp/yakima-merge.json + +# Direct merge, exact reconciliation, and intersections for the reported full-merge regression. +pnpm --filter osmix profile:merge -- --scenario eastern-washington --runs 1 \ + --output /tmp/eastern-washington-merge.json +``` + +The Yakima scenario uses `OsmixWorker.generateConflationChangeset`. Its generation stage includes CAR/WALK +routing diagnostics and the automatic network-attachment CAR safety projection. + +Yakima requires `fixtures/yakima-full.osm.pbf` and `fixtures/yakima.osw.pbf`. Eastern Washington requires +`fixtures/osmix-e_wa_osm.pbf` and `fixtures/east_washington_sidewalk_proviso_1.pbf`. The existing Eastern +Washington correctness test remains opt-in with `OSMIX_EASTERN_WASHINGTON_INTEGRATION=1`. + +`OSMIX_MERGE_PROFILE_SCENARIO`, `OSMIX_MERGE_PROFILE_RUNS`, and `OSMIX_MERGE_PROFILE_OUTPUT` are equivalent +to the command-line flags. Compare reports produced with the same commit, Node version, hardware, and idle +system. `processPeakRssBytes` is the process-lifetime high-water mark, so later repetitions can retain an +earlier run's peak. CI verifies operation counts and semantic fingerprints, but intentionally has no timing +threshold or compressed-PBF byte golden. + +Proximity matching for independently created imports is available as a separate opt-in review session. The +recommended defaults use a 1-meter radius and automatically apply only high-confidence candidates: + +```ts check-docs worker-pbf-inputs +import { createRemote } from "osmix"; + +using remote = await createRemote(); +const base = await remote.fromPbf(monacoPbf); +const patch = await remote.fromPbf(patchPbf, { id: "imported-data" }); + +const summary = await remote.discoverConflation(base.id, patch.id, { + propertyKeys: ["name", "operator", "surface"], + attachNetwork: true, +}); +const page = await remote.getConflationPage(base.id, 0, 100); + +// Page previews cover every candidate matching the worker's current filter, not +// just the rows returned on this page. +console.log(page.bulkActions["transfer-properties"]); +await remote.applyConflationBulkDecision(base.id, { + action: "transfer-properties", + filter: { status: "review" }, +}); + +for (const candidate of page.candidates) { + if (candidate.status !== "review") continue; + await remote.setConflationDecision(base.id, { + candidateId: candidate.id, + action: "reject", + }); +} + +const generated = await remote.generateConflationChangeset(base.id, { + directMerge: true, + deduplicateNodes: true, + deduplicateWays: true, +}); +console.log(summary, generated.routing.car, generated.routing.walk); +await remote.applyChangesAndReplace(base.id); +``` + +Property transfer changes only explicitly selected tags. Network attachment rewrites only patch-created way +references. The worker preserves discovery settings, filters, decisions, and generated changes across worker +restarts, and reports CAR/WALK node, edge, and component deltas before the changeset is applied. Automatic +pedestrian attachments are rejected if they alter routable CAR topology. + +Filter-wide decisions are computed and committed atomically in the worker. Property and network actions +accept eligible automatic and review candidates while skipping blocked, unmatched, ambiguous, or structurally +invalid matches. Reject includes every filtered candidate that is not already rejected. Each result returns the +complete decision snapshot for restart recovery, and accepted candidates can be queried with +`{ status: "accepted" }`. + #### Which mode am I in? `createRemote()` picks the best mode the current runtime supports and reports @@ -343,6 +435,17 @@ spec-compliant without staging everything in memory. - `remote.getRasterTile(osmId, tile, tileSize?)` - Generate raster in worker. - `remote.merge(baseId, patchId, options?)` - Merge datasets in worker (legacy). - `dataset.merge(patch, options?)` - Merge datasets via dataset handles. +- `remote.discoverConflation(baseId, patchId, options)` - Start a non-mutating imported-data match session. +- `remote.getConflationSummary(baseId)` - Retrieve decision-aware candidate counts. +- `remote.setConflationFilter(baseId, filter)` / `remote.getConflationPage(...)` - Page through candidate + evidence and review state. +- `remote.setConflationDecision(baseId, decision)` / `remote.setConflationDecisions(...)` - Persist individual + or batch review decisions. +- `remote.applyConflationBulkDecision(baseId, request)` - Atomically apply an action to all candidates matching + the request's filter and return preview counts, the updated summary, and the complete decision snapshot. +- `remote.generateConflationChangeset(baseId, mergeOptions)` - Build one cumulative direct, exact, and fuzzy + changeset and return routing diagnostics. +- `remote.clearConflation(baseId)` - Discard the active review session and any generated changeset. - `remote.search(osmId, key, val?)` - Search by tag. - `remote.toPbf(osmId, stream)` - Export to PBF. diff --git a/packages/osmix/package.json b/packages/osmix/package.json index 8fdbe7a5..349c6473 100644 --- a/packages/osmix/package.json +++ b/packages/osmix/package.json @@ -17,6 +17,7 @@ }, "scripts": { "build": "tsc -p tsconfig.build.json", + "profile:merge": "node --expose-gc --experimental-strip-types test/merge-profile-cli.ts", "test": "pnpm -w exec vitest run --project \"$npm_package_name\"", "typecheck": "tsc --noEmit" }, diff --git a/packages/osmix/src/index.ts b/packages/osmix/src/index.ts index 9e7b65fc..ab2c5d4a 100644 --- a/packages/osmix/src/index.ts +++ b/packages/osmix/src/index.ts @@ -20,7 +20,17 @@ export { type OsmixRemoteOptions, type OsmixWorkerLane, } from "./remote.ts"; -export { OsmixWorker, type RouteResult, type WaySegment } from "./worker.ts"; +export { + OsmixWorker, + type OsmConflationCandidateView, + type OsmConflationGenerationResult, + type OsmConflationPage, + type OsmConflationRoutingDelta, + type OsmConflationRoutingDiagnostics, + type OsmConflationRoutingGraphStats, + type RouteResult, + type WaySegment, +} from "./worker.ts"; export { drawToRasterTile, type DrawToRasterTileOptions } from "./raster.ts"; export { canShareArrayBuffers, @@ -62,7 +72,12 @@ export { // --- @osmix/change --- export { applyChangesetToOsm, + conflationEffectiveStatus, + discoverConflationCandidates, + filterConflationCandidates, generateChangeset, + generateConflationApplicationChangeset, + generateConflationChangeset, generateOscChanges, merge, OsmChangeset, @@ -76,6 +91,8 @@ export { osmTagsToOscTags, removeDuplicateAdjacentRelationMembers, removeDuplicateAdjacentWayRefs, + summarizeConflationCandidates, + validateConflationDecisions, waysIntersect, waysShouldConnect, } from "@osmix/change"; @@ -85,6 +102,26 @@ export type { OsmChanges, OsmChangesetStats, OsmChangeTypes, + OsmConflationActionAssessment, + OsmConflationAutomatic, + OsmConflationBulkAction, + OsmConflationBulkDecisionPreview, + OsmConflationBulkDecisionRequest, + OsmConflationBulkDecisionResult, + OsmConflationCandidate, + OsmConflationCandidateFilter, + OsmConflationDecision, + OsmConflationDiscovery, + OsmConflationEffectiveStatus, + OsmConflationEntityType, + OsmConflationEvidence, + OsmConflationOptions, + OsmConflationReasonCode, + OsmConflationRoutingFamily, + OsmConflationStatus, + OsmConflationSummary, + OsmConflationTagDiff, + ResolvedOsmConflationOptions, OsmEntityRef, OsmMergeOptions, } from "@osmix/change"; diff --git a/packages/osmix/src/remote.ts b/packages/osmix/src/remote.ts index e17ebf91..7a78969a 100644 --- a/packages/osmix/src/remote.ts +++ b/packages/osmix/src/remote.ts @@ -8,7 +8,14 @@ * @module */ -import type { OsmChangeTypes, OsmMergeOptions } from "@osmix/change"; +import type { + OsmChangeTypes, + OsmConflationBulkDecisionRequest, + OsmConflationCandidateFilter, + OsmConflationDecision, + OsmConflationOptions, + OsmMergeOptions, +} from "@osmix/change"; import { Osm, type OsmInfo, type OsmOptions, type OsmTransferables } from "@osmix/core"; import type { GeoParquetReadOptions } from "@osmix/geoparquet"; import { type GtfsConversionOptions, isGtfsZip as isGtfsZipBytes } from "@osmix/gtfs"; @@ -100,8 +107,21 @@ type DatasetProxyMethodName = | "setChangesetFilters" | "getChangesetPage"; +type ConflationDatasetProxyMethodName = + | "applyConflationBulkDecision" + | "discoverConflation" + | "getConflationSummary" + | "setConflationFilter" + | "getConflationPage" + | "setConflationDecision" + | "setConflationDecisions" + | "generateConflationChangeset" + | "clearConflation"; + type OsmRemoteDatasetMethods<T extends OsmixWorker> = { - [K in DatasetProxyMethodName]: BoundDatasetMethod<OsmixRemote<T>[K]>; + [K in DatasetProxyMethodName | ConflationDatasetProxyMethodName]: BoundDatasetMethod< + OsmixRemote<T>[K] + >; }; type DatasetMemberMethodName = "size" | "getById" | "search"; @@ -392,6 +412,18 @@ interface ActiveChangesetState { patchOsmId: string; } +interface ActiveConflationState { + baseOsmId: string; + changeTypes: OsmChangeTypes[]; + changesetGenerated: boolean; + decisions: OsmConflationDecision[]; + entityTypes: OsmEntityType[]; + filter: OsmConflationCandidateFilter; + mergeOptions: Partial<OsmMergeOptions>; + options: OsmConflationOptions; + patchOsmId: string; +} + type DatasetRestorer<T extends OsmixWorker> = ( worker: Comlink.Remote<T>, datasetId: string, @@ -399,6 +431,7 @@ type DatasetRestorer<T extends OsmixWorker> = ( export class OsmixRemote<T extends OsmixWorker = OsmixWorker> { private activeChangeset: ActiveChangesetState | null = null; + private readonly activeConflations = new Map<string, ActiveConflationState>(); private readonly datasetRestorers = new Map<string, DatasetRestorer<T> | null>(); private readonly retainedDatasets = new Map<string, OsmTransferables>(); private readonly retainedLoadDecisions = new Map<string, OsmLoadDecision | null>(); @@ -624,6 +657,29 @@ export class OsmixRemote<T extends OsmixWorker = OsmixWorker> { this.retainedRoutingGraphs.delete(id); } + private invalidateConflationsForDataset(osmId: OsmId): void { + const id = this.getId(osmId); + // Dataset IDs are logical keys and loaders may replace the contents under one. + // Candidate evidence and decisions are invalid as soon as either input changes. + for (const [baseOsmId, state] of this.activeConflations) { + if (baseOsmId === id || state.patchOsmId === id) { + this.activeConflations.delete(baseOsmId); + } + } + if ( + this.activeChangeset && + (this.activeChangeset.baseOsmId === id || this.activeChangeset.patchOsmId === id) + ) { + this.activeChangeset = null; + } + } + + private getActiveConflation(baseOsmId: string): ActiveConflationState { + const state = this.activeConflations.get(baseOsmId); + if (!state) throw Error("No active conflation session"); + return state; + } + /** Mark changed data as known but not reproducible from its original source. */ private markDatasetUnrecoverable(osmId: OsmId): void { const id = this.getId(osmId); @@ -693,6 +749,19 @@ export class OsmixRemote<T extends OsmixWorker = OsmixWorker> { await worker.generateChangeset(state.baseOsmId, state.patchOsmId, state.options); await worker.setChangesetFilters(state.changeTypes, state.entityTypes); } + if (index === 0) { + for (const state of this.activeConflations.values()) { + // Recovery reproduces review state by rediscovering from restored untouched + // inputs, then replaying stable ID-based decisions and filters. + await worker.discoverConflation(state.baseOsmId, state.patchOsmId, state.options); + await worker.setConflationFilter(state.baseOsmId, state.filter); + await worker.setConflationDecisions(state.baseOsmId, state.decisions); + if (state.changesetGenerated) { + await worker.generateConflationChangeset(state.baseOsmId, state.mergeOptions); + await worker.setChangesetFilters(state.changeTypes, state.entityTypes); + } + } + } } private async findMissingDatasets(worker: Comlink.Remote<T>): Promise<string[]> { @@ -789,6 +858,7 @@ export class OsmixRemote<T extends OsmixWorker = OsmixWorker> { (worker) => worker.fromPbf(transfer({ data: transferableData, options })), { lane: "control", retry: "never" }, ); + this.invalidateConflationsForDataset(osmInfo.id); const replayOptions = { ...options }; this.datasetRestorers.set( osmInfo.id, @@ -859,6 +929,7 @@ export class OsmixRemote<T extends OsmixWorker = OsmixWorker> { ), { lane: "control", retry: "never" }, ); + this.invalidateConflationsForDataset(osmInfo.id); const replayOptions = { ...options }; this.datasetRestorers.set( osmInfo.id, @@ -894,6 +965,7 @@ export class OsmixRemote<T extends OsmixWorker = OsmixWorker> { ), { lane: "control", retry: "never" }, ); + this.invalidateConflationsForDataset(osmInfo.id); const replayOptions = { ...options }; this.datasetRestorers.set( osmInfo.id, @@ -931,6 +1003,7 @@ export class OsmixRemote<T extends OsmixWorker = OsmixWorker> { ), { lane: "control", retry: "never" }, ); + this.invalidateConflationsForDataset(osmInfo.id); const replayOptions = { ...options }; const replayGtfsOptions = { ...gtfsOptions }; this.datasetRestorers.set( @@ -1058,6 +1131,7 @@ export class OsmixRemote<T extends OsmixWorker = OsmixWorker> { ), { lane: "control", retry: "never" }, ); + this.invalidateConflationsForDataset(osmInfo.id); const replayOptions = { ...options }; const replayReadOptions = { ...readOptions }; const replayableSource = @@ -1192,6 +1266,7 @@ export class OsmixRemote<T extends OsmixWorker = OsmixWorker> { if (this.workerCount > 1 && !isShared) { throw Error("Multiple workers require a SharedArrayBuffer-backed OSM dataset"); } + this.invalidateConflationsForDataset(transferables.id); this.markDatasetUnrecoverable(transferables.id); if (isShared) { this.retainedDatasets.set(transferables.id, transferables); @@ -1207,6 +1282,7 @@ export class OsmixRemote<T extends OsmixWorker = OsmixWorker> { */ async delete(osmId: OsmId): Promise<void> { const id = this.getId(osmId); + this.invalidateConflationsForDataset(id); this.unregisterDatasetForRecovery(id); if ( this.activeChangeset && @@ -1233,21 +1309,20 @@ export class OsmixRemote<T extends OsmixWorker = OsmixWorker> { }), { lane: "control", retry: "once" }, ); + // Invalidate sessions using either key: rename removes the source and may + // overwrite a different dataset already registered at the destination. + this.invalidateConflationsForDataset(from); + this.invalidateConflationsForDataset(toId); // Update the id in the transferables const updatedTransferables = { ...transferables, id: toId }; const restorer = this.datasetRestorers.get(from) ?? null; this.unregisterDatasetForRecovery(from); + this.unregisterDatasetForRecovery(toId); this.datasetRestorers.set(toId, restorer); if (hasOnlySharedBackingBuffers(updatedTransferables)) { this.retainedDatasets.set(toId, updatedTransferables); this.retainedLoadDecisions.set(toId, loadDecision); } - if ( - this.activeChangeset && - (this.activeChangeset.baseOsmId === from || this.activeChangeset.patchOsmId === from) - ) { - this.activeChangeset = null; - } // Delete old entries and transfer in with new ID await this.broadcastStateChange("dataset rename", async (worker) => { await worker.delete(from); @@ -1438,6 +1513,140 @@ export class OsmixRemote<T extends OsmixWorker = OsmixWorker> { // Merge & Changesets // --------------------------------------------------------------------------- + /** Discover fuzzy cross-dataset candidates without changing either input dataset. */ + async discoverConflation(baseOsmId: OsmId, patchOsmId: OsmId, options: OsmConflationOptions) { + const baseId = this.getId(baseOsmId); + const patchId = this.getId(patchOsmId); + // Recovery state must not share mutable decisions or option arrays with callers. + const storedOptions = structuredClone(options); + const result = await this.runWithWorker( + (worker) => worker.discoverConflation(baseId, patchId, storedOptions), + { lane: "control", retry: "never" }, + ); + this.activeConflations.set(baseId, { + baseOsmId: baseId, + changeTypes: ["create", "modify", "delete"], + changesetGenerated: false, + decisions: storedOptions.decisions ?? [], + entityTypes: ["node", "way", "relation"], + filter: {}, + mergeOptions: {}, + options: storedOptions, + patchOsmId: patchId, + }); + return result; + } + + /** Return the current, decision-aware candidate summary. */ + getConflationSummary(baseOsmId: OsmId) { + return this.runWithWorker((worker) => worker.getConflationSummary(this.getId(baseOsmId)), { + lane: "control", + retry: "once", + }); + } + + /** Set the filter used by subsequent candidate page requests. */ + async setConflationFilter(baseOsmId: OsmId, filter: OsmConflationCandidateFilter = {}) { + const baseId = this.getId(baseOsmId); + const state = this.getActiveConflation(baseId); + const storedFilter = structuredClone(filter); + await this.runWithWorker((worker) => worker.setConflationFilter(baseId, storedFilter), { + lane: "control", + retry: "never", + }); + state.filter = storedFilter; + } + + /** Retrieve one page of filtered candidates and their current decisions. */ + getConflationPage(baseOsmId: OsmId, page: number, pageSize: number) { + return this.runWithWorker( + (worker) => worker.getConflationPage(this.getId(baseOsmId), page, pageSize), + { lane: "control", retry: "once" }, + ); + } + + /** Record or replace a single candidate decision. */ + async setConflationDecision(baseOsmId: OsmId, decision: OsmConflationDecision) { + const baseId = this.getId(baseOsmId); + const state = this.getActiveConflation(baseId); + const storedDecision = structuredClone(decision); + const result = await this.runWithWorker( + (worker) => worker.setConflationDecision(baseId, storedDecision), + { lane: "control", retry: "never" }, + ); + state.decisions = [ + ...state.decisions.filter((existing) => existing.candidateId !== storedDecision.candidateId), + storedDecision, + ]; + state.changesetGenerated = false; + state.mergeOptions = {}; + return result; + } + + /** Replace all candidate decisions for the active session. */ + async setConflationDecisions(baseOsmId: OsmId, decisions: OsmConflationDecision[]) { + const baseId = this.getId(baseOsmId); + const state = this.getActiveConflation(baseId); + const storedDecisions = structuredClone(decisions); + const result = await this.runWithWorker( + (worker) => worker.setConflationDecisions(baseId, storedDecisions), + { lane: "control", retry: "never" }, + ); + state.decisions = storedDecisions; + state.changesetGenerated = false; + state.mergeOptions = {}; + return result; + } + + /** Apply one action to all eligible candidates matching a filter across every page. */ + async applyConflationBulkDecision(baseOsmId: OsmId, request: OsmConflationBulkDecisionRequest) { + const baseId = this.getId(baseOsmId); + const state = this.getActiveConflation(baseId); + const storedRequest = structuredClone(request); + const result = await this.runWithWorker( + (worker) => worker.applyConflationBulkDecision(baseId, storedRequest), + { lane: "control", retry: "never" }, + ); + state.decisions = result.decisions.map((decision) => ({ ...decision })); + if (result.preview.changedCandidates > 0) { + state.changesetGenerated = false; + state.mergeOptions = {}; + } + return { + decisions: result.decisions.map((decision) => ({ ...decision })), + preview: { ...result.preview }, + summary: { ...result.summary }, + }; + } + + /** + * Generate the cumulative direct, exact, and accepted fuzzy changeset. + * Inputs remain untouched until {@link applyChangesAndReplace} is called. + */ + async generateConflationChangeset(baseOsmId: OsmId, mergeOptions: Partial<OsmMergeOptions> = {}) { + const baseId = this.getId(baseOsmId); + const state = this.getActiveConflation(baseId); + const storedOptions = { ...mergeOptions, conflation: undefined }; + const result = await this.runWithWorker( + (worker) => worker.generateConflationChangeset(baseId, storedOptions), + { lane: "control", retry: "never" }, + ); + state.changesetGenerated = true; + state.mergeOptions = storedOptions; + this.activeChangeset = null; + return result; + } + + /** Cancel a conflation session and discard any generated changeset. */ + async clearConflation(baseOsmId: OsmId) { + const baseId = this.getId(baseOsmId); + await this.runWithWorker((worker) => worker.clearConflation(baseId), { + lane: "control", + retry: "never", + }); + this.activeConflations.delete(baseId); + } + /** * Merge two `Osm` instances in a worker. * Replaces the base instance with the merge result and deletes the patch instance. @@ -1448,6 +1657,8 @@ export class OsmixRemote<T extends OsmixWorker = OsmixWorker> { (worker) => worker.merge(this.getId(baseOsmId), this.getId(patchOsmId), options), { lane: "control", retry: "never" }, ); + this.invalidateConflationsForDataset(baseOsmId); + this.invalidateConflationsForDataset(patchOsmId); this.markDatasetUnrecoverable(osmId); await this.populateDatasetFromControl(osmId); await this.delete(patchOsmId); @@ -1487,6 +1698,7 @@ export class OsmixRemote<T extends OsmixWorker = OsmixWorker> { lane: "control", retry: "never", }); + this.invalidateConflationsForDataset(osmId); this.markDatasetUnrecoverable(osmId); await this.populateDatasetFromControl(osmId); this.activeChangeset = null; @@ -1501,6 +1713,11 @@ export class OsmixRemote<T extends OsmixWorker = OsmixWorker> { this.activeChangeset.changeTypes = [...changeTypes]; this.activeChangeset.entityTypes = [...entityTypes]; } + for (const state of this.activeConflations.values()) { + if (!state.changesetGenerated) continue; + state.changeTypes = [...changeTypes]; + state.entityTypes = [...entityTypes]; + } void this.runWithWorker((worker) => worker.setChangesetFilters(changeTypes, entityTypes), { lane: "control", retry: "never", @@ -1525,6 +1742,7 @@ export class OsmixRemote<T extends OsmixWorker = OsmixWorker> { const pool = this.workerPool; this.workerPool = null; this.activeChangeset = null; + this.activeConflations.clear(); this.datasetRestorers.clear(); this.retainedDatasets.clear(); this.retainedLoadDecisions.clear(); diff --git a/packages/osmix/src/worker.ts b/packages/osmix/src/worker.ts index f0251cf4..aa9dbb28 100644 --- a/packages/osmix/src/worker.ts +++ b/packages/osmix/src/worker.ts @@ -20,13 +20,31 @@ import { applyChangesetToOsm, + buildConflationBulkDecisionResult, generateChangeset, merge, + summarizeConflationCandidates, type OsmChange, type OsmChangeset, type OsmChangeTypes, + type OsmConflationBulkAction, + type OsmConflationBulkDecisionPreview, + type OsmConflationBulkDecisionRequest, + type OsmConflationBulkDecisionResult, + type OsmConflationCandidate, + type OsmConflationCandidateFilter, + type OsmConflationDecision, + type OsmConflationDiscovery, + type OsmConflationOptions, + type OsmConflationSummary, type OsmMergeOptions, + validateConflationDecisions, } from "@osmix/change"; +import { + discoverConflationCandidatesForTrustedMerge, + generateConflationApplicationArtifactsFromTrustedDiscovery, + generateConflationArtifactsFromTrustedDiscovery, +} from "@osmix/change/src/internal/conflation.ts"; import { Osm, type OsmOptions, type OsmTransferables } from "@osmix/core"; import { fromGeoJSON } from "@osmix/geojson"; import { fromGeoParquet, type GeoParquetReadOptions } from "@osmix/geoparquet"; @@ -40,6 +58,8 @@ import { RoutingGraph, type RoutingGraphTransferables, type WaySegment, + defaultHighwayFilter, + defaultPedestrianFilter, } from "@osmix/router"; import { fromShapefile } from "@osmix/shapefile"; import type { Progress, ProgressEvent } from "@osmix/shared/progress"; @@ -49,6 +69,193 @@ import type { LonLat, OsmEntityType, Tile } from "@osmix/types"; // Re-export types from router for backwards compatibility export type { RouteResult, WaySegment }; +/** A conflation candidate together with the user's current review decision, if any. */ +export interface OsmConflationCandidateView extends OsmConflationCandidate { + decision?: OsmConflationDecision; +} + +/** A stable, paginated view of the active conflation candidates. */ +export interface OsmConflationPage { + bulkActions: Record<OsmConflationBulkAction, OsmConflationBulkDecisionPreview>; + candidates: OsmConflationCandidateView[]; + page: number; + pageSize: number; + totalCandidates: number; + totalPages: number; +} + +/** Routing graph measurements captured before and after fuzzy conflation. */ +export interface OsmConflationRoutingGraphStats { + nodes: number; + routableNodes: number; + edges: number; + components: number; +} + +/** Per-mode routing impact of accepted fuzzy conflation candidates. */ +export interface OsmConflationRoutingDelta { + before: OsmConflationRoutingGraphStats; + after: OsmConflationRoutingGraphStats; + delta: OsmConflationRoutingGraphStats; +} + +/** CAR and WALK topology diagnostics for a generated conflation changeset. */ +export interface OsmConflationRoutingDiagnostics { + car: OsmConflationRoutingDelta; + walk: OsmConflationRoutingDelta; +} + +/** Result of generating the cumulative direct, exact, and fuzzy changeset. */ +export interface OsmConflationGenerationResult { + stats: OsmChangeset["stats"]; + routing: OsmConflationRoutingDiagnostics; +} + +interface ConflationSession { + changesetGenerated: boolean; + decisions: Map<string, OsmConflationDecision>; + discovery: OsmConflationDiscovery; + filter: OsmConflationCandidateFilter; + generatedChangeset?: OsmChangeset; + generatedResult?: Osm; + patchOsmId: string; + summary: OsmConflationSummary; +} + +// Comlink normally clones return values, but tests and in-process remotes can expose +// direct references. Clone every nested collection so UI code cannot mutate discovery. +function cloneConflationCandidateView( + candidate: OsmConflationCandidate, + decision: OsmConflationDecision | undefined, +): OsmConflationCandidateView { + return { + ...candidate, + reasons: [...candidate.reasons], + propertyTransfer: { + ...candidate.propertyTransfer, + reasons: [...candidate.propertyTransfer.reasons], + }, + networkAttachment: candidate.networkAttachment + ? { + ...candidate.networkAttachment, + reasons: [...candidate.networkAttachment.reasons], + } + : null, + evidence: { + ...candidate.evidence, + sourceRoutingFamilies: [...candidate.evidence.sourceRoutingFamilies], + targetRoutingFamilies: [...candidate.evidence.targetRoutingFamilies], + tagDiff: candidate.evidence.tagDiff.map((diff) => ({ ...diff })), + patchWayIds: candidate.evidence.patchWayIds ? [...candidate.evidence.patchWayIds] : undefined, + endpointDistancesMeters: candidate.evidence.endpointDistancesMeters + ? [...candidate.evidence.endpointDistancesMeters] + : undefined, + }, + decision: decision ? { ...decision } : undefined, + }; +} + +function conflationCandidateMatches( + candidate: OsmConflationCandidate, + decision: OsmConflationDecision | undefined, + filter: OsmConflationCandidateFilter, +) { + const status = + decision?.action === "accept" + ? "accepted" + : decision?.action === "reject" + ? "rejected" + : candidate.status; + if (filter.entityType != null && candidate.entityType !== filter.entityType) return false; + if (filter.status != null && status !== filter.status) return false; + if (filter.reason != null && !candidate.reasons.includes(filter.reason)) return false; + if (filter.sourceId != null && candidate.sourceId !== filter.sourceId) return false; + if ("targetId" in filter && candidate.targetId !== filter.targetId) return false; + return true; +} + +function routingGraphStats(osm: Osm, filter: HighwayFilter): OsmConflationRoutingGraphStats { + const graph = new RoutingGraph(osm, filter); + const parent = new Int32Array(graph.size); + parent.fill(-1); + let routableNodes = 0; + + for (let nodeIndex = 0; nodeIndex < graph.size; nodeIndex++) { + if (!graph.isRoutable(nodeIndex)) continue; + parent[nodeIndex] = nodeIndex; + routableNodes++; + } + + const find = (nodeIndex: number): number => { + let root = nodeIndex; + while (parent[root] !== root) root = parent[root]!; + let cursor = nodeIndex; + while (parent[cursor] !== cursor) { + const next = parent[cursor]!; + parent[cursor] = root; + cursor = next; + } + return root; + }; + + for (let nodeIndex = 0; nodeIndex < graph.size; nodeIndex++) { + if (parent[nodeIndex] === -1) continue; + for (const edge of graph.getEdges(nodeIndex)) { + if (parent[edge.targetNodeIndex] === -1) continue; + const left = find(nodeIndex); + const right = find(edge.targetNodeIndex); + if (left !== right) parent[right] = left; + } + } + + const roots = new Set<number>(); + for (let nodeIndex = 0; nodeIndex < graph.size; nodeIndex++) { + if (parent[nodeIndex] !== -1) roots.add(find(nodeIndex)); + } + + return { + nodes: graph.size, + routableNodes, + edges: graph.edges, + components: roots.size, + }; +} + +function routingDelta( + before: OsmConflationRoutingGraphStats, + after: OsmConflationRoutingGraphStats, +): OsmConflationRoutingDelta { + return { + before, + after, + delta: { + nodes: after.nodes - before.nodes, + routableNodes: after.routableNodes - before.routableNodes, + edges: after.edges - before.edges, + components: after.components - before.components, + }, + }; +} + +function routingDiagnostics(baseline: Osm, conflated: Osm): OsmConflationRoutingDiagnostics { + const walkFilter: HighwayFilter = (tags) => + defaultHighwayFilter(tags) || defaultPedestrianFilter(tags); + return { + car: routingDelta( + routingGraphStats(baseline, defaultHighwayFilter), + routingGraphStats(conflated, defaultHighwayFilter), + ), + walk: routingDelta( + routingGraphStats(baseline, walkFilter), + routingGraphStats(conflated, walkFilter), + ), + }; +} + +function carTopologyChanged(delta: OsmConflationRoutingDelta) { + return delta.delta.routableNodes !== 0 || delta.delta.edges !== 0 || delta.delta.components !== 0; +} + import { fromPbf, getOsmLoadDecision as getStoredOsmLoadDecision, @@ -78,6 +285,7 @@ export class OsmixWorker extends EventTarget { private vtEncoders = new Map<string, OsmixVtEncoder>(); private graphs = new Map<string, RoutingGraph>(); private changesets = new Map<string, OsmChangeset>(); + private conflations = new Map<string, ConflationSession>(); private changeTypes: OsmChangeTypes[] = ["create", "modify", "delete"]; private entityTypes: OsmEntityType[] = ["node", "way", "relation"]; private filteredChanges = new Map<string, OsmChange[]>(); @@ -287,6 +495,7 @@ export class OsmixWorker extends EventTarget { * rebuild it. */ protected set(id: string, osm: Osm) { + this.invalidateConflationsForDataset(id); this.osm.set(id, osm); this.loadDecisions.delete(id); this.vtEncoders.set(id, new OsmixVtEncoder(osm)); @@ -306,10 +515,24 @@ export class OsmixWorker extends EventTarget { * Remove an Osm instance from this worker, freeing its memory. */ delete(id: string) { + this.invalidateConflationsForDataset(id); this.osm.delete(id); this.loadDecisions.delete(id); this.vtEncoders.delete(id); this.graphs.delete(id); + this.changesets.delete(id); + this.filteredChanges.delete(id); + } + + private invalidateConflationsForDataset(osmId: string) { + for (const [baseOsmId, session] of this.conflations) { + if (baseOsmId !== osmId && session.patchOsmId !== osmId) continue; + this.conflations.delete(baseOsmId); + if (session.changesetGenerated) { + this.changesets.delete(baseOsmId); + this.filteredChanges.delete(baseOsmId); + } + } } // --------------------------------------------------------------------------- @@ -486,6 +709,248 @@ export class OsmixWorker extends EventTarget { return this.get(osmId).relations.search(key, val); } + /** Discover non-exact, cross-dataset conflation candidates without mutating either input. */ + discoverConflation( + baseOsmId: string, + patchOsmId: string, + options: OsmConflationOptions, + ): OsmConflationSummary { + const discovery = discoverConflationCandidatesForTrustedMerge( + this.get(baseOsmId), + this.get(patchOsmId), + options, + ); + const initialDecisions = options.decisions === undefined ? [] : options.decisions; + validateConflationDecisions(discovery.candidates, initialDecisions); + const decisions = new Map<string, OsmConflationDecision>(); + for (const decision of initialDecisions) { + decisions.set(decision.candidateId, { ...decision }); + } + const previous = this.conflations.get(baseOsmId); + if (previous?.changesetGenerated) { + this.changesets.delete(baseOsmId); + this.filteredChanges.delete(baseOsmId); + } + const summary = + decisions.size === 0 + ? discovery.summary + : summarizeConflationCandidates(discovery.candidates, [...decisions.values()]); + this.conflations.set(baseOsmId, { + changesetGenerated: false, + decisions, + discovery, + filter: {}, + patchOsmId, + summary, + }); + return { ...summary }; + } + + /** Return the decision-aware summary for an active conflation session. */ + getConflationSummary(baseOsmId: string): OsmConflationSummary { + return { ...this.getConflationSession(baseOsmId).summary }; + } + + /** Replace the active candidate filter used by {@link getConflationPage}. */ + setConflationFilter(baseOsmId: string, filter: OsmConflationCandidateFilter = {}) { + this.getConflationSession(baseOsmId).filter = { ...filter }; + } + + /** Retrieve a stable page of candidates together with their current review decisions. */ + getConflationPage(baseOsmId: string, page: number, pageSize: number): OsmConflationPage { + if (!Number.isInteger(page) || page < 0) throw Error("page must be a non-negative integer"); + if (!Number.isInteger(pageSize) || pageSize <= 0) { + throw Error("pageSize must be a positive integer"); + } + const session = this.getConflationSession(baseOsmId); + const candidates = session.discovery.candidates.filter((candidate) => + conflationCandidateMatches(candidate, session.decisions.get(candidate.id), session.filter), + ); + const start = page * pageSize; + const decisions = [...session.decisions.values()]; + const bulkActions = Object.fromEntries( + (["transfer-properties", "attach-network", "reject"] as const).map((action) => [ + action, + buildConflationBulkDecisionResult(session.discovery.candidates, decisions, { + action, + filter: session.filter, + }).preview, + ]), + ) as Record<OsmConflationBulkAction, OsmConflationBulkDecisionPreview>; + return { + bulkActions, + candidates: candidates + .slice(start, start + pageSize) + .map((candidate) => + cloneConflationCandidateView(candidate, session.decisions.get(candidate.id)), + ), + page, + pageSize, + totalCandidates: candidates.length, + totalPages: Math.ceil(candidates.length / pageSize), + }; + } + + /** Record or replace one candidate decision and invalidate any generated changeset. */ + setConflationDecision(baseOsmId: string, decision: OsmConflationDecision) { + const session = this.getConflationSession(baseOsmId); + // Validate before touching session state so malformed RPC input is atomic. + validateConflationDecisions(session.discovery.candidates, [decision]); + this.invalidateGeneratedConflationChangeset(baseOsmId, session); + session.decisions.set(decision.candidateId, { ...decision }); + session.summary = summarizeConflationCandidates(session.discovery.candidates, [ + ...session.decisions.values(), + ]); + return { ...session.summary }; + } + + /** Replace every candidate decision and invalidate any generated changeset. */ + setConflationDecisions(baseOsmId: string, decisions: OsmConflationDecision[]) { + const session = this.getConflationSession(baseOsmId); + // Build and validate the replacement set before discarding reviewed output. + validateConflationDecisions(session.discovery.candidates, decisions); + const next = new Map<string, OsmConflationDecision>(); + for (const decision of decisions) { + next.set(decision.candidateId, { ...decision }); + } + this.invalidateGeneratedConflationChangeset(baseOsmId, session); + session.decisions = next; + session.summary = + next.size === 0 + ? session.discovery.summary + : summarizeConflationCandidates(session.discovery.candidates, [...next.values()]); + return { ...session.summary }; + } + + /** Apply one action to every eligible candidate matching the supplied filter. */ + applyConflationBulkDecision( + baseOsmId: string, + request: OsmConflationBulkDecisionRequest, + ): OsmConflationBulkDecisionResult { + const session = this.getConflationSession(baseOsmId); + const result = buildConflationBulkDecisionResult( + session.discovery.candidates, + [...session.decisions.values()], + request, + ); + if (result.preview.changedCandidates > 0) { + this.invalidateGeneratedConflationChangeset(baseOsmId, session); + session.decisions = new Map( + result.decisions.map((decision) => [decision.candidateId, { ...decision }]), + ); + } + session.summary = { ...result.summary }; + return { + decisions: result.decisions.map((decision) => ({ ...decision })), + preview: { ...result.preview }, + summary: { ...result.summary }, + }; + } + + /** + * Generate one cumulative direct, exact, and fuzzy changeset from the untouched inputs. + * Intersections remain a subsequent merge stage so routing diagnostics isolate conflation. + */ + generateConflationChangeset( + baseOsmId: string, + mergeOptions: Partial<OsmMergeOptions> = {}, + ): OsmConflationGenerationResult { + if (mergeOptions.createIntersections) { + throw Error( + "Generate and apply conflation before creating intersections; createIntersections must be false", + ); + } + const session = this.getConflationSession(baseOsmId); + const base = this.get(baseOsmId); + const patch = this.get(session.patchOsmId); + const decisions = [...session.decisions.values()]; + const conflation = { + ...session.discovery.options, + decisions, + }; + const options: Partial<OsmMergeOptions> = { + ...mergeOptions, + createIntersections: false, + conflation, + }; + const artifacts = generateConflationArtifactsFromTrustedDiscovery( + base, + patch, + options, + decisions, + session.discovery, + this.onProgress, + ); + const diagnostics = routingDiagnostics(artifacts.ordinaryBaseline, artifacts.result); + // The full result may contain manually reviewed motor-network changes. Project + // automatic attachments alone so the automatic WALK-only CAR invariant is exact. + let hasAutomaticNetworkAttachment = false; + const automaticAttachmentDecisions: OsmConflationDecision[] = []; + for (const candidate of session.discovery.candidates) { + const decision = session.decisions.get(candidate.id); + const attachNetwork = + candidate.networkAttachment?.status === "automatic" && + decision?.action !== "reject" && + decision?.attachNetwork !== false; + hasAutomaticNetworkAttachment ||= attachNetwork; + if (attachNetwork) { + automaticAttachmentDecisions.push({ + candidateId: candidate.id, + action: "accept", + transferProperties: false, + attachNetwork: true, + }); + } else if ( + candidate.propertyTransfer.status === "automatic" || + candidate.networkAttachment?.status === "automatic" + ) { + // A missing decision enables automatic actions. Explicitly reject only + // automatic candidates that must be absent from this attachment-only + // projection; review, blocked, and unmatched rows already apply nothing. + automaticAttachmentDecisions.push({ + candidateId: candidate.id, + action: "reject", + }); + } + } + if (hasAutomaticNetworkAttachment) { + const automaticAttachment = generateConflationApplicationArtifactsFromTrustedDiscovery( + artifacts.ordinaryBaseline, + patch, + session.discovery, + base, + automaticAttachmentDecisions, + ); + const automaticCarDelta = routingDelta( + diagnostics.car.before, + routingGraphStats(automaticAttachment.result, defaultHighwayFilter), + ); + if (carTopologyChanged(automaticCarDelta)) { + throw Error( + "Automatic walk-only conflation changed the CAR graph; review the candidate instead", + ); + } + } + + this.changesets.set(baseOsmId, artifacts.changeset); + // Candidate review does not imply changeset review. Defer the large filtered + // change list until a caller actually opens a changeset page; automatic runs + // apply the already validated materialized result without building it. + this.filteredChanges.delete(baseOsmId); + session.changesetGenerated = true; + session.generatedChangeset = artifacts.changeset; + session.generatedResult = artifacts.result; + return { stats: artifacts.changeset.stats, routing: diagnostics }; + } + + /** Clear an active conflation session and its generated changeset, if present. */ + clearConflation(baseOsmId: string) { + const session = this.conflations.get(baseOsmId); + if (!session) return; + this.invalidateGeneratedConflationChangeset(baseOsmId, session); + this.conflations.delete(baseOsmId); + } + /** * Perform a full merge of two Osm indexes inside of a worker. Both Osm indexes must be loaded already. * Replaces the base Osm and deletes the patch Osm. @@ -516,7 +981,7 @@ export class OsmixWorker extends EventTarget { this.onProgress, ); this.changesets.set(baseOsmId, changeset); - this.sortChangeset(baseOsmId, changeset); + this.filteredChanges.delete(baseOsmId); return changeset.stats; } @@ -545,6 +1010,7 @@ export class OsmixWorker extends EventTarget { getChangesetPage(osmId: string, page: number, pageSize: number) { const changeset = this.changesets.get(osmId); if (!changeset) throw Error("No active changeset"); + if (!this.filteredChanges.has(osmId)) this.sortChangeset(osmId, changeset); const filteredChanges = this.filteredChanges.get(osmId); const changes = filteredChanges?.slice(page * pageSize, (page + 1) * pageSize); return { @@ -560,13 +1026,35 @@ export class OsmixWorker extends EventTarget { applyChangesAndReplace(osmId: string) { const changeset = this.changesets.get(osmId); if (!changeset) throw Error("No active changeset"); - const newOsm = applyChangesetToOsm(changeset); + const session = this.conflations.get(osmId); + const newOsm = + session?.changesetGenerated && session.generatedChangeset === changeset + ? session.generatedResult + : applyChangesetToOsm(changeset); + if (!newOsm) throw Error("Generated conflation result is missing"); this.set(osmId, newOsm); this.changesets.delete(osmId); this.filteredChanges.delete(osmId); return newOsm.id; } + private getConflationSession(baseOsmId: string) { + const session = this.conflations.get(baseOsmId); + if (!session) throw Error("No active conflation session"); + return session; + } + + private invalidateGeneratedConflationChangeset(baseOsmId: string, session: ConflationSession) { + if (!session.changesetGenerated) return; + // A reviewed changeset is a snapshot of its decisions. Never allow a later + // decision edit to apply that stale snapshot. + this.changesets.delete(baseOsmId); + this.filteredChanges.delete(baseOsmId); + session.changesetGenerated = false; + session.generatedChangeset = undefined; + session.generatedResult = undefined; + } + /** * Filter and sort changeset entries by the current entity type and change type filters. * Updates the filteredChanges cache for efficient pagination. diff --git a/packages/osmix/test/conflation-yakima.test.ts b/packages/osmix/test/conflation-yakima.test.ts new file mode 100644 index 00000000..1175ac56 --- /dev/null +++ b/packages/osmix/test/conflation-yakima.test.ts @@ -0,0 +1,211 @@ +import { access } from "node:fs/promises"; + +import type { Osm } from "@osmix/core"; +import { getFixtureFileReadStream, getFixturePath } from "@osmix/test-utils/fixtures"; +import { describe, expect, it } from "vitest"; + +import { + discoverConflationCandidates, + fromPbf, + type OsmConflationCandidate, + type OsmConflationDiscovery, +} from "../src/index.ts"; + +const BASE_FIXTURE = "yakima-full.osm.pbf"; +const PATCH_FIXTURE = "yakima.osw.pbf"; +const fixturesExist = await Promise.all( + [BASE_FIXTURE, PATCH_FIXTURE].map((fixture) => + access(getFixturePath(fixture)) + .then(() => true) + .catch(() => false), + ), +).then((results) => results.every(Boolean)); + +function getCandidate(discovery: OsmConflationDiscovery, id: string) { + const candidate = discovery.candidates.find((item) => item.id === id); + if (!candidate) throw new Error(`Missing Yakima conflation witness ${id}`); + return candidate; +} + +function getTargetId(candidate: OsmConflationCandidate) { + if (candidate.targetId == null) throw new Error(`${candidate.id} does not have a target`); + return candidate.targetId; +} + +function getIncidentWays(osm: Osm, nodeId: number) { + return [...osm.ways].filter((way) => way.refs.includes(nodeId)); +} + +function expectNonExact(candidate: OsmConflationCandidate, base: Osm, patch: Osm) { + expect(candidate.targetId).not.toBeNull(); + const source = patch.nodes.getById(candidate.sourceId); + const target = candidate.targetId == null ? null : base.nodes.getById(candidate.targetId); + expect(source).not.toBeNull(); + expect(target).not.toBeNull(); + expect([source?.lon, source?.lat]).not.toEqual([target?.lon, target?.lat]); + expect(candidate.evidence.distanceMeters).toBeGreaterThan(0); + expect(candidate.evidence.distanceMeters).toBeLessThanOrEqual(1); +} + +function expectSchoolBoundaryBlocked( + discovery: OsmConflationDiscovery, + base: Osm, + patch: Osm, + witness: { + candidateId: string; + patchWayId: number; + schoolName: string; + targetWayId: number; + }, +) { + const candidate = getCandidate(discovery, witness.candidateId); + expectNonExact(candidate, base, patch); + expect(candidate).toMatchObject({ + entityType: "node", + status: "blocked", + networkAttachment: { status: "blocked" }, + }); + expect(candidate.reasons).toContain("non-routing-target"); + + const sourceWay = getIncidentWays(patch, candidate.sourceId).find( + (way) => way.id === witness.patchWayId, + ); + const targetWay = getIncidentWays(base, getTargetId(candidate)).find( + (way) => way.id === witness.targetWayId, + ); + expect(sourceWay?.tags).toMatchObject({ highway: "footway" }); + expect(targetWay?.tags).toMatchObject({ amenity: "school", name: witness.schoolName }); + expect(targetWay?.tags?.["highway"]).toBeUndefined(); + expect(targetWay?.refs[0]).toBe(targetWay?.refs.at(-1)); +} + +describe("Yakima fuzzy conflation", () => { + it.runIf(fixturesExist)( + "classifies real non-exact OSW candidates conservatively", + async () => { + const [base, patch] = await Promise.all([ + fromPbf(getFixtureFileReadStream(BASE_FIXTURE), { id: BASE_FIXTURE }), + fromPbf(getFixtureFileReadStream(PATCH_FIXTURE), { id: PATCH_FIXTURE }), + ]); + const discovery = discoverConflationCandidates(base, patch, { + propertyKeys: ["barrier", "crossing", "kerb", "tactile_paving"], + attachNetwork: true, + }); + + expect(discovery.options).toEqual({ + propertyKeys: ["barrier", "crossing", "kerb", "tactile_paving"], + attachNetwork: true, + maxDistanceMeters: 1, + automatic: "high-confidence", + }); + expect(discovery.summary).toEqual({ + total: 11_689, + accepted: 0, + automatic: 145, + review: 212, + blocked: 88, + unmatched: 11_244, + rejected: 0, + }); + + const matched = discovery.candidates.filter((candidate) => candidate.targetId != null); + const targetCountBySource = new Map<number, number>(); + for (const candidate of matched) { + targetCountBySource.set( + candidate.sourceId, + (targetCountBySource.get(candidate.sourceId) ?? 0) + 1, + ); + } + expect(matched).toHaveLength(445); + expect(targetCountBySource.size).toBe(399); + expect([...targetCountBySource.values()].filter((count) => count === 1)).toHaveLength(356); + expect([...targetCountBySource.values()].filter((count) => count > 1)).toHaveLength(43); + expect(matched.every((candidate) => candidate.evidence.distanceMeters > 0)).toBe(true); + + const accessibleCrossing = getCandidate(discovery, "node:2220318->11643002707"); + expectNonExact(accessibleCrossing, base, patch); + expect(accessibleCrossing).toMatchObject({ + status: "review", + reasons: ["node-context-conflict"], + propertyTransfer: { status: "automatic", reasons: [] }, + networkAttachment: { status: "review", reasons: ["node-context-conflict"] }, + evidence: { + distanceMeters: 0.40797, + sourceRoutingFamilies: ["pedestrian"], + targetRoutingFamilies: ["pedestrian"], + tagDiff: [ + { + key: "tactile_paving", + patchValue: "yes", + protected: false, + routing: false, + }, + ], + }, + }); + expect( + getIncidentWays(patch, accessibleCrossing.sourceId).find((way) => way.id === 850268)?.tags, + ).toMatchObject({ footway: "crossing", highway: "footway" }); + expect( + getIncidentWays(base, getTargetId(accessibleCrossing)).find( + (way) => way.id === 1_252_605_649, + )?.tags, + ).toMatchObject({ footway: "crossing", highway: "footway" }); + + const kerbConflict = getCandidate(discovery, "node:2475012->11643237283"); + expectNonExact(kerbConflict, base, patch); + expect(kerbConflict).toMatchObject({ + status: "blocked", + networkAttachment: { + status: "blocked", + reasons: expect.arrayContaining(["routing-family-conflict"]), + }, + }); + expect(patch.nodes.getById(kerbConflict.sourceId)?.tags).toMatchObject({ + barrier: "kerb", + }); + expect(patch.nodes.getById(kerbConflict.sourceId)?.tags?.["kerb"]).toBeUndefined(); + expect(base.nodes.getById(getTargetId(kerbConflict))?.tags).toMatchObject({ + barrier: "kerb", + kerb: "raised", + }); + + const sidewalk = getCandidate(discovery, "node:2213758->8075647920"); + expectNonExact(sidewalk, base, patch); + expect(sidewalk).toMatchObject({ + status: "automatic", + propertyTransfer: { + status: "blocked", + reasons: ["no-transferable-properties"], + }, + networkAttachment: { status: "automatic", reasons: [] }, + }); + expect( + getIncidentWays(patch, sidewalk.sourceId).find((way) => way.id === 848575)?.tags, + ).toMatchObject({ footway: "sidewalk", highway: "footway" }); + expect( + getIncidentWays(base, getTargetId(sidewalk)).find((way) => way.id === 866_417_077)?.tags, + ).toMatchObject({ footway: "sidewalk", highway: "footway" }); + + expectSchoolBoundaryBlocked(discovery, base, patch, { + candidateId: "node:2193697->7201121727", + patchWayId: 840053, + targetWayId: 771_378_493, + schoolName: "West Valley High School", + }); + expectSchoolBoundaryBlocked(discovery, base, patch, { + candidateId: "node:9412890->9508231896", + patchWayId: 4_256_164, + targetWayId: 1_031_701_052, + schoolName: "White Swan High School", + }); + expectSchoolBoundaryBlocked(discovery, base, patch, { + candidateId: "node:9885001->2172323056", + patchWayId: 4_490_365, + targetWayId: 207_104_786, + schoolName: "Terrace Heights Elementary School", + }); + }, + 60_000, + ); +}); diff --git a/packages/osmix/test/eastern-washington-merge.test.ts b/packages/osmix/test/eastern-washington-merge.test.ts new file mode 100644 index 00000000..828b12a9 --- /dev/null +++ b/packages/osmix/test/eastern-washington-merge.test.ts @@ -0,0 +1,109 @@ +import { execFile } from "node:child_process"; +import { createReadStream, createWriteStream } from "node:fs"; +import { mkdtemp, rm } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { Readable, Writable } from "node:stream"; +import { promisify } from "node:util"; + +import type { Osm } from "@osmix/core"; +import { getFixtureFileReadStream } from "@osmix/test-utils/fixtures"; +import { describe, expect, it } from "vitest"; + +import { fromPbf, merge, toPbfStream } from "../src/index.ts"; + +const execFileAsync = promisify(execFile); +const RUN_INTEGRATION = process.env["OSMIX_EASTERN_WASHINGTON_INTEGRATION"] === "1"; +const BASE_FIXTURE = "osmix-e_wa_osm.pbf"; +const PATCH_FIXTURE = "east_washington_sidewalk_proviso_1.pbf"; +const COLLAPSE_WITNESSES = [ + 853_782, 855_126, 898_741, 1_030_054, 1_869_808, 1_870_520, 1_870_540, 1_870_637, 4_036_007, + 4_079_228, 4_471_764, +] as const; + +function sizes(osm: Osm) { + return { + nodes: osm.nodes.size, + relations: osm.relations.size, + ways: osm.ways.size, + }; +} + +function expectValidWayTopology(osm: Osm) { + const danglingRefs: string[] = []; + const degenerateHighways: number[] = []; + for (const way of osm.ways) { + if (way.tags?.["highway"] != null && new Set(way.refs).size < 2) { + degenerateHighways.push(way.id); + } + for (const ref of way.refs) { + if (!osm.nodes.ids.has(ref)) danglingRefs.push(`${way.id}->${ref}`); + } + } + expect(degenerateHighways).toEqual([]); + expect(danglingRefs).toEqual([]); +} + +describe("Eastern Washington full merge", () => { + it.runIf(RUN_INTEGRATION)( + "preserves short imported footways through export and reload", + async () => { + let base: Osm | null = await fromPbf( + getFixtureFileReadStream(BASE_FIXTURE), + { id: BASE_FIXTURE }, + () => {}, + ); + let patch: Osm | null = await fromPbf( + getFixtureFileReadStream(PATCH_FIXTURE), + { id: PATCH_FIXTURE }, + () => {}, + ); + expect(sizes(base)).toEqual({ nodes: 2_819_575, relations: 0, ways: 244_822 }); + expect(sizes(patch)).toEqual({ nodes: 1_107_476, relations: 0, ways: 368_648 }); + + const progress: string[] = []; + const merged = await merge( + base, + patch, + { + createIntersections: true, + deduplicateNodes: true, + deduplicateWays: true, + directMerge: true, + }, + (event) => progress.push(event.detail.msg), + ); + base = null; + patch = null; + + expectValidWayTopology(merged); + for (const wayId of COLLAPSE_WITNESSES) { + expect(new Set(merged.ways.getById(wayId)?.refs).size, `way ${wayId}`).toBeGreaterThan(1); + } + expect(progress).toContain("Intersection creation progress: 368,648 of 368,648 ways checked"); + + const temporaryDirectory = await mkdtemp(join(tmpdir(), "osmix-eastern-washington-")); + const outputPath = join(temporaryDirectory, "merged.osm.pbf"); + try { + await toPbfStream(merged).pipeTo( + Writable.toWeb(createWriteStream(outputPath)) as WritableStream<Uint8Array>, + ); + await execFileAsync("osmium", ["check-refs", outputPath]); + + const reloaded = await fromPbf( + Readable.toWeb(createReadStream(outputPath)) as ReadableStream<Uint8Array>, + { id: "eastern-washington-round-trip" }, + () => {}, + ); + expect(sizes(reloaded)).toEqual(sizes(merged)); + expectValidWayTopology(reloaded); + for (const wayId of COLLAPSE_WITNESSES) { + expect(reloaded.ways.getById(wayId)?.refs).toEqual(merged.ways.getById(wayId)?.refs); + } + } finally { + await rm(temporaryDirectory, { force: true, recursive: true }); + } + }, + 20 * 60_000, + ); +}); diff --git a/packages/osmix/test/fixtures/routing-cases.ts b/packages/osmix/test/fixtures/routing-cases.ts new file mode 100644 index 00000000..f85d42d4 --- /dev/null +++ b/packages/osmix/test/fixtures/routing-cases.ts @@ -0,0 +1,433 @@ +import type { LonLat, RouteOptions } from "../../src/index.ts"; + +export type RoutingTestMode = "car" | "walk"; + +export type RoutingTestEndpoint = + | { nodeId: number } + | { + coordinates: LonLat; + maxSnapDistanceMeters: number; + }; + +export interface RoutingPolicyLimitation { + kind: "access" | "turn-restriction"; + reason: string; + r5Expectation: string; + witness: { + type: "relation" | "way"; + id: number; + tags: Readonly<Record<string, string>>; + }; +} + +export interface RoutingTestExpectation { + reachable?: boolean; + distanceMeters?: { min: number; max: number }; + timeSeconds?: { min: number; max: number }; + requiredWayIds?: readonly number[]; + forbiddenWayIds?: readonly number[]; +} + +export interface RoutingTestCase { + id: string; + description: string; + mode: RoutingTestMode; + metric: RouteOptions["metric"]; + from: RoutingTestEndpoint; + to: RoutingTestEndpoint; + expect: RoutingTestExpectation; + /** Optional test-only policy refinement; this is not a public Osmix routing profile. */ + graphPolicy?: "access-aware"; + policyLimitation?: RoutingPolicyLimitation; +} + +/** + * Routes whose OSM IDs and broad measurements are stable in the checked-in Monaco fixture. + * Exact coordinate arrays are deliberately not golden data: they are too sensitive to harmless + * encoding and graph-storage changes. + */ +export const MONACO_ROUTING_CASES = [ + { + id: "monaco-short-drive", + description: "Short drive through central Monaco", + mode: "car", + metric: "distance", + from: { coordinates: [7.4229093, 43.7371175], maxSnapDistanceMeters: 100 }, + to: { coordinates: [7.4259193, 43.7377731], maxSnapDistanceMeters: 100 }, + expect: { + reachable: true, + distanceMeters: { min: 240, max: 270 }, + timeSeconds: { min: 15, max: 25 }, + requiredWayIds: [157719644, 254596486, 166624050], + }, + }, + { + id: "monaco-short-walk", + description: "Walk between the same central Monaco points", + mode: "walk", + metric: "distance", + from: { coordinates: [7.4229093, 43.7371175], maxSnapDistanceMeters: 100 }, + to: { coordinates: [7.4259193, 43.7377731], maxSnapDistanceMeters: 100 }, + expect: { + reachable: true, + distanceMeters: { min: 200, max: 500 }, + }, + }, + { + id: "monaco-cross-town-drive", + description: "West-to-east drive across Monaco's largest connected road component", + mode: "car", + metric: "distance", + from: { nodeId: 4329343083 }, + to: { nodeId: 7779445520 }, + expect: { + reachable: true, + distanceMeters: { min: 5_650, max: 5_900 }, + timeSeconds: { min: 360, max: 390 }, + requiredWayIds: [239592573, 952419570, 161627743], + }, + }, + { + id: "monaco-streets-and-steps-walk", + description: "Walk through ordinary streets, pedestrian ways, footways, and steps", + mode: "walk", + metric: "distance", + from: { nodeId: 25182927 }, + to: { nodeId: 25181969 }, + expect: { + reachable: true, + distanceMeters: { min: 440, max: 480 }, + requiredWayIds: [4227157, 1082312632, 4227155, 167014909, 165636031], + }, + }, + { + id: "monaco-oneway-forward", + description: "Drive with the tagged direction of Avenue des Papalins", + mode: "car", + metric: "distance", + from: { nodeId: 25177418 }, + to: { nodeId: 25177397 }, + expect: { + reachable: true, + distanceMeters: { min: 30, max: 40 }, + timeSeconds: { min: 3, max: 5 }, + requiredWayIds: [4224972], + }, + }, + { + id: "monaco-oneway-reverse", + description: "Reverse drive detours around Avenue des Papalins", + mode: "car", + metric: "distance", + from: { nodeId: 25177397 }, + to: { nodeId: 25177418 }, + expect: { + reachable: true, + distanceMeters: { min: 120, max: 130 }, + timeSeconds: { min: 10, max: 13 }, + requiredWayIds: [4229273, 503462459, 804900035, 4229900, 503462460, 4229274], + forbiddenWayIds: [4224972], + }, + }, + { + id: "monaco-reverse-oneway-legal", + description: "Drive against stored node order where oneway=-1 permits that direction", + mode: "car", + metric: "distance", + from: { nodeId: 4437836938 }, + to: { nodeId: 254470730 }, + expect: { + reachable: true, + distanceMeters: { min: 10, max: 12 }, + requiredWayIds: [158215200], + }, + }, + { + id: "monaco-reverse-oneway-detour", + description: "Drive around the loop rather than forward against a oneway=-1 tag", + mode: "car", + metric: "distance", + from: { nodeId: 254470730 }, + to: { nodeId: 4437836938 }, + expect: { + reachable: true, + distanceMeters: { min: 40, max: 43 }, + requiredWayIds: [158215200], + }, + }, + { + id: "monaco-implicit-roundabout-oneway", + description: "Drive in the legal direction around the Avenue Albert II roundabout", + mode: "car", + metric: "distance", + from: { nodeId: 25204713 }, + to: { nodeId: 25238111 }, + expect: { + reachable: true, + distanceMeters: { min: 60, max: 75 }, + timeSeconds: { min: 2, max: 6 }, + requiredWayIds: [4229900, 503462460, 503462476, 503462459, 804900035], + }, + }, + { + id: "monaco-motor-vehicle-access", + description: "Car access witness on motor_vehicle=no Impasse du Stade", + mode: "car", + metric: "distance", + from: { nodeId: 254470916 }, + to: { nodeId: 1704462513 }, + expect: {}, + policyLimitation: { + kind: "access", + reason: "Osmix's default vehicle filter currently checks highway class but not access tags.", + r5Expectation: "A normal car route must not traverse way 158215187 because motor_vehicle=no.", + witness: { + type: "way", + id: 158215187, + tags: { highway: "service", motor_vehicle: "no" }, + }, + }, + }, + { + id: "monaco-no-left-turn-restriction", + description: "Prohibited turn witness for restriction relation 4261963", + mode: "car", + metric: "distance", + from: { nodeId: 1704462546 }, + to: { nodeId: 1778433989 }, + expect: {}, + policyLimitation: { + kind: "turn-restriction", + reason: "Osmix's routing graph does not currently interpret restriction relations.", + r5Expectation: + "Do not transition directly from way 176527122 to way 166399477 through node 25177185.", + witness: { + type: "relation", + id: 4261963, + tags: { restriction: "no_left_turn", type: "restriction" }, + }, + }, + }, + { + id: "monaco-tunnel-layer-regression", + description: "Driving route that must not shortcut between nearby road levels", + mode: "car", + metric: "distance", + from: { nodeId: 1866510534 }, + to: { nodeId: 937988247 }, + expect: { + reachable: true, + distanceMeters: { min: 1_000, max: 1_100 }, + timeSeconds: { min: 55, max: 70 }, + }, + }, + { + id: "monaco-reachability-regression", + description: "Driving route that became disconnected after unsafe node deduplication", + mode: "car", + metric: "distance", + from: { nodeId: 1875118274 }, + to: { nodeId: 12281555152 }, + expect: { + reachable: true, + distanceMeters: { min: 150, max: 180 }, + timeSeconds: { min: 8, max: 15 }, + }, + }, + { + id: "outside-monaco", + description: "A point outside the extract cannot snap to its routing graph", + mode: "car", + metric: "distance", + from: { coordinates: [0, 0], maxSnapDistanceMeters: 50 }, + to: { coordinates: [0.001, 0.001], maxSnapDistanceMeters: 50 }, + expect: { reachable: false }, + }, +] as const satisfies readonly RoutingTestCase[]; + +export const SYNTHETIC_ROUTING_CASES = [ + { + id: "synthetic-car-extension", + description: "Cars use the residential base road and its merged extension", + mode: "car", + metric: "distance", + from: { nodeId: 1 }, + to: { nodeId: 4 }, + expect: { + reachable: true, + distanceMeters: { min: 540, max: 570 }, + requiredWayIds: [100, 101], + forbiddenWayIds: [102], + }, + }, + { + id: "synthetic-walk-shortcut", + description: "Walkers can use the foot-only direct connection", + mode: "walk", + metric: "distance", + from: { nodeId: 1 }, + to: { nodeId: 4 }, + expect: { + reachable: true, + distanceMeters: { min: 330, max: 340 }, + requiredWayIds: [102], + forbiddenWayIds: [100, 101], + }, + }, + { + id: "synthetic-oneway-forward", + description: "Driving follows a one-way road in its tagged direction", + mode: "car", + metric: "distance", + from: { nodeId: 10 }, + to: { nodeId: 12 }, + expect: { + reachable: true, + distanceMeters: { min: 210, max: 230 }, + requiredWayIds: [110], + }, + }, + { + id: "synthetic-oneway-reverse", + description: "Driving cannot reverse along a one-way road", + mode: "car", + metric: "distance", + from: { nodeId: 12 }, + to: { nodeId: 10 }, + expect: { reachable: false }, + }, + { + id: "synthetic-reverse-oneway-forward", + description: "Driving cannot follow the stored order of a reverse one-way road", + mode: "car", + metric: "distance", + from: { nodeId: 13 }, + to: { nodeId: 15 }, + expect: { reachable: false }, + }, + { + id: "synthetic-reverse-oneway-reverse", + description: "Driving follows a reverse one-way road against its stored node order", + mode: "car", + metric: "distance", + from: { nodeId: 15 }, + to: { nodeId: 13 }, + expect: { + reachable: true, + distanceMeters: { min: 210, max: 230 }, + requiredWayIds: [111], + }, + }, + { + id: "synthetic-grade-separation", + description: "A tunnel remains disconnected from the nearby surface road", + mode: "car", + metric: "distance", + from: { nodeId: 21 }, + to: { nodeId: 23 }, + expect: { reachable: false }, + }, + { + id: "synthetic-same-grade-crossing", + description: "A same-grade crossing creates a routable connection", + mode: "car", + metric: "distance", + from: { nodeId: 40 }, + to: { nodeId: 51 }, + expect: { + reachable: true, + distanceMeters: { min: 325, max: 345 }, + requiredWayIds: [130, 131], + }, + }, + { + id: "synthetic-reverse-multiple-intersections", + description: "Routing follows a reverse-ordered way after two intersections are inserted", + mode: "car", + metric: "distance", + from: { nodeId: 61 }, + to: { nodeId: 63 }, + expect: { + reachable: true, + distanceMeters: { min: 435, max: 455 }, + requiredWayIds: [140, 141], + }, + }, + { + id: "synthetic-access-car", + description: "Cars detour around a residential way tagged motor_vehicle=no", + mode: "car", + metric: "distance", + from: { nodeId: 70 }, + to: { nodeId: 72 }, + graphPolicy: "access-aware", + expect: { + reachable: true, + distanceMeters: { min: 305, max: 325 }, + requiredWayIds: [151], + forbiddenWayIds: [150], + }, + }, + { + id: "synthetic-access-walk", + description: "Walking may use a residential way explicitly designated for foot access", + mode: "walk", + metric: "distance", + from: { nodeId: 70 }, + to: { nodeId: 72 }, + expect: { + reachable: true, + distanceMeters: { min: 215, max: 230 }, + requiredWayIds: [150], + forbiddenWayIds: [151], + }, + }, +] as const satisfies readonly RoutingTestCase[]; + +/** The offset sidewalk inputs before fuzzy attachment remain separate WALK components. */ +export const SYNTHETIC_CONFLATION_DISCONNECTED_CASES = [ + { + id: "synthetic-conflation-walk", + description: "The imported sidewalk is disconnected before fuzzy network attachment", + mode: "walk", + metric: "distance", + from: { nodeId: 801 }, + to: { nodeId: 902 }, + expect: { reachable: false }, + }, + { + id: "synthetic-conflation-car", + description: "Pedestrian-only source and target geometry is unavailable to cars", + mode: "car", + metric: "distance", + from: { nodeId: 801 }, + to: { nodeId: 902 }, + expect: { reachable: false }, + }, +] as const satisfies readonly RoutingTestCase[]; + +/** The same sidewalk pair after accepting its high-confidence pedestrian attachment. */ +export const SYNTHETIC_CONFLATION_ATTACHED_CASES = [ + { + id: "synthetic-conflation-walk", + description: "Fuzzy attachment joins the aligned imported and base sidewalks", + mode: "walk", + metric: "distance", + from: { nodeId: 801 }, + to: { nodeId: 902 }, + expect: { + reachable: true, + distanceMeters: { min: 215, max: 230 }, + requiredWayIds: [810, 910], + }, + }, + { + id: "synthetic-conflation-car", + description: "A pedestrian attachment does not introduce a car route", + mode: "car", + metric: "distance", + from: { nodeId: 801 }, + to: { nodeId: 902 }, + expect: { reachable: false }, + }, +] as const satisfies readonly RoutingTestCase[]; diff --git a/packages/osmix/test/merge-profile-cli.ts b/packages/osmix/test/merge-profile-cli.ts new file mode 100644 index 00000000..f730ea50 --- /dev/null +++ b/packages/osmix/test/merge-profile-cli.ts @@ -0,0 +1,297 @@ +import { access, writeFile } from "node:fs/promises"; +import { arch, cpus, platform, totalmem } from "node:os"; + +import { getFixtureFileReadStream, getFixturePath, PBFs } from "@osmix/test-utils/fixtures"; + +import { fromPbf, toPbfBuffer, type Osm, type OsmMergeOptions } from "../src/index.ts"; +import { + measureMergeProfileTask, + osmEntityCounts, + profileMerge, + profileWorkerConflation, + type MergeProfileOperationCounts, + type MergeProfileRun, + type MergeProfileStage, +} from "./merge-profile-harness.ts"; +import { createMonacoRoutingPatch } from "./synthetic-routing-fixture.ts"; + +type MergeProfileScenario = "monaco" | "yakima" | "eastern-washington"; + +interface ScenarioDefinition { + baseFixture: string; + patchFixture: string; + defaultRuns: number; + options: Partial<OsmMergeOptions>; + workerConflation?: boolean; +} + +interface MergeProfileReport { + schemaVersion: 1; + scenario: MergeProfileScenario; + fixtures: { base: string; patch: string }; + mergeOptions: Partial<OsmMergeOptions>; + startedAt: string; + runtime: { + node: string; + platform: string; + architecture: string; + cpu: string; + logicalCpus: number; + totalMemoryBytes: number; + commit?: string; + }; + runs: MergeProfileRun[]; + medianStageDurationMs: Record<string, number>; + equivalence: { + identicalFingerprints: boolean; + identicalOperationCounts: boolean; + }; +} + +const ALL_MERGE_STEPS = { + directMerge: true, + deduplicateNodes: true, + deduplicateWays: true, + createIntersections: true, +} as const; + +const SCENARIOS: Record<MergeProfileScenario, ScenarioDefinition> = { + monaco: { + baseFixture: PBFs["monaco"]!.url, + patchFixture: "generated Monaco routing patch", + defaultRuns: 5, + options: ALL_MERGE_STEPS, + }, + yakima: { + baseFixture: "yakima-full.osm.pbf", + patchFixture: "yakima.osw.pbf", + defaultRuns: 3, + options: { + ...ALL_MERGE_STEPS, + conflation: { + propertyKeys: ["barrier", "crossing", "kerb", "tactile_paving"], + attachNetwork: true, + maxDistanceMeters: 1, + automatic: "high-confidence", + }, + }, + workerConflation: true, + }, + "eastern-washington": { + baseFixture: "osmix-e_wa_osm.pbf", + patchFixture: "east_washington_sidewalk_proviso_1.pbf", + defaultRuns: 1, + options: ALL_MERGE_STEPS, + }, +}; + +function usage(): string { + return `Usage: pnpm --filter osmix profile:merge -- [options] + +Options: + --scenario <monaco|yakima|eastern-washington> Fixture pair (default: monaco) + --runs <count> Repetitions (defaults: 5/3/1) + --output <path> Also write the JSON report to a file + --help Show this help + +The same values can be set with OSMIX_MERGE_PROFILE_SCENARIO, +OSMIX_MERGE_PROFILE_RUNS, and OSMIX_MERGE_PROFILE_OUTPUT.`; +} + +function optionValue(arguments_: string[], name: string): string | undefined { + const index = arguments_.indexOf(name); + if (index === -1) return undefined; + const value = arguments_[index + 1]; + if (!value || value.startsWith("--")) throw Error(`${name} requires a value`); + return value; +} + +function parseScenario(value: string | undefined): MergeProfileScenario { + if (value === undefined) return "monaco"; + if (value === "monaco" || value === "yakima" || value === "eastern-washington") return value; + throw Error(`Unknown merge profile scenario: ${value}`); +} + +function parseRuns(value: string | undefined, fallback: number): number { + if (value === undefined) return fallback; + const runs = Number(value); + if (!Number.isSafeInteger(runs) || runs < 1) throw Error(`Invalid run count: ${value}`); + return runs; +} + +async function requireFixture(name: string): Promise<void> { + const path = getFixturePath(name); + try { + await access(path); + } catch { + throw Error(`Required local fixture is missing: ${path}`); + } +} + +async function loadInputs( + scenario: MergeProfileScenario, + definition: ScenarioDefinition, +): Promise<{ base: Osm; patch: Osm; stages: MergeProfileStage[] }> { + await requireFixture(definition.baseFixture); + const baseProfile = await measureMergeProfileTask("load-base-pbf", async () => { + const base = await fromPbf( + getFixtureFileReadStream(definition.baseFixture), + { + id: definition.baseFixture, + }, + () => undefined, + ); + return { value: base, operations: { ...osmEntityCounts(base) } }; + }); + const base = baseProfile.value; + if (scenario === "monaco") { + const patchProfile = await measureMergeProfileTask("load-patch-pbf", async () => { + const patch = await fromPbf( + await toPbfBuffer(createMonacoRoutingPatch(base)), + { + id: "monaco-profile-patch", + }, + () => undefined, + ); + return { value: patch, operations: { ...osmEntityCounts(patch) } }; + }); + return { + base, + patch: patchProfile.value, + stages: [baseProfile.stage, patchProfile.stage], + }; + } + await requireFixture(definition.patchFixture); + const patchProfile = await measureMergeProfileTask("load-patch-pbf", async () => { + const patch = await fromPbf( + getFixtureFileReadStream(definition.patchFixture), + { + id: definition.patchFixture, + }, + () => undefined, + ); + return { value: patch, operations: { ...osmEntityCounts(patch) } }; + }); + return { base, patch: patchProfile.value, stages: [baseProfile.stage, patchProfile.stage] }; +} + +function median(values: number[]): number { + const sorted = [...values].sort((left, right) => left - right); + const middle = Math.floor(sorted.length / 2); + const value = + sorted.length % 2 === 0 ? (sorted[middle - 1]! + sorted[middle]!) / 2 : sorted[middle]!; + return Math.round(value * 1_000) / 1_000; +} + +function medianStageDurations(runs: MergeProfileRun[]): Record<string, number> { + const durations = new Map<string, number[]>(); + for (const run of runs) { + for (const stage of run.stages) { + const values = durations.get(stage.name) ?? []; + values.push(stage.durationMs); + durations.set(stage.name, values); + } + } + return Object.fromEntries([...durations].map(([name, values]) => [name, median(values)])); +} + +function stableOperations(run: MergeProfileRun): Record<string, MergeProfileOperationCounts> { + return Object.fromEntries(run.stages.map((stage) => [stage.name, stage.operations])); +} + +function assertEquivalentRuns(runs: MergeProfileRun[]) { + const expectedFingerprint = JSON.stringify(runs[0]!.fingerprints); + const expectedOperations = JSON.stringify(stableOperations(runs[0]!)); + const identicalFingerprints = runs.every( + (run) => JSON.stringify(run.fingerprints) === expectedFingerprint, + ); + const identicalOperationCounts = runs.every( + (run) => JSON.stringify(stableOperations(run)) === expectedOperations, + ); + if (!identicalFingerprints || !identicalOperationCounts) { + throw Error( + `Profile runs were not deterministic (fingerprints: ${identicalFingerprints}, operations: ${identicalOperationCounts})`, + ); + } + return { identicalFingerprints, identicalOperationCounts }; +} + +function silenceLibraryTimings(): () => void { + const time = console.time; + const timeEnd = console.timeEnd; + console.time = () => undefined; + console.timeEnd = () => undefined; + return () => { + console.time = time; + console.timeEnd = timeEnd; + }; +} + +async function main(): Promise<void> { + const arguments_ = process.argv.slice(2); + if (arguments_.includes("--help")) { + process.stdout.write(`${usage()}\n`); + return; + } + const scenario = parseScenario( + optionValue(arguments_, "--scenario") ?? process.env["OSMIX_MERGE_PROFILE_SCENARIO"], + ); + const definition = SCENARIOS[scenario]; + const runCount = parseRuns( + optionValue(arguments_, "--runs") ?? process.env["OSMIX_MERGE_PROFILE_RUNS"], + definition.defaultRuns, + ); + const output = optionValue(arguments_, "--output") ?? process.env["OSMIX_MERGE_PROFILE_OUTPUT"]; + const startedAt = new Date().toISOString(); + const runs: MergeProfileRun[] = []; + + const restoreLibraryTimings = silenceLibraryTimings(); + try { + for (let run = 1; run <= runCount; run++) { + globalThis.gc?.(); + const { base, patch, stages } = await loadInputs(scenario, definition); + const profile = definition.workerConflation + ? await profileWorkerConflation(base, patch, definition.options, { run }) + : await profileMerge(base, patch, definition.options, { run }); + profile.stages.unshift(...stages); + profile.wallDurationMs = + Math.round( + (profile.wallDurationMs + stages.reduce((sum, stage) => sum + stage.durationMs, 0)) * + 1_000, + ) / 1_000; + profile.processPeakRssBytes = Math.max( + profile.processPeakRssBytes, + ...stages.map((stage) => stage.processPeakRssBytes), + ); + runs.push(profile); + } + } finally { + restoreLibraryTimings(); + } + + const cpuList = cpus(); + const report: MergeProfileReport = { + schemaVersion: 1, + scenario, + fixtures: { base: definition.baseFixture, patch: definition.patchFixture }, + mergeOptions: definition.options, + startedAt, + runtime: { + node: process.version, + platform: platform(), + architecture: arch(), + cpu: cpuList[0]?.model ?? "unknown", + logicalCpus: cpuList.length, + totalMemoryBytes: totalmem(), + ...(process.env["GITHUB_SHA"] ? { commit: process.env["GITHUB_SHA"] } : {}), + }, + runs, + medianStageDurationMs: medianStageDurations(runs), + equivalence: assertEquivalentRuns(runs), + }; + const json = `${JSON.stringify(report, null, 2)}\n`; + if (output) await writeFile(output, json); + process.stdout.write(json); +} + +await main(); diff --git a/packages/osmix/test/merge-profile-harness.ts b/packages/osmix/test/merge-profile-harness.ts new file mode 100644 index 00000000..308cc8d0 --- /dev/null +++ b/packages/osmix/test/merge-profile-harness.ts @@ -0,0 +1,525 @@ +import { createHash } from "node:crypto"; +import { performance } from "node:perf_hooks"; + +import { + discoverConflationCandidatesForTrustedMerge, + generateConflationApplicationArtifactsFromTrustedDiscovery, +} from "@osmix/change/src/internal/conflation.ts"; +import type { Osm } from "@osmix/core"; +import type { OsmEntity } from "@osmix/types"; + +import { + applyChangesetToOsm, + createOsmJsonReadableStream, + OsmBlocksToPbfBytesTransformStream, + OsmJsonToBlocksTransformStream, + OsmChangeset, + OsmixWorker, + type OsmConflationGenerationResult, + type OsmConflationSummary, + type OsmMergeOptions, + type OsmChangesetStats, +} from "../src/index.ts"; + +export type MergeProfileOperationCounts = Record<string, number>; + +export interface MergeProfileStage { + name: string; + durationMs: number; + cpuUserMs: number; + cpuSystemMs: number; + rssBytesBefore: number; + rssBytesAfter: number; + heapUsedBytesBefore: number; + heapUsedBytesAfter: number; + /** Process-lifetime RSS high-water at stage completion, not a stage-local maximum. */ + processPeakRssBytes: number; + operations: MergeProfileOperationCounts; +} + +export interface MergeProfileFingerprints { + /** The built-in storage-level fingerprint, including typed-buffer ordering. */ + contentHash: string; + /** A semantic fingerprint with sorted entities and object keys but ordered refs/members. */ + canonicalSha256: string; + /** A PBF byte fingerprint after normalizing the serializer's current-time header. */ + normalizedPbfSha256: string; + pbfBytes: number; +} + +export interface MergeProfileRun { + run: number; + stages: MergeProfileStage[]; + inputs: { + base: MergeProfileEntityCounts; + patch: MergeProfileEntityCounts; + }; + output: MergeProfileEntityCounts; + fingerprints: MergeProfileFingerprints; + wallDurationMs: number; + /** Process-lifetime RSS high-water at run completion. */ + processPeakRssBytes: number; +} + +export interface MergeProfileEntityCounts { + nodes: number; + ways: number; + relations: number; +} + +export interface ProfileMergeOptions { + /** A stable one-based run number included in reports. */ + run?: number; + /** Include semantic and serialized fingerprints. Enabled by default. */ + fingerprint?: boolean; +} + +interface StageResult<T> { + value: T; + operations?: MergeProfileOperationCounts; +} + +interface MemorySnapshot { + rss: number; + heapUsed: number; + peakRss: number; +} + +class ProfileOsmixWorker extends OsmixWorker { + register(osm: Osm): void { + this.set(osm.id, osm); + } + + read(osmId: string): Osm { + return this.get(osmId); + } +} + +function roundMilliseconds(value: number): number { + return Math.round(value * 1_000) / 1_000; +} + +function memorySnapshot(): MemorySnapshot { + const memory = process.memoryUsage(); + return { + rss: memory.rss, + heapUsed: memory.heapUsed, + // Node reports maxRSS in KiB on every supported platform. + peakRss: process.resourceUsage().maxRSS * 1_024, + }; +} + +class MergeProfileRecorder { + readonly stages: MergeProfileStage[] = []; + + async measure<T>(name: string, task: () => StageResult<T> | Promise<StageResult<T>>): Promise<T> { + const memoryBefore = memorySnapshot(); + const cpuBefore = process.cpuUsage(); + const started = performance.now(); + const result = await task(); + const durationMs = performance.now() - started; + const cpu = process.cpuUsage(cpuBefore); + const memoryAfter = memorySnapshot(); + + this.stages.push({ + name, + durationMs: roundMilliseconds(durationMs), + cpuUserMs: roundMilliseconds(cpu.user / 1_000), + cpuSystemMs: roundMilliseconds(cpu.system / 1_000), + rssBytesBefore: memoryBefore.rss, + rssBytesAfter: memoryAfter.rss, + heapUsedBytesBefore: memoryBefore.heapUsed, + heapUsedBytesAfter: memoryAfter.heapUsed, + processPeakRssBytes: Math.max(memoryBefore.peakRss, memoryAfter.peakRss), + operations: result.operations ?? {}, + }); + return result.value; + } +} + +/** Measure setup work, such as fixture loading, with the same stage schema. */ +export async function measureMergeProfileTask<T>( + name: string, + task: () => StageResult<T> | Promise<StageResult<T>>, +): Promise<{ value: T; stage: MergeProfileStage }> { + const recorder = new MergeProfileRecorder(); + const value = await recorder.measure(name, task); + return { value, stage: recorder.stages[0]! }; +} + +export function osmEntityCounts(osm: Osm): MergeProfileEntityCounts { + return { + nodes: osm.nodes.size, + ways: osm.ways.size, + relations: osm.relations.size, + }; +} + +function changesetCounts(stats: OsmChangesetStats): MergeProfileOperationCounts { + return { + totalChanges: stats.totalChanges, + nodeChanges: stats.nodeChanges, + wayChanges: stats.wayChanges, + relationChanges: stats.relationChanges, + deduplicatedNodes: stats.deduplicatedNodes, + deduplicatedNodesReplaced: stats.deduplicatedNodesReplaced, + deduplicatedWays: stats.deduplicatedWays, + intersectionPointsFound: stats.intersectionPointsFound, + intersectionNodesCreated: stats.intersectionNodesCreated, + }; +} + +function prefixedCounts( + prefix: string, + counts: MergeProfileEntityCounts | OsmConflationSummary, +): MergeProfileOperationCounts { + return Object.fromEntries( + Object.entries(counts).map(([key, value]) => [ + `${prefix}${key[0]!.toUpperCase()}${key.slice(1)}`, + value, + ]), + ); +} + +/** + * Serialize a JSON-compatible value with stable object-key order. Array order is + * intentionally retained because way refs and relation members are structural. + */ +function stableJson(value: unknown): string { + if (value === null || typeof value !== "object") return JSON.stringify(value); + if (Array.isArray(value)) return `[${value.map(stableJson).join(",")}]`; + const entries = Object.entries(value as Record<string, unknown>) + .filter(([, entry]) => entry !== undefined) + .sort(([left], [right]) => (left < right ? -1 : left > right ? 1 : 0)); + return `{${entries.map(([key, entry]) => `${JSON.stringify(key)}:${stableJson(entry)}`).join(",")}}`; +} + +/** Create a semantic digest independent of entity and tag insertion order. */ +export function canonicalOsmSha256(osm: Osm): string { + const hash = createHash("sha256"); + const update = (type: string, entities: Iterable<OsmEntity>) => { + hash.update(`${type}\n`); + for (const entity of entities) hash.update(`${stableJson(entity)}\n`); + }; + update("nodes", osm.nodes.sorted()); + update("ways", osm.ways.sorted()); + update("relations", osm.relations.sorted()); + return hash.digest("hex"); +} + +async function* sortedEntities(osm: Osm): AsyncGenerator<OsmEntity> { + for (const node of osm.nodes.osmSorted()) yield node; + for (const way of osm.ways.osmSorted()) yield way; + for (const relation of osm.relations.osmSorted()) yield relation; +} + +async function pbfFingerprint(osm: Osm): Promise<{ sha256: string; bytes: number }> { + const hash = createHash("sha256"); + let bytes = 0; + // Production export records Date.now() in this header. Normalizing that one + // volatile field makes byte comparisons useful without changing serialization. + const stream = createOsmJsonReadableStream( + { + ...osm.header, + writingprogram: "@osmix/core", + osmosis_replication_timestamp: 1_700_000_000_000, + }, + sortedEntities(osm), + ) + .pipeThrough(new OsmJsonToBlocksTransformStream()) + .pipeThrough(new OsmBlocksToPbfBytesTransformStream()); + await stream.pipeTo( + new WritableStream<Uint8Array>({ + write(chunk) { + hash.update(chunk); + bytes += chunk.byteLength; + }, + }), + ); + return { sha256: hash.digest("hex"), bytes }; +} + +async function collectFingerprints( + recorder: MergeProfileRecorder, + osm: Osm, + enabled: boolean, +): Promise<MergeProfileFingerprints> { + if (!enabled) { + return { + contentHash: osm.contentHash(), + canonicalSha256: "not-collected", + normalizedPbfSha256: "not-collected", + pbfBytes: 0, + }; + } + const canonicalSha256 = await recorder.measure("fingerprint-canonical-entities", () => ({ + value: canonicalOsmSha256(osm), + operations: prefixedCounts("entity", osmEntityCounts(osm)), + })); + const pbf = await recorder.measure("fingerprint-pbf-output", async () => { + const result = await pbfFingerprint(osm); + return { value: result, operations: { pbfBytes: result.bytes } }; + }); + return { + contentHash: osm.contentHash(), + canonicalSha256, + normalizedPbfSha256: pbf.sha256, + pbfBytes: pbf.bytes, + }; +} + +function routingDiagnosticCounts( + diagnostics: OsmConflationGenerationResult["routing"], +): MergeProfileOperationCounts { + const counts: MergeProfileOperationCounts = {}; + for (const mode of ["car", "walk"] as const) { + for (const view of ["before", "after", "delta"] as const) { + for (const [key, value] of Object.entries(diagnostics[mode][view])) { + counts[ + `${mode}${view[0]!.toUpperCase()}${view.slice(1)}${key[0]!.toUpperCase()}${key.slice(1)}` + ] = value; + } + } + } + return counts; +} + +/** + * Run the merge pipeline through its public changeset operations while recording + * each expensive boundary separately. This intentionally follows the same order + * as `merge`: ordinary direct/exact changes, optional conflation, then intersections. + */ +export async function profileMerge( + base: Osm, + patch: Osm, + options: Partial<OsmMergeOptions>, + profileOptions: ProfileMergeOptions = {}, +): Promise<MergeProfileRun> { + const recorder = new MergeProfileRecorder(); + const wallStarted = performance.now(); + let modifiedBase = base; + + if (options.directMerge || options.deduplicateNodes || options.deduplicateWays) { + const changeset = await recorder.measure("prepare-direct-exact-changeset", () => ({ + value: new OsmChangeset(base), + operations: prefixedCounts("base", osmEntityCounts(base)), + })); + + if (options.directMerge) { + await recorder.measure("generate-direct-changes", () => { + changeset.generateDirectChanges(patch); + return { value: undefined, operations: changesetCounts(changeset.stats) }; + }); + } + + if (options.deduplicateNodes) { + await recorder.measure("reconcile-exact-nodes", () => { + changeset.deduplicateNodes(patch.nodes); + return { value: undefined, operations: changesetCounts(changeset.stats) }; + }); + } + + if (options.deduplicateWays) { + await recorder.measure("reconcile-exact-ways", () => { + let waysChecked = 0; + let waysReconciled = 0; + for (const reconciled of changeset.deduplicateWaysGenerator(patch.ways)) { + waysChecked++; + waysReconciled += reconciled; + } + return { + value: undefined, + operations: { + ...changesetCounts(changeset.stats), + waysChecked, + waysReconciled, + }, + }; + }); + } + + modifiedBase = await recorder.measure("apply-direct-exact-changes", () => { + const result = applyChangesetToOsm(changeset); + return { + value: result, + operations: { + ...changesetCounts(changeset.stats), + ...prefixedCounts("output", osmEntityCounts(result)), + }, + }; + }); + } + + if (options.conflation) { + if (!options.directMerge) { + throw Error("Fuzzy conflation requires directMerge to preserve unmatched patch entities"); + } + const discovery = await recorder.measure("discover-conflation-candidates", () => { + const result = discoverConflationCandidatesForTrustedMerge(base, patch, options.conflation!); + return { + value: result, + operations: prefixedCounts("candidate", result.summary), + }; + }); + const conflation = await recorder.measure("generate-conflation-changes", () => { + const result = generateConflationApplicationArtifactsFromTrustedDiscovery( + modifiedBase, + patch, + discovery, + base, + options.conflation?.decisions ?? [], + ); + return { value: result, operations: changesetCounts(result.changeset.stats) }; + }); + modifiedBase = await recorder.measure("apply-conflation-changes", () => { + // Production installs the exact result already materialized and validated + // during generation. Keep this boundary visible without doing the work twice. + const result = conflation.result; + return { + value: result, + operations: { + ...changesetCounts(conflation.changeset.stats), + ...prefixedCounts("output", osmEntityCounts(result)), + }, + }; + }); + } + + if (options.createIntersections) { + const changeset = await recorder.measure("prepare-intersection-changeset", () => ({ + value: new OsmChangeset(modifiedBase), + operations: prefixedCounts("base", osmEntityCounts(modifiedBase)), + })); + await recorder.measure("create-safe-intersections", () => { + let waysChecked = 0; + for (const _result of changeset.createIntersectionsForWaysGenerator(patch.ways)) { + waysChecked++; + } + return { + value: undefined, + operations: { ...changesetCounts(changeset.stats), waysChecked }, + }; + }); + modifiedBase = await recorder.measure("apply-intersection-changes", () => { + const result = applyChangesetToOsm(changeset); + return { + value: result, + operations: { + ...changesetCounts(changeset.stats), + ...prefixedCounts("output", osmEntityCounts(result)), + }, + }; + }); + } + + const fingerprints = await collectFingerprints( + recorder, + modifiedBase, + profileOptions.fingerprint ?? true, + ); + + return { + run: profileOptions.run ?? 1, + stages: recorder.stages, + inputs: { base: osmEntityCounts(base), patch: osmEntityCounts(patch) }, + output: osmEntityCounts(modifiedBase), + fingerprints, + wallDurationMs: roundMilliseconds(performance.now() - wallStarted), + processPeakRssBytes: memorySnapshot().peakRss, + }; +} + +/** + * Profile the production worker conflation path, including routing diagnostics, + * the automatic-attachment CAR projection, and installation of the materialized result. + */ +export async function profileWorkerConflation( + base: Osm, + patch: Osm, + options: Partial<OsmMergeOptions>, + profileOptions: ProfileMergeOptions = {}, +): Promise<MergeProfileRun> { + if (!options.conflation) throw Error("Worker conflation profiling requires conflation options"); + if (!options.directMerge) throw Error("Worker conflation profiling requires directMerge"); + const recorder = new MergeProfileRecorder(); + const wallStarted = performance.now(); + const worker = new ProfileOsmixWorker(); + await recorder.measure("register-worker-inputs", () => { + worker.register(base); + worker.register(patch); + return { + value: undefined, + operations: { + ...prefixedCounts("base", osmEntityCounts(base)), + ...prefixedCounts("patch", osmEntityCounts(patch)), + }, + }; + }); + + await recorder.measure("worker-discover-conflation-candidates", () => { + const summary = worker.discoverConflation(base.id, patch.id, options.conflation!); + return { value: undefined, operations: prefixedCounts("candidate", summary) }; + }); + const generation = await recorder.measure("worker-generate-conflation-changeset", () => { + const result = worker.generateConflationChangeset(base.id, { + directMerge: true, + deduplicateNodes: options.deduplicateNodes ?? false, + deduplicateWays: options.deduplicateWays ?? false, + createIntersections: false, + }); + return { + value: result, + operations: { + ...changesetCounts(result.stats), + ...routingDiagnosticCounts(result.routing), + }, + }; + }); + await recorder.measure("worker-apply-conflation-result", () => { + worker.applyChangesAndReplace(base.id); + const result = worker.read(base.id); + return { + value: undefined, + operations: { + ...changesetCounts(generation.stats), + ...prefixedCounts("output", osmEntityCounts(result)), + }, + }; + }); + + if (options.createIntersections) { + const stats = await recorder.measure("worker-create-safe-intersections", async () => { + const result = await worker.generateChangeset(base.id, patch.id, { + createIntersections: true, + }); + return { value: result, operations: changesetCounts(result) }; + }); + await recorder.measure("worker-apply-intersection-changes", () => { + worker.applyChangesAndReplace(base.id); + const result = worker.read(base.id); + return { + value: undefined, + operations: { + ...changesetCounts(stats), + ...prefixedCounts("output", osmEntityCounts(result)), + }, + }; + }); + } + + const output = worker.read(base.id); + const fingerprints = await collectFingerprints( + recorder, + output, + profileOptions.fingerprint ?? true, + ); + return { + run: profileOptions.run ?? 1, + stages: recorder.stages, + inputs: { base: osmEntityCounts(base), patch: osmEntityCounts(patch) }, + output: osmEntityCounts(output), + fingerprints, + wallDurationMs: roundMilliseconds(performance.now() - wallStarted), + processPeakRssBytes: memorySnapshot().peakRss, + }; +} diff --git a/packages/osmix/test/merge-profile.test.ts b/packages/osmix/test/merge-profile.test.ts new file mode 100644 index 00000000..7420a7ca --- /dev/null +++ b/packages/osmix/test/merge-profile.test.ts @@ -0,0 +1,202 @@ +import { getFixtureFileReadStream, PBFs } from "@osmix/test-utils/fixtures"; +import { describe, expect, it } from "vitest"; + +import { fromPbf, merge, Osm, toPbfBuffer } from "../src/index.ts"; +import { + canonicalOsmSha256, + profileMerge, + profileWorkerConflation, +} from "./merge-profile-harness.ts"; +import { + createMonacoRoutingPatch, + createSyntheticConflationRoutingInputs, + createSyntheticRoutingBase, + createSyntheticRoutingPatch, + roundTripRoutingOsm, +} from "./synthetic-routing-fixture.ts"; + +const ALL_MERGE_STEPS = { + directMerge: true, + deduplicateNodes: true, + deduplicateWays: true, + createIntersections: true, +} as const; + +function complete(osm: Osm): Osm { + osm.buildIndexes(); + osm.buildSpatialIndexes(); + return osm; +} + +describe("merge performance harness", () => { + it("uses a semantic fingerprint that ignores insertion and object-key order", () => { + const first = new Osm({ id: "first" }); + first.nodes.addNode({ id: 2, lon: 1, lat: 1, tags: { name: "Two", source: "survey" } }); + first.nodes.addNode({ id: 1, lon: 0, lat: 0 }); + first.ways.addWay({ id: 10, refs: [1, 2], tags: { name: "Way", highway: "footway" } }); + + const second = new Osm({ id: "second" }); + second.nodes.addNode({ id: 1, lat: 0, lon: 0 }); + second.nodes.addNode({ id: 2, lat: 1, lon: 1, tags: { source: "survey", name: "Two" } }); + second.ways.addWay({ id: 10, refs: [1, 2], tags: { highway: "footway", name: "Way" } }); + + expect(canonicalOsmSha256(complete(first))).toBe(canonicalOsmSha256(complete(second))); + const reversed = new Osm({ id: "reversed" }); + reversed.nodes.addNode({ id: 1, lon: 0, lat: 0 }); + reversed.nodes.addNode({ id: 2, lon: 1, lat: 1, tags: { name: "Two", source: "survey" } }); + reversed.ways.addWay({ id: 10, refs: [2, 1], tags: { highway: "footway", name: "Way" } }); + expect(canonicalOsmSha256(complete(reversed))).not.toBe(canonicalOsmSha256(first)); + }); + + it("profiles the same ordered full merge as the public pipeline", async () => { + const [base, patch] = await Promise.all([ + roundTripRoutingOsm(createSyntheticRoutingBase(), "profile-synthetic-base"), + roundTripRoutingOsm(createSyntheticRoutingPatch(), "profile-synthetic-patch"), + ]); + const report = await profileMerge(base, patch, ALL_MERGE_STEPS); + const publicResult = await merge(base, patch, ALL_MERGE_STEPS, () => undefined); + + expect(report.stages.map(({ name }) => name)).toEqual([ + "prepare-direct-exact-changeset", + "generate-direct-changes", + "reconcile-exact-nodes", + "reconcile-exact-ways", + "apply-direct-exact-changes", + "prepare-intersection-changeset", + "create-safe-intersections", + "apply-intersection-changes", + "fingerprint-canonical-entities", + "fingerprint-pbf-output", + ]); + expect(report.output).toEqual({ nodes: 33, ways: 14, relations: 1 }); + expect(report.fingerprints.contentHash).toBe(publicResult.contentHash()); + expect(report.fingerprints.canonicalSha256).toBe(canonicalOsmSha256(publicResult)); + expect( + report.stages.find(({ name }) => name === "reconcile-exact-nodes")?.operations, + ).toMatchObject({ deduplicatedNodes: 1, deduplicatedNodesReplaced: 2 }); + expect( + report.stages.find(({ name }) => name === "reconcile-exact-ways")?.operations, + ).toMatchObject({ waysChecked: 7, waysReconciled: 0 }); + expect( + report.stages.find(({ name }) => name === "create-safe-intersections")?.operations, + ).toMatchObject({ + waysChecked: 7, + intersectionPointsFound: 3, + intersectionNodesCreated: 3, + }); + }); + + it("profiles worker conflation generation with routing safety diagnostics", async () => { + const { base, patch } = createSyntheticConflationRoutingInputs(); + const report = await profileWorkerConflation(base, patch, { + directMerge: true, + deduplicateNodes: true, + deduplicateWays: true, + createIntersections: false, + conflation: { + propertyKeys: ["name"], + attachNetwork: true, + maxDistanceMeters: 1, + automatic: "high-confidence", + }, + }); + + expect(report.stages.map(({ name }) => name)).toEqual([ + "register-worker-inputs", + "worker-discover-conflation-candidates", + "worker-generate-conflation-changeset", + "worker-apply-conflation-result", + "fingerprint-canonical-entities", + "fingerprint-pbf-output", + ]); + expect(report.output).toEqual({ nodes: 82, ways: 80, relations: 0 }); + expect( + report.stages.find(({ name }) => name === "worker-discover-conflation-candidates") + ?.operations, + ).toMatchObject({ + candidateTotal: 81, + candidateAutomatic: 1, + candidateReview: 0, + candidateBlocked: 0, + candidateUnmatched: 80, + }); + const generation = report.stages.find( + ({ name }) => name === "worker-generate-conflation-changeset", + )?.operations; + expect(generation).toMatchObject({ + totalChanges: 82, + nodeChanges: 42, + wayChanges: 40, + carDeltaRoutableNodes: 0, + carDeltaEdges: 0, + carDeltaComponents: 0, + walkDeltaRoutableNodes: -1, + walkDeltaComponents: -1, + }); + }); + + it("profiles high-level conflation with the production discovery reuse path", async () => { + const { base, patch } = createSyntheticConflationRoutingInputs(); + const options = { + directMerge: true, + deduplicateNodes: true, + deduplicateWays: true, + createIntersections: false, + conflation: { + propertyKeys: ["name"], + attachNetwork: true, + maxDistanceMeters: 1, + automatic: "high-confidence" as const, + }, + }; + const report = await profileMerge(base, patch, options); + const publicResult = await merge(base, patch, options, () => undefined); + + expect(report.stages.map(({ name }) => name)).toEqual([ + "prepare-direct-exact-changeset", + "generate-direct-changes", + "reconcile-exact-nodes", + "reconcile-exact-ways", + "apply-direct-exact-changes", + "discover-conflation-candidates", + "generate-conflation-changes", + "apply-conflation-changes", + "fingerprint-canonical-entities", + "fingerprint-pbf-output", + ]); + expect(report.fingerprints.contentHash).toBe(publicResult.contentHash()); + expect(report.fingerprints.canonicalSha256).toBe(canonicalOsmSha256(publicResult)); + }); + + it("locks Monaco full-merge operations and output fingerprints", async () => { + const fixture = PBFs["monaco"]!; + const base = await fromPbf(getFixtureFileReadStream(fixture.url), { id: "profile-monaco" }); + const patch = await fromPbf(await toPbfBuffer(createMonacoRoutingPatch(base)), { + id: "profile-monaco-patch", + }); + const report = await profileMerge(base, patch, ALL_MERGE_STEPS); + + expect(report.inputs).toEqual({ + base: { nodes: 14_286, ways: 3_346, relations: 46 }, + patch: { nodes: 2, ways: 1, relations: 0 }, + }); + expect(report.output).toEqual({ nodes: 14_287, ways: 3_347, relations: 46 }); + expect( + report.stages.find(({ name }) => name === "reconcile-exact-nodes")?.operations, + ).toMatchObject({ deduplicatedNodes: 1, deduplicatedNodesReplaced: 1 }); + expect( + report.stages.find(({ name }) => name === "reconcile-exact-ways")?.operations, + ).toMatchObject({ waysChecked: 1, waysReconciled: 0 }); + expect( + report.stages.find(({ name }) => name === "create-safe-intersections")?.operations, + ).toMatchObject({ waysChecked: 1 }); + expect(report.fingerprints).toMatchObject({ + contentHash: "c941a5b8", + canonicalSha256: "4f47037cf117c361dfc36113a7734eebd37b0f4a9e4d84861dc0ca5e3527ea5d", + }); + // The compressed byte stream can vary with Node's zlib version. Reports keep + // that useful same-runtime fingerprint, while CI locks semantic output above. + expect(report.fingerprints.normalizedPbfSha256).toMatch(/^[a-f\d]{64}$/); + expect(report.fingerprints.pbfBytes).toBeGreaterThan(0); + }, 30_000); +}); diff --git a/packages/osmix/test/merge.test.ts b/packages/osmix/test/merge.test.ts index bcc3eb31..20c01cc6 100644 --- a/packages/osmix/test/merge.test.ts +++ b/packages/osmix/test/merge.test.ts @@ -85,20 +85,20 @@ describe("merge osm", () => { changeset = new OsmChangeset(baseOsm); changeset.createIntersectionsForWays(osm2.ways); - // Pending intersection nodes are resolved from the changeset when a way is spliced - // again. This keeps every ref aligned with real geometry instead of aliasing a node - // from the base index, and can expose additional legitimate intersections. + // Intersections are grouped by their containing segment and inserted in geometric + // order. Endpoint reuse that would create duplicate or degenerate refs falls back + // to a dedicated exact intersection node. expect(changeset.stats).toEqual({ osmId: baseOsm.id, - totalChanges: 9_461, - nodeChanges: 5_824, - wayChanges: 3_637, + totalChanges: 9_508, + nodeChanges: 5_869, + wayChanges: 3_639, relationChanges: 0, deduplicatedNodes: 0, deduplicatedNodesReplaced: 0, deduplicatedWays: 0, - intersectionPointsFound: 3_187, - intersectionNodesCreated: 2_623, + intersectionPointsFound: 3_105, + intersectionNodesCreated: 2_609, }); baseOsm = applyChangesetToOsm(changeset); @@ -122,7 +122,10 @@ describe("merge osm", () => { }, }); }, - 30_000, + // This optional integration fixture loads and indexes nearly one million + // entities before creating intersections. Keep enough headroom for a full + // workspace run where other Vitest projects compete for CPU and memory. + 120_000, ); it.skip("should merge seattle with deduplication", async () => { diff --git a/packages/osmix/test/r5/R5RoutingOracle.java b/packages/osmix/test/r5/R5RoutingOracle.java new file mode 100644 index 00000000..f5f03eef --- /dev/null +++ b/packages/osmix/test/r5/R5RoutingOracle.java @@ -0,0 +1,221 @@ +import com.conveyal.r5.profile.StreetMode; +import com.conveyal.r5.streets.StreetRouter; +import com.conveyal.r5.streets.VertexStore; +import com.conveyal.r5.transit.TransportNetwork; + +import java.io.BufferedWriter; +import java.io.IOException; +import java.nio.file.Files; +import java.nio.file.Path; +import java.util.ArrayList; +import java.util.List; + +/** + * Local-only point-to-point R5 oracle for Osmix's Monaco routing manifest. + * + * <p>This source is compiled against an adjacent R5 checkout by r5-oracle.init.gradle. It is not + * part of either product's runtime or CI dependency graph.</p> + */ +public final class R5RoutingOracle { + + private record RouteCase( + String id, + StreetMode mode, + double fromLon, + double fromLat, + double toLon, + double toLat, + boolean exactOrigin, + boolean exactDestination + ) {} + + private record VertexResolution(int index, String method) {} + + private record RouteResult( + boolean originLinked, + boolean destinationLinked, + boolean reachable, + double distanceMeters, + int durationSeconds + ) {} + + private R5RoutingOracle() {} + + public static void main(String[] args) throws Exception { + if (args.length < 4 || args.length % 2 != 0) { + throw new IllegalArgumentException( + "Usage: R5RoutingOracle <manifest.tsv> <output-directory> " + + "<dataset-id> <input.osm.pbf> [<dataset-id> <input.osm.pbf> ...]" + ); + } + + Path manifest = Path.of(args[0]); + Path outputDirectory = Path.of(args[1]); + Files.createDirectories(outputDirectory); + List<RouteCase> routeCases = readCases(manifest); + + for (int argument = 2; argument < args.length; argument += 2) { + String datasetId = args[argument]; + Path pbf = Path.of(args[argument + 1]).toAbsolutePath(); + runDataset(datasetId, pbf, routeCases, outputDirectory); + } + } + + private static List<RouteCase> readCases(Path manifest) throws IOException { + List<RouteCase> cases = new ArrayList<>(); + List<String> lines = Files.readAllLines(manifest); + for (int lineNumber = 1; lineNumber < lines.size(); lineNumber += 1) { + String line = lines.get(lineNumber); + if (line.isBlank()) continue; + String[] cells = line.split("\t", -1); + if (cells.length < 10) { + throw new IllegalArgumentException( + "Malformed routing manifest line " + (lineNumber + 1) + ": " + line + ); + } + cases.add(new RouteCase( + cells[0], + StreetMode.valueOf(cells[1]), + Double.parseDouble(cells[2]), + Double.parseDouble(cells[3]), + Double.parseDouble(cells[4]), + Double.parseDouble(cells[5]), + cells[6].equals("osm-node"), + cells[8].equals("osm-node") + )); + } + return cases; + } + + private static void runDataset( + String datasetId, + Path pbf, + List<RouteCase> routeCases, + Path outputDirectory + ) throws Exception { + TransportNetwork network = TransportNetwork.fromFiles(pbf.toString(), List.of()); + Path output = outputDirectory.resolve("r5-" + datasetId + ".tsv"); + try (BufferedWriter writer = Files.newBufferedWriter(output)) { + writer.write("case_id\tmode\torigin_vertex_resolution\tdestination_vertex_resolution" + + "\tcoordinate_origin_linked\tcoordinate_destination_linked" + + "\tcoordinate_reachable\tcoordinate_distance_m\tcoordinate_duration_s" + + "\texact_vertex_reachable\texact_vertex_distance_m" + + "\texact_vertex_duration_s\n"); + for (RouteCase routeCase : routeCases) { + VertexResolution origin = resolveVertex( + network, + routeCase.fromLat, + routeCase.fromLon, + routeCase.exactOrigin + ); + VertexResolution destination = resolveVertex( + network, + routeCase.toLat, + routeCase.toLon, + routeCase.exactDestination + ); + RouteResult coordinateResult = routeCoordinates(network, routeCase); + RouteResult exactResult = origin.index >= 0 && destination.index >= 0 + ? routeVertices(network, routeCase.mode, origin.index, destination.index) + : null; + + writer.write(String.join("\t", + routeCase.id, + routeCase.mode.name(), + origin.method, + destination.method, + Boolean.toString(coordinateResult.originLinked), + Boolean.toString(coordinateResult.destinationLinked), + Boolean.toString(coordinateResult.reachable), + coordinateResult.reachable + ? Double.toString(coordinateResult.distanceMeters) + : "", + coordinateResult.reachable + ? Integer.toString(coordinateResult.durationSeconds) + : "", + exactResult == null ? "" : Boolean.toString(exactResult.reachable), + exactResult == null || !exactResult.reachable + ? "" + : Double.toString(exactResult.distanceMeters), + exactResult == null || !exactResult.reachable + ? "" + : Integer.toString(exactResult.durationSeconds) + )); + writer.newLine(); + } + } + System.out.println("Wrote " + output); + } + + private static RouteResult routeCoordinates(TransportNetwork network, RouteCase routeCase) { + StreetRouter router = new StreetRouter(network.streetLayer); + router.streetMode = routeCase.mode; + boolean originLinked = router.setOrigin(routeCase.fromLat, routeCase.fromLon); + boolean destinationLinked = router.setDestination(routeCase.toLat, routeCase.toLon); + StreetRouter.State state = null; + if (originLinked && destinationLinked) { + router.route(); + state = router.getState(router.getDestinationSplit()); + } + return result(originLinked, destinationLinked, state); + } + + private static RouteResult routeVertices( + TransportNetwork network, + StreetMode mode, + int originVertex, + int destinationVertex + ) { + StreetRouter router = new StreetRouter(network.streetLayer); + router.streetMode = mode; + router.setOrigin(originVertex); + router.toVertex = destinationVertex; + router.route(); + return result(true, true, router.getStateAtVertex(destinationVertex)); + } + + private static RouteResult result( + boolean originLinked, + boolean destinationLinked, + StreetRouter.State state + ) { + return new RouteResult( + originLinked, + destinationLinked, + state != null, + state == null ? Double.NaN : state.distance / 1_000d, + state == null ? -1 : state.getDurationSeconds() + ); + } + + /** + * Resolve OSM-node cases to an exact R5 street vertex whenever R5 retained that node as a + * topological vertex. Intermediate shape nodes fall back to normal coordinate-to-edge linking. + */ + private static VertexResolution resolveVertex( + TransportNetwork network, + double lat, + double lon, + boolean exact + ) { + if (!exact) return new VertexResolution(-1, "coordinate"); + + VertexStore vertices = network.streetLayer.vertexStore; + int fixedLat = VertexStore.floatingDegreesToFixed(lat); + int fixedLon = VertexStore.floatingDegreesToFixed(lon); + int match = -1; + int matches = 0; + for (int index = 0; index < vertices.getVertexCount(); index += 1) { + boolean sameCoordinate = + vertices.fixedLats.get(index) == fixedLat + && vertices.fixedLons.get(index) == fixedLon; + if (sameCoordinate) { + match = index; + matches += 1; + } + } + if (matches == 1) return new VertexResolution(match, "exact-osm-node"); + if (matches == 0) return new VertexResolution(-1, "coordinate-fallback-no-vertex"); + return new VertexResolution(-1, "coordinate-fallback-ambiguous-vertex"); + } +} diff --git a/packages/osmix/test/r5/README.md b/packages/osmix/test/r5/README.md new file mode 100644 index 00000000..e34d98c8 --- /dev/null +++ b/packages/osmix/test/r5/README.md @@ -0,0 +1,157 @@ +# Local R5 routing oracle + +R5 is the authority for Conveyal street-mode legality. This local-only runner sends the exact +Monaco endpoints used by the Osmix regression suite through `TransportNetwork.fromFiles`, +`StreetRouter`, and `StreetMode.CAR` or `StreetMode.WALK`. It does not add R5 to Osmix's CI or +package dependency graph. + +The checked-in Osmix expectations cover topology, route shape, broad measurements, and +Dijkstra/A* agreement. Two cases are intentionally policy diagnostics rather than absolute +Osmix goldens: + +- `monaco-motor-vehicle-access`: R5 must not drive on `motor_vehicle=no` way `158215187`. +- `monaco-no-left-turn-restriction`: R5 must honor `no_left_turn` relation `4261963`. + +The implicit-roundabout and reverse-oneway cases are absolute goldens: Osmix and R5 both take the +legal direction implied by `junction=roundabout` and `oneway=-1`. + +The test-only Osmix WALK graph changes highway eligibility and speeds, but the generic +`RoutingGraph` still applies way-level `oneway` and roundabout direction. No accepted Monaco walk +case depends on that limitation; R5 WALK results remain authoritative for modal legality. + +## 1. Export the inputs and matrix from Osmix + +Run from the Osmix repository. Use a fresh temporary directory because R5 creates MapDB sidecar +files beside each input PBF. + +```sh +OSMIX_DIR="$PWD" +ORACLE_DIR="$(mktemp -d /tmp/osmix-r5-oracle.XXXXXX)" +OSMIX_ROUTING_ORACLE_DIR="$ORACLE_DIR" \ + pnpm -w exec vitest run --project osmix \ + packages/osmix/test/routing-after-merge.test.ts +``` + +This opt-in command writes: + +- `routing-cases.tsv`: modes, OSM node IDs, and exact snapped coordinates for R5. +- Raw, empty-merged, synthetic-patched, and reloaded Monaco PBFs: exact oracle inputs. +- `oracle-matrix.json`: current Osmix raw, merged, and PBF-reloaded reports. +- Per-dataset JSON and GeoJSON diagnostics for visual review. +- `synthetic/`: generated merged and PBF-reloaded synthetic networks and reports. +- `conflation-property/`: ordinary, property-only, and property-only-reloaded PBFs. Their + disconnected WALK topology must remain identical. +- `conflation-attachment/`: attached and attached-reloaded pedestrian PBFs. Both must expose the + newly connected WALK route. + +Normal tests never write these files and no command auto-updates checked-in expectations. + +## 2. Run the same matrix through a local R5 checkout + +Run from the R5 repository. The init script adds only a temporary source set and task. Supplying a +temporary build directory keeps generated R5 build files out of the adjacent checkout. + +```sh +R5_ORACLE_BUILD="$(mktemp -d /tmp/osmix-r5-build.XXXXXX)" +gradle --no-daemon --init-script \ + "$OSMIX_DIR/packages/osmix/test/r5/r5-oracle.init.gradle" \ + -PosmixOracleBuildDir="$R5_ORACLE_BUILD" \ + -PosmixOracleSourceDir="$OSMIX_DIR/packages/osmix/test/r5" \ + -PosmixOracleManifest="$ORACLE_DIR/routing-cases.tsv" \ + -PosmixOracleOutputDir="$ORACLE_DIR" \ + runOsmixRoutingOracle +``` + +Use `--offline` when the R5 Gradle dependencies are already cached. The runner produces +one TSV per Monaco dataset in `ORACLE_DIR`. + +The init script accepts a comma-separated `osmixOracleDatasets` override for the generated +conflation variants. Run the property-only matrix and attachment matrix separately so each uses its +matching endpoint expectations: + +```sh +gradle --offline --no-daemon --init-script \ + "$OSMIX_DIR/packages/osmix/test/r5/r5-oracle.init.gradle" \ + -PosmixOracleBuildDir="$R5_ORACLE_BUILD" \ + -PosmixOracleSourceDir="$OSMIX_DIR/packages/osmix/test/r5" \ + -PosmixOracleManifest="$ORACLE_DIR/conflation-property/routing-cases.tsv" \ + -PosmixOracleOutputDir="$ORACLE_DIR/conflation-property" \ + -PosmixOracleDatasets=synthetic-conflation-ordinary,synthetic-conflation-property,synthetic-conflation-property-roundtrip \ + runOsmixRoutingOracle + +gradle --offline --no-daemon --init-script \ + "$OSMIX_DIR/packages/osmix/test/r5/r5-oracle.init.gradle" \ + -PosmixOracleBuildDir="$R5_ORACLE_BUILD" \ + -PosmixOracleSourceDir="$OSMIX_DIR/packages/osmix/test/r5" \ + -PosmixOracleManifest="$ORACLE_DIR/conflation-attachment/routing-cases.tsv" \ + -PosmixOracleOutputDir="$ORACLE_DIR/conflation-attachment" \ + -PosmixOracleDatasets=synthetic-conflation-attachment,synthetic-conflation-attachment-roundtrip \ + runOsmixRoutingOracle +``` + +The primary result columns use normal R5 coordinate-to-edge linking. For node-ID cases, the runner +also reports an exact-vertex result when both OSM nodes survive as unambiguous R5 topological +vertices. R5 collapses intermediate shape nodes, so the endpoint-resolution columns record +`coordinate-fallback-no-vertex` when no exact vertex exists. Exact-vertex results are left blank +rather than guessed when either endpoint is missing or more than one R5 vertex has that coordinate. + +## 3. Review the oracle matrix + +For absolute-golden cases, raw, merged, patched, and reloaded R5 reachability and measurements +should agree; any difference indicates a merge or serialization defect. R5 and Osmix measurements +can differ because their snapping, speed, and policy models are different, so compare reachability, +direction, and plausible bounded metrics rather than exact equality. + +For the two policy diagnostics, inspect the expectation in the last column of +`routing-cases.tsv`. A reachable result alone is insufficient: compare its distance with the +Osmix route in `oracle-matrix.json` and inspect the matching GeoJSON when necessary to confirm R5 +used the legal detour. Do not promote current Osmix behavior for these cases into a golden unless +the missing policy is implemented. + +## Observed Monaco matrix + +The runner was verified on 2026-07-21 with a local R5 checkout at commit +`ac95649c7094bf394b3be43fa523d0fb4447633e` (with unrelated existing local changes). All five raw, +empty-merged, synthetic-patched, and reloaded TSV outputs were byte-for-byte identical. These +values document that run; they are not a CI golden because R5 remains a local oracle. + +| Case | Mode | Coordinate distance | Duration | Exact-vertex result | +| ----------------------------------- | ---- | ------------------: | -------: | ------------------- | +| `monaco-short-drive` | CAR | 254.162 m | 19 s | n/a | +| `monaco-short-walk` | WALK | 254.162 m | 196 s | n/a | +| `monaco-cross-town-drive` | CAR | 5,690.595 m | 1,166 s | n/a | +| `monaco-streets-and-steps-walk` | WALK | 459.287 m | 359 s | n/a | +| `monaco-oneway-forward` | CAR | 48.672 m | 46 s | 34.200 m / 4 s | +| `monaco-oneway-reverse` | CAR | 737.016 m | 204 s | 124.167 m / 103 s | +| `monaco-reverse-oneway-legal` | CAR | 12.456 m | 99 s | n/a | +| `monaco-reverse-oneway-detour` | CAR | 23.473 m | 108 s | n/a | +| `monaco-implicit-roundabout-oneway` | CAR | 73.007 m | 123 s | n/a | +| `monaco-motor-vehicle-access` | CAR | 215.600 m | 50 s | unreachable | +| `monaco-no-left-turn-restriction` | CAR | 490.496 m | 277 s | 288.450 m / 134 s | +| `monaco-tunnel-layer-regression` | CAR | 21.771 m | 106 s | n/a | +| `monaco-reachability-regression` | CAR | 7.148 m | 95 s | n/a | + +The last two cases deliberately assert Osmix node-to-node topology. R5 collapses one endpoint in +each case into an intermediate shape point, then normal coordinate linking can snap to a different +nearby level. Their R5 distance is therefore not compared with the Osmix node-ID golden; raw versus +merged equality remains the valid R5 check for those witnesses. + +## Observed synthetic conflation matrix + +The same 2026-07-21 R5 checkout was also used for the explicit 1-meter conflation variants. The +ordinary fixture contains two aligned footway components whose endpoints are about 0.56 meters +apart. Property transfer changes only a selected `name`; network attachment rewrites the first +imported way reference to the preserved base endpoint. + +| Variant | WALK reachable | Coordinate result | Exact-vertex result | +| ----------------------------------- | -------------- | ----------------: | ------------------: | +| ordinary direct merge | no | n/a | n/a | +| property transfer | no | n/a | n/a | +| property transfer after PBF reload | no | n/a | n/a | +| network attachment | yes | 222.244 m / 237 s | 222.245 m / 240 s | +| network attachment after PBF reload | yes | 222.244 m / 237 s | 222.245 m / 240 s | + +The three property-side TSVs were byte-identical, as were the two attachment-side TSVs. Osmix +separately asserts that the attachment changes WALK from two components to one while its CAR graph +is unchanged. `osmium check-refs` reported zero missing way-node references for the generated and +reloaded conflation PBFs. diff --git a/packages/osmix/test/r5/r5-oracle.init.gradle b/packages/osmix/test/r5/r5-oracle.init.gradle new file mode 100644 index 00000000..ec397c4d --- /dev/null +++ b/packages/osmix/test/r5/r5-oracle.init.gradle @@ -0,0 +1,58 @@ +gradle.beforeProject { project -> + def oracleBuildDirectory = gradle.startParameter.projectProperties['osmixOracleBuildDir'] + if (oracleBuildDirectory != null) { + project.layout.buildDirectory.set(new File(oracleBuildDirectory, project.name)) + } +} + +gradle.afterProject { project, state -> + if (project != project.rootProject || state.failure != null) return + + def requiredProperty = { String name -> + def value = project.findProperty(name) + if (value == null || value.toString().isBlank()) { + throw new GradleException("Missing required -P${name}=... property") + } + return value.toString() + } + + def oracleSourceDirectory = requiredProperty('osmixOracleSourceDir') + def oracleManifest = requiredProperty('osmixOracleManifest') + def oracleOutputDirectory = requiredProperty('osmixOracleOutputDir') + def defaultOracleDatasets = [ + 'monaco-raw', + 'monaco-empty-merge', + 'monaco-empty-merge-roundtrip', + 'monaco-synthetic-patch', + 'monaco-synthetic-patch-roundtrip' + ] + def configuredDatasets = project.findProperty('osmixOracleDatasets') + def oracleDatasets = configuredDatasets == null + ? defaultOracleDatasets + : configuredDatasets.toString().split(',') + .collect { it.trim() } + .findAll { !it.isBlank() } + if (oracleDatasets.isEmpty()) { + throw new GradleException('The -PosmixOracleDatasets list must not be empty') + } + + def oracleSourceSet = project.sourceSets.create('osmixRoutingOracle') { + java.srcDir(oracleSourceDirectory) + compileClasspath += project.sourceSets.main.output + project.configurations.runtimeClasspath + runtimeClasspath += output + compileClasspath + } + + project.tasks.named(oracleSourceSet.compileJavaTaskName) { + dependsOn(project.tasks.named('classes')) + } + + project.tasks.register('runOsmixRoutingOracle', JavaExec) { + dependsOn(project.tasks.named(oracleSourceSet.classesTaskName)) + classpath = oracleSourceSet.runtimeClasspath + mainClass = 'R5RoutingOracle' + args(oracleManifest, oracleOutputDirectory) + oracleDatasets.each { datasetId -> + args(datasetId, new File(oracleOutputDirectory, "${datasetId}.osm.pbf").absolutePath) + } + } +} diff --git a/packages/osmix/test/remote.test.ts b/packages/osmix/test/remote.test.ts index 0368b6df..d0c73f2e 100644 --- a/packages/osmix/test/remote.test.ts +++ b/packages/osmix/test/remote.test.ts @@ -10,6 +10,27 @@ const occupiedMonacoTile: [number, number, number] = [17059, 11948, 15]; // Increase timeout for worker tests const workerTestTimeout = 30_000; +function createParallelFootway( + id: string, + nodeId: number, + wayId: number, + lat: number, + name: string, +) { + const osm = new Osm({ id }); + osm.nodes.addNode({ id: nodeId, lon: 0, lat }); + osm.nodes.addNode({ id: nodeId + 1, lon: 0.001, lat }); + osm.nodes.buildIndex(); + osm.ways.addWay({ + id: wayId, + refs: [nodeId, nodeId + 1], + tags: { highway: "footway", name }, + }); + osm.buildIndexes(); + osm.buildSpatialIndexes(); + return osm; +} + class RecoveryTestRemote extends OsmixRemote { private readonly customSources = new Map<string, Uint8Array>(); @@ -312,6 +333,125 @@ describe("OsmixRemote", () => { expect(addProgressListener).toHaveBeenCalledOnce(); }); + + it("restores conflation discovery, review decisions, filters, and generated changes", async () => { + using remote = new RecoveryTestRemote(); + await remote.initializeWorkerPool(1, undefined, undefined, true); + const base = createParallelFootway("recovery-base", 1, 10, 0, "Base path"); + const patch = createParallelFootway("recovery-patch", 11, 20, 0.000004, "Imported path"); + await remote.transferIn(base); + await remote.transferIn(patch); + await remote.discoverConflation(base.id, patch.id, { + propertyKeys: ["name"], + attachNetwork: false, + }); + const wayCandidate = (await remote.getConflationPage(base.id, 0, 100)).candidates.find( + (candidate) => candidate.entityType === "way", + ); + if (!wayCandidate) throw Error("Expected a way conflation candidate"); + const bulkResult = await remote.applyConflationBulkDecision(base.id, { + action: "reject", + filter: { entityType: "way" }, + }); + expect(bulkResult.decisions).toContainEqual({ + candidateId: wayCandidate.id, + action: "reject", + }); + await expect( + remote.setConflationDecision(base.id, { + candidateId: wayCandidate.id, + action: "invalid", + } as never), + ).rejects.toThrow(`Invalid conflation decision action for ${wayCandidate.id}`); + await remote.setConflationFilter(base.id, { status: "rejected" }); + await remote.generateConflationChangeset(base.id, { + directMerge: true, + deduplicateNodes: true, + deduplicateWays: true, + }); + const unchanged = await remote.applyConflationBulkDecision(base.id, { + action: "reject", + filter: { entityType: "way" }, + }); + expect(unchanged.preview.changedCandidates).toBe(0); + + await remote.getWorker().clearConflation(base.id); + await remote.restoreForTest(); + + const restoredPage = await remote.getConflationPage(base.id, 0, 100); + expect(restoredPage.totalCandidates).toBe(1); + expect(restoredPage.candidates[0]?.decision).toEqual({ + candidateId: wayCandidate.id, + action: "reject", + }); + expect((await remote.getChangesetPage(base.id, 0, 100)).changes?.length).toBeGreaterThan(0); + }); + + it("does not replay conflation state after a loader replaces an input ID", async () => { + using remote = new RecoveryTestRemote(); + await remote.initializeWorkerPool(1, undefined, undefined, true); + const base = createParallelFootway("loader-base", 1, 10, 0, "Base path"); + const patch = createParallelFootway("loader-patch", 11, 20, 0.000004, "Imported path"); + await remote.transferIn(base); + await remote.transferIn(patch); + await remote.discoverConflation(base.id, patch.id, { + propertyKeys: ["name"], + attachNetwork: false, + }); + const candidate = (await remote.getConflationPage(base.id, 0, 100)).candidates.find( + (row) => row.entityType === "way", + )!; + await remote.setConflationDecision(base.id, { + candidateId: candidate.id, + action: "reject", + }); + + const replacement: FeatureCollection<Point> = { + type: "FeatureCollection", + features: [ + { + type: "Feature", + geometry: { type: "Point", coordinates: [1, 1] }, + properties: { name: "Replacement" }, + }, + ], + }; + await remote.fromGeoJSON(new TextEncoder().encode(JSON.stringify(replacement)), { + id: patch.id, + }); + + await expect(remote.restoreForTest()).resolves.toBeUndefined(); + await expect(remote.getConflationSummary(base.id)).rejects.toThrow( + "No active conflation session", + ); + }); + + it("invalidates sessions for both sides of an overwriting rename", async () => { + using remote = new RecoveryTestRemote(); + await remote.initializeWorkerPool(1, undefined, undefined, true); + const from = createParallelFootway("rename-from", 1, 10, 0, "From"); + const fromPatch = createParallelFootway("rename-from-patch", 11, 20, 0.000004, "Patch"); + const otherBase = createParallelFootway("rename-other-base", 21, 30, 0, "Other"); + const to = createParallelFootway("rename-to", 31, 40, 0.000004, "Destination"); + for (const osm of [from, fromPatch, otherBase, to]) await remote.transferIn(osm); + await remote.discoverConflation(from.id, fromPatch.id, { + propertyKeys: ["name"], + attachNetwork: false, + }); + await remote.discoverConflation(otherBase.id, to.id, { + propertyKeys: ["name"], + attachNetwork: false, + }); + + await remote.rename(from.id, to.id); + await expect(remote.restoreForTest()).resolves.toBeUndefined(); + await expect(remote.getConflationSummary(from.id)).rejects.toThrow( + "No active conflation session", + ); + await expect(remote.getConflationSummary(otherBase.id)).rejects.toThrow( + "No active conflation session", + ); + }); }); describe("partial state broadcasts", () => { diff --git a/packages/osmix/test/routing-after-merge.test.ts b/packages/osmix/test/routing-after-merge.test.ts new file mode 100644 index 00000000..7527a0b5 --- /dev/null +++ b/packages/osmix/test/routing-after-merge.test.ts @@ -0,0 +1,473 @@ +import { createHash } from "node:crypto"; +import { join } from "node:path"; + +import { getFixtureFile, PBFs } from "@osmix/test-utils/fixtures"; +import { beforeAll, describe, expect, it } from "vitest"; + +import { fromPbf, merge, Osm, type OsmEntity, toPbfBuffer } from "../src/index.ts"; +import { + MONACO_ROUTING_CASES, + SYNTHETIC_CONFLATION_ATTACHED_CASES, + SYNTHETIC_CONFLATION_DISCONNECTED_CASES, + SYNTHETIC_ROUTING_CASES, + type RoutingTestCase, +} from "./fixtures/routing-cases.ts"; +import { + type RoutingCaseReport, + RoutingTestHarness, + stableRoutingReport, + writeR5OracleArtifacts, + writeRoutingDiagnostics, +} from "./routing-harness.ts"; +import { + createMonacoRoutingPatch, + createMergedSyntheticRoutingOsm, + createSyntheticConflationRoutingVariants, + roundTripRoutingOsm, +} from "./synthetic-routing-fixture.ts"; + +const ALL_MERGE_STEPS = { + createIntersections: true, + deduplicateNodes: true, + deduplicateWays: true, + directMerge: true, +} as const; + +function canonicalOsmDigest(osm: Osm): string { + const hash = createHash("sha256"); + const updateEntities = (type: string, entities: Iterable<OsmEntity>): void => { + hash.update(type); + for (const entity of entities) hash.update(JSON.stringify(entity)); + }; + updateEntities("nodes", osm.nodes.sorted()); + updateEntities("ways", osm.ways.sorted()); + updateEntities("relations", osm.relations.sorted()); + return hash.digest("hex"); +} + +function topologyFingerprint(osm: Osm): string { + return JSON.stringify({ + nodes: [...osm.nodes.sorted()].map(({ id, lat, lon }) => ({ id, lat, lon })), + ways: [...osm.ways.sorted()].map(({ id, refs }) => ({ id, refs })), + relations: [...osm.relations.sorted()].map(({ id, members }) => ({ id, members })), + }); +} + +function expectReportToMatchCase(report: RoutingCaseReport, testCase: RoutingTestCase): void { + expect(report.caseId).toBe(testCase.id); + expect(report.graphPolicy).toBe(testCase.graphPolicy ?? "osmix-default"); + expect + .soft( + report.algorithmAgreement, + `${testCase.id}: Dijkstra and A* disagree (${JSON.stringify(report.algorithmCosts)})`, + ) + .toBe(true); + + if (testCase.policyLimitation) { + expect(report.from, `${testCase.id}: policy-witness origin did not resolve`).not.toBeNull(); + expect(report.to, `${testCase.id}: policy-witness destination did not resolve`).not.toBeNull(); + return; + } + + if (testCase.expect.reachable === undefined) { + throw new Error(`${testCase.id}: non-policy cases must declare reachability`); + } + expect(report.reachable).toBe(testCase.expect.reachable); + + if (!testCase.expect.reachable) { + expect(report.path).toBeNull(); + return; + } + + expect(report.from, `${testCase.id}: origin did not resolve`).not.toBeNull(); + expect(report.to, `${testCase.id}: destination did not resolve`).not.toBeNull(); + expect(report.path, `${testCase.id}: expected a route`).not.toBeNull(); + if (!report.path) return; + + const { distanceMeters, timeSeconds, wayIds } = report.path; + const distance = testCase.expect.distanceMeters; + if (distance) { + expect(distanceMeters).toBeGreaterThanOrEqual(distance.min); + expect(distanceMeters).toBeLessThanOrEqual(distance.max); + } + const time = testCase.expect.timeSeconds; + if (time) { + expect(timeSeconds).toBeGreaterThanOrEqual(time.min); + expect(timeSeconds).toBeLessThanOrEqual(time.max); + } + for (const wayId of testCase.expect.requiredWayIds ?? []) expect(wayIds).toContain(wayId); + for (const wayId of testCase.expect.forbiddenWayIds ?? []) { + expect(wayIds).not.toContain(wayId); + } +} + +function expectReportsToMatchCases( + reports: readonly RoutingCaseReport[], + testCases: readonly RoutingTestCase[], +): void { + expect(reports).toHaveLength(testCases.length); + for (const [index, testCase] of testCases.entries()) { + expectReportToMatchCase(reports[index]!, testCase); + } +} + +function stableReports( + reports: readonly RoutingCaseReport[], + testCases: readonly RoutingTestCase[], +) { + return reports.map((report, index) => { + const testCase = testCases[index]!; + if (!testCase.policyLimitation) return stableRoutingReport(report); + return { + caseId: report.caseId, + mode: report.mode, + graphPolicy: report.graphPolicy, + metric: report.metric, + graph: report.graph, + fromNodeId: report.from?.nodeId ?? null, + toNodeId: report.to?.nodeId ?? null, + algorithmAgreement: report.algorithmAgreement, + policyLimitation: report.policyLimitation, + }; + }); +} + +function stableRouteBehavior( + reports: readonly RoutingCaseReport[], + testCases: readonly RoutingTestCase[], +) { + return stableReports(reports, testCases).map(({ graph: _graph, ...report }) => report); +} + +function expectPolicyWitnesses(osm: Osm, testCases: readonly RoutingTestCase[]): void { + for (const testCase of testCases) { + const witness = testCase.policyLimitation?.witness; + if (!witness) continue; + const entity = + witness.type === "way" ? osm.ways.getById(witness.id) : osm.relations.getById(witness.id); + expect( + entity, + `${testCase.id}: policy witness ${witness.type} ${witness.id} is missing`, + ).not.toBeNull(); + expect(entity?.tags).toMatchObject(witness.tags); + } +} + +describe("routing after a Monaco merge", () => { + let raw: Osm; + let merged: Osm; + let roundTripped: Osm; + let patched: Osm; + let patchedRoundTripped: Osm; + let rawReports: RoutingCaseReport[]; + let mergedReports: RoutingCaseReport[]; + let roundTripReports: RoutingCaseReport[]; + let patchedReports: RoutingCaseReport[]; + let patchedRoundTripReports: RoutingCaseReport[]; + + beforeAll(async () => { + raw = await fromPbf(await getFixtureFile(PBFs["monaco"]!.url), { id: "monaco-raw" }); + const emptyPatch = new Osm({ id: "empty-patch" }); + emptyPatch.buildIndexes(); + emptyPatch.buildSpatialIndexes(); + merged = await merge(raw, emptyPatch, ALL_MERGE_STEPS, () => undefined); + roundTripped = await roundTripRoutingOsm(merged, "monaco-merged-roundtrip"); + const syntheticPatch = await roundTripRoutingOsm(createMonacoRoutingPatch(raw)); + patched = await merge(raw, syntheticPatch, ALL_MERGE_STEPS, () => undefined); + patchedRoundTripped = await roundTripRoutingOsm(patched, "monaco-synthetic-patch-roundtrip"); + + rawReports = new RoutingTestHarness(raw).runAll(MONACO_ROUTING_CASES); + mergedReports = new RoutingTestHarness(merged).runAll(MONACO_ROUTING_CASES); + roundTripReports = new RoutingTestHarness(roundTripped).runAll(MONACO_ROUTING_CASES); + patchedReports = new RoutingTestHarness(patched).runAll(MONACO_ROUTING_CASES); + patchedRoundTripReports = new RoutingTestHarness(patchedRoundTripped).runAll( + MONACO_ROUTING_CASES, + ); + + const diagnosticsDirectory = process.env["OSMIX_ROUTING_DIAGNOSTICS_DIR"]; + if (diagnosticsDirectory) await writeRoutingDiagnostics(rawReports, diagnosticsDirectory); + const r5OracleDirectory = process.env["OSMIX_ROUTING_ORACLE_DIR"]; + if (r5OracleDirectory) { + await writeR5OracleArtifacts( + [ + { id: "monaco-raw", osm: raw, reports: rawReports }, + { id: "monaco-empty-merge", osm: merged, reports: mergedReports }, + { + id: "monaco-empty-merge-roundtrip", + osm: roundTripped, + reports: roundTripReports, + }, + { id: "monaco-synthetic-patch", osm: patched, reports: patchedReports }, + { + id: "monaco-synthetic-patch-roundtrip", + osm: patchedRoundTripped, + reports: patchedRoundTripReports, + }, + ], + MONACO_ROUTING_CASES, + r5OracleDirectory, + ); + } + }, 30_000); + + it("keeps an empty all-steps merge as a canonical identity operation", () => { + expect({ + nodes: merged.nodes.size, + ways: merged.ways.size, + relations: merged.relations.size, + }).toEqual({ nodes: 14_286, ways: 3_346, relations: 46 }); + expect(canonicalOsmDigest(merged)).toBe(canonicalOsmDigest(raw)); + }); + + it("preserves driving and walking routes after the empty merge", () => { + expect(rawReports.find((report) => report.caseId === "monaco-short-drive")?.graph).toEqual({ + nodes: 14_286, + edges: 10_831, + weakComponents: 6, + }); + expect(rawReports.find((report) => report.caseId === "monaco-short-walk")?.graph).toEqual({ + nodes: 14_286, + edges: 25_750, + weakComponents: 27, + }); + expectReportsToMatchCases(rawReports, MONACO_ROUTING_CASES); + expectReportsToMatchCases(mergedReports, MONACO_ROUTING_CASES); + expectPolicyWitnesses(raw, MONACO_ROUTING_CASES); + expectPolicyWitnesses(merged, MONACO_ROUTING_CASES); + expect(stableReports(mergedReports, MONACO_ROUTING_CASES)).toEqual( + stableReports(rawReports, MONACO_ROUTING_CASES), + ); + }); + + it("preserves stable routing topology through PBF serialization", () => { + expect(canonicalOsmDigest(roundTripped)).toBe(canonicalOsmDigest(merged)); + expectReportsToMatchCases(roundTripReports, MONACO_ROUTING_CASES); + expectPolicyWitnesses(roundTripped, MONACO_ROUTING_CASES); + expect(stableReports(roundTripReports, MONACO_ROUTING_CASES)).toEqual( + stableReports(mergedReports, MONACO_ROUTING_CASES), + ); + }); + + it("preserves Monaco routes after a real, PBF-decoded synthetic patch", () => { + expect({ + nodes: patched.nodes.size, + ways: patched.ways.size, + relations: patched.relations.size, + }).toEqual({ nodes: 14_287, ways: 3_347, relations: 46 }); + expect(patched.nodes.getById(13_000_000_001)).toBeNull(); + expect(patched.ways.getById(3_000_000_001)?.refs).toEqual([7779445520, 13_000_000_002]); + expectReportsToMatchCases(patchedReports, MONACO_ROUTING_CASES); + expect(stableRouteBehavior(patchedReports, MONACO_ROUTING_CASES)).toEqual( + stableRouteBehavior(rawReports, MONACO_ROUTING_CASES), + ); + + expect(canonicalOsmDigest(patchedRoundTripped)).toBe(canonicalOsmDigest(patched)); + expectReportsToMatchCases(patchedRoundTripReports, MONACO_ROUTING_CASES); + expect(stableReports(patchedRoundTripReports, MONACO_ROUTING_CASES)).toEqual( + stableReports(patchedReports, MONACO_ROUTING_CASES), + ); + }); +}); + +describe("routing on a synthetic merged network", () => { + let merged: Osm; + let roundTripped: Osm; + let mergedReports: RoutingCaseReport[]; + let roundTripReports: RoutingCaseReport[]; + + beforeAll(async () => { + merged = await createMergedSyntheticRoutingOsm(); + roundTripped = await fromPbf(await toPbfBuffer(merged), { + id: "synthetic-merged-roundtrip", + }); + mergedReports = new RoutingTestHarness(merged).runAll(SYNTHETIC_ROUTING_CASES); + roundTripReports = new RoutingTestHarness(roundTripped).runAll(SYNTHETIC_ROUTING_CASES); + const r5OracleDirectory = process.env["OSMIX_ROUTING_ORACLE_DIR"]; + if (r5OracleDirectory) { + await writeR5OracleArtifacts( + [ + { id: "synthetic-merged", osm: merged, reports: mergedReports }, + { + id: "synthetic-merged-roundtrip", + osm: roundTripped, + reports: roundTripReports, + }, + ], + SYNTHETIC_ROUTING_CASES, + join(r5OracleDirectory, "synthetic"), + ); + } + }); + + it("keeps mode-specific paths, one-way direction, and grade separation correct", () => { + expect(merged.nodes.getById(30)).toBeNull(); + expect(merged.ways.getById(101)?.refs).toEqual([3, 4]); + expect(merged.relations.getById(200)?.members).toEqual([ + { type: "way", ref: 100, role: "from" }, + { type: "node", ref: 3, role: "via" }, + { type: "way", ref: 101, role: "to" }, + ]); + expect(merged.nodes.getById(21)).not.toBeNull(); + expect(merged.nodes.getById(23)).not.toBeNull(); + expect(merged.ways.getById(150)?.tags).toMatchObject({ + foot: "designated", + highway: "residential", + motor_vehicle: "no", + }); + const way130 = merged.ways.getById(130)!; + const way131 = merged.ways.getById(131)!; + expect(way130.refs.map((ref) => merged.nodes.getNodeLonLat({ id: ref })?.[0])).toEqual([ + 0, 0.002, 0.004, + ]); + expect(way130.refs.filter((ref) => way131.refs.includes(ref))).toHaveLength(1); + + const reverseWay = merged.ways.getById(140)!; + expect(reverseWay.refs.map((ref) => merged.nodes.getNodeLonLat({ id: ref })?.[0])).toEqual([ + 0.004, 0.003, 0.001, 0, + ]); + expectReportsToMatchCases(mergedReports, SYNTHETIC_ROUTING_CASES); + }); + + it("keeps synthetic route behavior stable through a PBF round trip", () => { + expectReportsToMatchCases(roundTripReports, SYNTHETIC_ROUTING_CASES); + expect(stableReports(roundTripReports, SYNTHETIC_ROUTING_CASES)).toEqual( + stableReports(mergedReports, SYNTHETIC_ROUTING_CASES), + ); + }); +}); + +describe("routing after explicit fuzzy conflation", () => { + let ordinary: Osm; + let propertyTransfer: Osm; + let networkAttachment: Osm; + let propertyRoundTrip: Osm; + let attachmentRoundTrip: Osm; + let ordinaryReports: RoutingCaseReport[]; + let propertyReports: RoutingCaseReport[]; + let attachmentReports: RoutingCaseReport[]; + let propertyRoundTripReports: RoutingCaseReport[]; + let attachmentRoundTripReports: RoutingCaseReport[]; + + beforeAll(async () => { + ({ ordinary, propertyTransfer, networkAttachment } = + await createSyntheticConflationRoutingVariants()); + [propertyRoundTrip, attachmentRoundTrip] = await Promise.all([ + roundTripRoutingOsm(propertyTransfer, "synthetic-conflation-property-roundtrip"), + roundTripRoutingOsm(networkAttachment, "synthetic-conflation-attachment-roundtrip"), + ]); + + ordinaryReports = new RoutingTestHarness(ordinary).runAll( + SYNTHETIC_CONFLATION_DISCONNECTED_CASES, + ); + propertyReports = new RoutingTestHarness(propertyTransfer).runAll( + SYNTHETIC_CONFLATION_DISCONNECTED_CASES, + ); + attachmentReports = new RoutingTestHarness(networkAttachment).runAll( + SYNTHETIC_CONFLATION_ATTACHED_CASES, + ); + propertyRoundTripReports = new RoutingTestHarness(propertyRoundTrip).runAll( + SYNTHETIC_CONFLATION_DISCONNECTED_CASES, + ); + attachmentRoundTripReports = new RoutingTestHarness(attachmentRoundTrip).runAll( + SYNTHETIC_CONFLATION_ATTACHED_CASES, + ); + + const r5OracleDirectory = process.env["OSMIX_ROUTING_ORACLE_DIR"]; + if (r5OracleDirectory) { + await Promise.all([ + writeR5OracleArtifacts( + [ + { id: "synthetic-conflation-ordinary", osm: ordinary, reports: ordinaryReports }, + { + id: "synthetic-conflation-property", + osm: propertyTransfer, + reports: propertyReports, + }, + { + id: "synthetic-conflation-property-roundtrip", + osm: propertyRoundTrip, + reports: propertyRoundTripReports, + }, + ], + SYNTHETIC_CONFLATION_DISCONNECTED_CASES, + join(r5OracleDirectory, "conflation-property"), + ), + writeR5OracleArtifacts( + [ + { + id: "synthetic-conflation-attachment", + osm: networkAttachment, + reports: attachmentReports, + }, + { + id: "synthetic-conflation-attachment-roundtrip", + osm: attachmentRoundTrip, + reports: attachmentRoundTripReports, + }, + ], + SYNTHETIC_CONFLATION_ATTACHED_CASES, + join(r5OracleDirectory, "conflation-attachment"), + ), + ]); + } + }); + + it("leaves property-only topology and routing unchanged", () => { + expect(propertyTransfer.nodes.getById(802)?.tags?.["name"]).toBe("Imported endpoint"); + expect(propertyTransfer.nodes.getById(901)?.tags).toMatchObject({ + name: "Imported endpoint", + source: "synthetic survey", + }); + expect(propertyTransfer.ways.getById(810)?.refs.at(0)).toBe(801); + expect(propertyTransfer.ways.getById(849)?.refs.at(-1)).toBe(802); + expect(propertyTransfer.ways.getById(910)?.refs.at(0)).toBe(901); + expect(propertyTransfer.ways.getById(949)?.refs.at(-1)).toBe(902); + expect(topologyFingerprint(propertyTransfer)).toBe(topologyFingerprint(ordinary)); + expectReportsToMatchCases(ordinaryReports, SYNTHETIC_CONFLATION_DISCONNECTED_CASES); + expectReportsToMatchCases(propertyReports, SYNTHETIC_CONFLATION_DISCONNECTED_CASES); + expect(stableReports(propertyReports, SYNTHETIC_CONFLATION_DISCONNECTED_CASES)).toEqual( + stableReports(ordinaryReports, SYNTHETIC_CONFLATION_DISCONNECTED_CASES), + ); + }); + + it("attaches the WALK network while preserving the CAR graph", () => { + expect(networkAttachment.nodes.getById(802)).toMatchObject({ lon: 0, lat: 0 }); + expect(networkAttachment.ways.getById(810)?.refs.at(0)).toBe(801); + expect(networkAttachment.ways.getById(849)?.refs.at(-1)).toBe(802); + expect(networkAttachment.ways.getById(910)?.refs.at(0)).toBe(802); + expect(networkAttachment.ways.getById(949)?.refs.at(-1)).toBe(902); + expect(networkAttachment.nodes.getById(901)).not.toBeNull(); + expectReportsToMatchCases(attachmentReports, SYNTHETIC_CONFLATION_ATTACHED_CASES); + + const ordinaryCar = ordinaryReports.find( + (report) => report.caseId === "synthetic-conflation-car", + ); + const attachedCar = attachmentReports.find( + (report) => report.caseId === "synthetic-conflation-car", + ); + expect(attachedCar?.graph).toEqual(ordinaryCar?.graph); + + const ordinaryWalk = ordinaryReports.find( + (report) => report.caseId === "synthetic-conflation-walk", + ); + const attachedWalk = attachmentReports.find( + (report) => report.caseId === "synthetic-conflation-walk", + ); + expect(attachedWalk?.graph.edges).toBe(ordinaryWalk?.graph.edges); + expect(attachedWalk?.graph.weakComponents).toBe(1); + expect(ordinaryWalk?.graph.weakComponents).toBe(2); + }); + + it("preserves both conflation variants through PBF serialization", () => { + expect(topologyFingerprint(propertyRoundTrip)).toBe(topologyFingerprint(propertyTransfer)); + expect(topologyFingerprint(attachmentRoundTrip)).toBe(topologyFingerprint(networkAttachment)); + expectReportsToMatchCases(propertyRoundTripReports, SYNTHETIC_CONFLATION_DISCONNECTED_CASES); + expectReportsToMatchCases(attachmentRoundTripReports, SYNTHETIC_CONFLATION_ATTACHED_CASES); + expect( + stableReports(propertyRoundTripReports, SYNTHETIC_CONFLATION_DISCONNECTED_CASES), + ).toEqual(stableReports(propertyReports, SYNTHETIC_CONFLATION_DISCONNECTED_CASES)); + expect(stableReports(attachmentRoundTripReports, SYNTHETIC_CONFLATION_ATTACHED_CASES)).toEqual( + stableReports(attachmentReports, SYNTHETIC_CONFLATION_ATTACHED_CASES), + ); + }); +}); diff --git a/packages/osmix/test/routing-harness.ts b/packages/osmix/test/routing-harness.ts new file mode 100644 index 00000000..9e65e313 --- /dev/null +++ b/packages/osmix/test/routing-harness.ts @@ -0,0 +1,494 @@ +import { mkdir, writeFile } from "node:fs/promises"; +import { join } from "node:path"; + +import type { FeatureCollection, LineString } from "geojson"; + +import { + defaultHighwayFilter, + type HighwayFilter, + type LonLat, + type Osm, + type OsmTags, + Router, + RoutingGraph, + toPbfBuffer, +} from "../src/index.ts"; +import type { + RoutingTestCase, + RoutingTestEndpoint, + RoutingTestMode, + RoutingPolicyLimitation, +} from "./fixtures/routing-cases.ts"; + +const WALK_SPEEDS = { + bridleway: 5, + cycleway: 5, + footway: 5, + living_street: 5, + path: 5, + pedestrian: 5, + primary: 5, + residential: 5, + secondary: 5, + service: 5, + steps: 2, + tertiary: 5, + track: 5, + unclassified: 5, +}; + +const WALKABLE_HIGHWAYS = new Set(Object.keys(WALK_SPEEDS)); +const POSITIVE_ACCESS_VALUES = new Set(["designated", "destination", "permissive", "yes"]); +const NEGATIVE_ACCESS_VALUES = new Set(["no", "private"]); + +/** + * Test-only subset of R5's motor-vehicle access policy. This deliberately does not model + * conditional access, barriers, or every OSM vehicle class and is not a public routing profile. + */ +export const routingTestCarAccessFilter: HighwayFilter = (tags?: OsmTags): boolean => { + if (!defaultHighwayFilter(tags)) return false; + const access = + tags?.["motorcar"] ?? tags?.["motor_vehicle"] ?? tags?.["vehicle"] ?? tags?.["access"]; + return !NEGATIVE_ACCESS_VALUES.has(String(access)); +}; + +/** Test-only pedestrian policy. R5 remains the authority for production access semantics. */ +export const routingTestWalkFilter: HighwayFilter = (tags?: OsmTags): boolean => { + const highway = tags?.["highway"]; + if (!highway || !WALKABLE_HIGHWAYS.has(String(highway))) return false; + if (tags["foot"] === "no" || tags["foot"] === "private") return false; + + const access = tags["access"]; + if (access === "no" || access === "private") { + return POSITIVE_ACCESS_VALUES.has(String(tags["foot"])); + } + + return true; +}; + +export interface RoutingGraphReport { + nodes: number; + edges: number; + weakComponents: number; +} + +export interface RoutingEndpointReport { + nodeId: number; + coordinates: LonLat; + snapDistanceMeters: number; +} + +export interface RoutingPathReport { + nodeIds: number[]; + wayIds: number[]; + highways: string[]; + coordinates: LonLat[]; + distanceMeters: number; + timeSeconds: number; + optimizedCost: number; +} + +export interface RoutingCaseReport { + caseId: string; + mode: RoutingTestMode; + graphPolicy: "access-aware" | "osmix-default"; + metric: RoutingTestCase["metric"]; + graph: RoutingGraphReport; + from: RoutingEndpointReport | null; + to: RoutingEndpointReport | null; + reachable: boolean; + algorithmAgreement: boolean; + algorithmCosts: { astar: number | null; dijkstra: number | null }; + policyLimitation?: RoutingPolicyLimitation; + path: RoutingPathReport | null; +} + +interface RoutingContext { + graph: RoutingGraph; + report: RoutingGraphReport; + router: Router; +} + +type RoutingContextKey = RoutingTestMode | "car-access-aware"; + +function countWeakComponents(graph: RoutingGraph): number { + const parents = Uint32Array.from({ length: graph.size }, (_, index) => index); + const routable = new Uint8Array(graph.size); + + const find = (value: number): number => { + let root = value; + while (parents[root] !== root) root = parents[root]!; + let cursor = value; + while (parents[cursor] !== cursor) { + const next = parents[cursor]!; + parents[cursor] = root; + cursor = next; + } + return root; + }; + + const union = (left: number, right: number): void => { + const leftRoot = find(left); + const rightRoot = find(right); + if (leftRoot !== rightRoot) parents[rightRoot] = leftRoot; + }; + + for (let nodeIndex = 0; nodeIndex < graph.size; nodeIndex++) { + if (!graph.isRoutable(nodeIndex)) continue; + routable[nodeIndex] = 1; + for (const edge of graph.getEdges(nodeIndex)) union(nodeIndex, edge.targetNodeIndex); + } + + const roots = new Set<number>(); + for (let nodeIndex = 0; nodeIndex < graph.size; nodeIndex++) { + if (routable[nodeIndex]) roots.add(find(nodeIndex)); + } + return roots.size; +} + +function buildContext(osm: Osm, mode: RoutingContextKey): RoutingContext { + let graph: RoutingGraph; + if (mode === "walk") graph = new RoutingGraph(osm, routingTestWalkFilter, WALK_SPEEDS); + else if (mode === "car-access-aware") graph = new RoutingGraph(osm, routingTestCarAccessFilter); + else graph = new RoutingGraph(osm, defaultHighwayFilter); + return { + graph, + router: new Router(osm, graph), + report: { + nodes: graph.size, + edges: graph.edges, + weakComponents: countWeakComponents(graph), + }, + }; +} + +function resolveEndpoint( + osm: Osm, + graph: RoutingGraph, + endpoint: RoutingTestEndpoint, +): RoutingEndpointReport | null { + if ("nodeId" in endpoint) { + const nodeIndex = osm.nodes.ids.getIndexFromId(endpoint.nodeId); + if (nodeIndex === -1 || !graph.isRoutable(nodeIndex)) return null; + return { + nodeId: endpoint.nodeId, + coordinates: osm.nodes.getNodeLonLat({ index: nodeIndex }), + snapDistanceMeters: 0, + }; + } + + const nearest = graph.findNearestRoutableNode( + osm, + endpoint.coordinates, + endpoint.maxSnapDistanceMeters, + ); + if (!nearest) return null; + return { + nodeId: osm.nodes.ids.at(nearest.nodeIndex), + coordinates: nearest.coordinates, + snapDistanceMeters: nearest.distance, + }; +} + +function uniqueConsecutive<T>(values: readonly T[]): T[] { + return values.filter((value, index) => index === 0 || value !== values[index - 1]); +} + +function routeCase( + osm: Osm, + context: RoutingContext, + testCase: RoutingTestCase, +): RoutingCaseReport { + const from = resolveEndpoint(osm, context.graph, testCase.from); + const to = resolveEndpoint(osm, context.graph, testCase.to); + if (!from || !to) { + return { + caseId: testCase.id, + mode: testCase.mode, + graphPolicy: testCase.graphPolicy ?? "osmix-default", + metric: testCase.metric, + graph: context.report, + from, + to, + reachable: false, + algorithmAgreement: true, + algorithmCosts: { astar: null, dijkstra: null }, + policyLimitation: testCase.policyLimitation, + path: null, + }; + } + + const fromIndex = osm.nodes.ids.getIndexFromId(from.nodeId); + const toIndex = osm.nodes.ids.getIndexFromId(to.nodeId); + const routeOptions = { metric: testCase.metric } as const; + const dijkstra = context.router.route(fromIndex, toIndex, { + ...routeOptions, + algorithm: "dijkstra", + }); + const astar = context.router.route(fromIndex, toIndex, { + ...routeOptions, + algorithm: "astar", + }); + const bothReachable = dijkstra !== null && astar !== null; + const bothUnreachable = dijkstra === null && astar === null; + const algorithmAgreement = + bothUnreachable || + (bothReachable && + Math.abs(dijkstra.at(-1)!.cost - astar.at(-1)!.cost) <= + Math.max(0.001, Math.abs(dijkstra.at(-1)!.cost) * 1e-6)); + + if (!dijkstra) { + return { + caseId: testCase.id, + mode: testCase.mode, + graphPolicy: testCase.graphPolicy ?? "osmix-default", + metric: testCase.metric, + graph: context.report, + from, + to, + reachable: false, + algorithmAgreement, + algorithmCosts: { astar: astar?.at(-1)?.cost ?? null, dijkstra: null }, + policyLimitation: testCase.policyLimitation, + path: null, + }; + } + + const stats = context.router.getRouteStatistics(dijkstra); + const wayIndexes = uniqueConsecutive( + dijkstra.flatMap((segment) => (segment.wayIndex === undefined ? [] : [segment.wayIndex])), + ); + return { + caseId: testCase.id, + mode: testCase.mode, + graphPolicy: testCase.graphPolicy ?? "osmix-default", + metric: testCase.metric, + graph: context.report, + from, + to, + reachable: true, + algorithmAgreement, + algorithmCosts: { + astar: astar?.at(-1)?.cost ?? null, + dijkstra: dijkstra.at(-1)!.cost, + }, + policyLimitation: testCase.policyLimitation, + path: { + nodeIds: dijkstra.map((segment) => osm.nodes.ids.at(segment.nodeIndex)), + wayIds: wayIndexes.map((wayIndex) => osm.ways.ids.at(wayIndex)), + highways: wayIndexes.map((wayIndex) => + String(osm.ways.tags.getTags(wayIndex)?.["highway"] ?? ""), + ), + coordinates: dijkstra.map((segment) => osm.nodes.getNodeLonLat({ index: segment.nodeIndex })), + distanceMeters: stats.distance, + timeSeconds: stats.time, + optimizedCost: dijkstra.at(-1)!.cost, + }, + }; +} + +export class RoutingTestHarness { + readonly osm: Osm; + readonly contexts: Record<RoutingContextKey, RoutingContext>; + + constructor(osm: Osm) { + this.osm = osm; + this.contexts = { + car: buildContext(osm, "car"), + "car-access-aware": buildContext(osm, "car-access-aware"), + walk: buildContext(osm, "walk"), + }; + } + + run(testCase: RoutingTestCase): RoutingCaseReport { + const contextKey = + testCase.mode === "car" && testCase.graphPolicy === "access-aware" + ? "car-access-aware" + : testCase.mode; + return routeCase(this.osm, this.contexts[contextKey], testCase); + } + + runAll(testCases: readonly RoutingTestCase[]): RoutingCaseReport[] { + return testCases.map((testCase) => this.run(testCase)); + } +} + +export function stableRoutingReport(report: RoutingCaseReport) { + return { + caseId: report.caseId, + mode: report.mode, + graphPolicy: report.graphPolicy, + metric: report.metric, + graph: report.graph, + fromNodeId: report.from?.nodeId ?? null, + toNodeId: report.to?.nodeId ?? null, + reachable: report.reachable, + algorithmAgreement: report.algorithmAgreement, + algorithmCosts: { + astar: + report.algorithmCosts.astar === null + ? null + : Number(report.algorithmCosts.astar.toFixed(3)), + dijkstra: + report.algorithmCosts.dijkstra === null + ? null + : Number(report.algorithmCosts.dijkstra.toFixed(3)), + }, + policyLimitation: report.policyLimitation, + path: report.path + ? { + nodeIds: report.path.nodeIds, + wayIds: report.path.wayIds, + highways: report.path.highways, + distanceMeters: Number(report.path.distanceMeters.toFixed(3)), + timeSeconds: Number(report.path.timeSeconds.toFixed(3)), + optimizedCost: Number(report.path.optimizedCost.toFixed(3)), + } + : null, + }; +} + +export function routingReportsToGeoJson( + reports: readonly RoutingCaseReport[], +): FeatureCollection<LineString> { + return { + type: "FeatureCollection", + features: reports.flatMap((report) => + report.path + ? [ + { + type: "Feature" as const, + properties: { + caseId: report.caseId, + mode: report.mode, + distanceMeters: report.path.distanceMeters, + timeSeconds: report.path.timeSeconds, + wayIds: report.path.wayIds.join(","), + }, + geometry: { + type: "LineString" as const, + coordinates: report.path.coordinates, + }, + }, + ] + : [], + ), + }; +} + +/** Write diagnostics only when explicitly called by a developer or debugging script. */ +export async function writeRoutingDiagnostics( + reports: readonly RoutingCaseReport[], + directory: string, +): Promise<void> { + await mkdir(directory, { recursive: true }); + await Promise.all([ + writeFile( + join(directory, "routing-report.json"), + `${JSON.stringify(reports.map(stableRoutingReport), null, 2)}\n`, + ), + writeFile( + join(directory, "routing-routes.geojson"), + `${JSON.stringify(routingReportsToGeoJson(reports), null, 2)}\n`, + ), + ]); +} + +export interface R5OracleDataset { + id: string; + osm: Osm; + reports: readonly RoutingCaseReport[]; +} + +function tsvCell(value: boolean | number | string | undefined): string { + return String(value ?? "") + .replaceAll("\t", " ") + .replaceAll("\r", " ") + .replaceAll("\n", " "); +} + +/** + * Export exact PBF inputs, snapped coordinates, and Osmix diagnostics for an opt-in local R5 run. + * This is intentionally called only behind an environment variable in the regression test. + */ +export async function writeR5OracleArtifacts( + datasets: readonly R5OracleDataset[], + testCases: readonly RoutingTestCase[], + directory: string, +): Promise<void> { + await mkdir(directory, { recursive: true }); + + const referenceReports = new Map(datasets[0]?.reports.map((report) => [report.caseId, report])); + const routeRows = testCases.flatMap((testCase) => { + const report = referenceReports.get(testCase.id); + if (!report?.from || !report.to) return []; + return [ + [ + testCase.id, + testCase.mode.toUpperCase(), + report.from.coordinates[0], + report.from.coordinates[1], + report.to.coordinates[0], + report.to.coordinates[1], + "nodeId" in testCase.from ? "osm-node" : "coordinate", + "nodeId" in testCase.from ? testCase.from.nodeId : undefined, + "nodeId" in testCase.to ? "osm-node" : "coordinate", + "nodeId" in testCase.to ? testCase.to.nodeId : undefined, + testCase.policyLimitation?.kind ?? "absolute-golden", + testCase.expect.reachable, + testCase.expect.distanceMeters?.min, + testCase.expect.distanceMeters?.max, + testCase.policyLimitation?.r5Expectation ?? "", + ].map(tsvCell), + ]; + }); + const manifestRows = [ + [ + "case_id", + "mode", + "from_lon", + "from_lat", + "to_lon", + "to_lat", + "from_endpoint_kind", + "from_osm_node_id", + "to_endpoint_kind", + "to_osm_node_id", + "expectation_kind", + "expected_reachable", + "expected_distance_min_m", + "expected_distance_max_m", + "r5_expectation", + ], + ...routeRows, + ]; + + await Promise.all([ + writeFile( + join(directory, "routing-cases.tsv"), + `${manifestRows.map((row) => row.join("\t")).join("\n")}\n`, + ), + writeFile( + join(directory, "oracle-matrix.json"), + `${JSON.stringify( + { + schemaVersion: 1, + note: "Local diagnostic output; it is not a checked-in golden and is never auto-updated.", + datasets: datasets.map((dataset) => ({ + id: dataset.id, + pbf: `${dataset.id}.osm.pbf`, + osmix: dataset.reports.map(stableRoutingReport), + })), + }, + null, + 2, + )}\n`, + ), + ...datasets.flatMap((dataset) => [ + toPbfBuffer(dataset.osm).then((pbf) => + writeFile(join(directory, `${dataset.id}.osm.pbf`), pbf), + ), + writeRoutingDiagnostics(dataset.reports, join(directory, dataset.id)), + ]), + ]); +} diff --git a/packages/osmix/test/synthetic-routing-fixture.ts b/packages/osmix/test/synthetic-routing-fixture.ts new file mode 100644 index 00000000..054da126 --- /dev/null +++ b/packages/osmix/test/synthetic-routing-fixture.ts @@ -0,0 +1,274 @@ +import { fromPbf, merge, Osm, toPbfBuffer } from "../src/index.ts"; + +function complete(osm: Osm): Osm { + osm.buildIndexes(); + osm.buildSpatialIndexes(); + return osm; +} + +export function createSyntheticRoutingBase(): Osm { + const osm = new Osm({ id: "synthetic-routing-base" }); + + for (const node of [ + { id: 1, lon: 0, lat: 0 }, + { id: 2, lon: 0, lat: 0.001 }, + { id: 3, lon: 0.003, lat: 0.001 }, + { id: 10, lon: 0, lat: 0.01 }, + { id: 11, lon: 0.001, lat: 0.01 }, + { id: 12, lon: 0.002, lat: 0.01 }, + { id: 13, lon: 0, lat: 0.012 }, + { id: 14, lon: 0.001, lat: 0.012 }, + { id: 15, lon: 0.002, lat: 0.012 }, + { id: 20, lon: 0.004, lat: 0 }, + { id: 21, lon: 0.005, lat: 0 }, + { id: 22, lon: 0.006, lat: 0 }, + { id: 40, lon: 0, lat: 0.02 }, + { id: 41, lon: 0.004, lat: 0.02 }, + { id: 60, lon: 0, lat: 0.03 }, + { id: 61, lon: 0.004, lat: 0.03 }, + { id: 70, lon: 0, lat: 0.04 }, + { id: 72, lon: 0.002, lat: 0.04 }, + { id: 73, lon: 0.001, lat: 0.041 }, + ]) { + osm.nodes.addNode(node); + } + + osm.ways.addWay({ + id: 100, + refs: [1, 2, 3], + tags: { highway: "residential", name: "Base Road" }, + }); + osm.ways.addWay({ + id: 110, + refs: [10, 11, 12], + tags: { highway: "residential", name: "One Way", oneway: "yes" }, + }); + osm.ways.addWay({ + id: 111, + refs: [13, 14, 15], + tags: { highway: "residential", name: "Reverse One Way", oneway: "-1" }, + }); + osm.ways.addWay({ + id: 120, + refs: [20, 21, 22], + tags: { highway: "residential", layer: "0", name: "Surface Road" }, + }); + osm.ways.addWay({ + id: 130, + refs: [40, 41], + tags: { highway: "residential", name: "Crossing Base Road" }, + }); + osm.ways.addWay({ + id: 140, + refs: [61, 60], + tags: { highway: "residential", name: "Reverse Base Road" }, + }); + osm.ways.addWay({ + id: 151, + refs: [70, 73, 72], + tags: { highway: "residential", name: "Public Detour" }, + }); + + return complete(osm); +} + +export function createSyntheticRoutingPatch(): Osm { + const osm = new Osm({ id: "synthetic-routing-patch" }); + + for (const node of [ + { id: 1, lon: 0, lat: 0 }, + // A different ID at the exact base-road endpoint exercises safe cross-input reconciliation. + { id: 30, lon: 0.003, lat: 0.001 }, + { id: 4, lon: 0.003, lat: 0 }, + { id: 23, lon: 0.005, lat: 0.000003 }, + { id: 24, lon: 0.005, lat: -0.001 }, + { id: 25, lon: 0.005, lat: 0.001 }, + { id: 50, lon: 0.002, lat: 0.019 }, + { id: 51, lon: 0.002, lat: 0.021 }, + { id: 62, lon: 0.001, lat: 0.029 }, + { id: 63, lon: 0.001, lat: 0.031 }, + { id: 64, lon: 0.003, lat: 0.029 }, + { id: 65, lon: 0.003, lat: 0.031 }, + { id: 70, lon: 0, lat: 0.04 }, + { id: 71, lon: 0.001, lat: 0.04 }, + { id: 72, lon: 0.002, lat: 0.04 }, + ]) { + osm.nodes.addNode(node); + } + + osm.ways.addWay({ + id: 101, + refs: [30, 4], + tags: { highway: "residential", name: "Patch Extension" }, + }); + osm.ways.addWay({ + id: 102, + refs: [1, 4], + tags: { foot: "designated", highway: "footway", motor_vehicle: "no" }, + }); + osm.ways.addWay({ + id: 121, + refs: [24, 23, 25], + tags: { highway: "primary", layer: "-1", name: "Tunnel Road", tunnel: "yes" }, + }); + osm.ways.addWay({ + id: 131, + refs: [50, 51], + tags: { highway: "residential", name: "Same Grade Crossing" }, + }); + osm.ways.addWay({ + id: 141, + refs: [62, 63], + tags: { highway: "residential", name: "First Reverse Crossing" }, + }); + osm.ways.addWay({ + id: 142, + refs: [64, 65], + tags: { highway: "residential", name: "Second Reverse Crossing" }, + }); + osm.ways.addWay({ + id: 150, + refs: [70, 71, 72], + tags: { + foot: "designated", + highway: "residential", + motor_vehicle: "no", + name: "Foot-Designated Access Road", + }, + }); + osm.relations.addRelation({ + id: 200, + members: [ + { type: "way", ref: 100, role: "from" }, + { type: "node", ref: 30, role: "via" }, + { type: "way", ref: 101, role: "to" }, + ], + tags: { restriction: "no_left_turn", type: "restriction" }, + }); + + return complete(osm); +} + +/** A PBF-roundtrippable dead-end extension on Monaco's eastern boundary. */ +export function createMonacoRoutingPatch(base: Osm): Osm { + const sharedBaseNode = base.nodes.getById(7779445520); + if (!sharedBaseNode) throw Error("Monaco routing patch anchor node is missing"); + + const osm = new Osm({ id: "monaco-synthetic-routing-patch" }); + osm.nodes.addNode({ ...sharedBaseNode, id: 13_000_000_001 }); + osm.nodes.addNode({ + id: 13_000_000_002, + lat: sharedBaseNode.lat, + lon: sharedBaseNode.lon + 0.001, + }); + osm.ways.addWay({ + id: 3_000_000_001, + refs: [13_000_000_001, 13_000_000_002], + tags: { highway: "service", name: "Synthetic Monaco boundary extension" }, + }); + return complete(osm); +} + +export async function roundTripRoutingOsm(osm: Osm, id = `${osm.id}-roundtrip`): Promise<Osm> { + return fromPbf(await toPbfBuffer(osm), { id }); +} + +/** Merge PBF-decoded synthetic inputs exactly as the Merge app's all-steps workflow does. */ +export async function createMergedSyntheticRoutingOsm(): Promise<Osm> { + const [base, patch] = await Promise.all([ + roundTripRoutingOsm(createSyntheticRoutingBase()), + roundTripRoutingOsm(createSyntheticRoutingPatch()), + ]); + return merge(base, patch, { + createIntersections: true, + deduplicateNodes: true, + deduplicateWays: true, + directMerge: true, + }); +} + +/** + * A pair of offset pedestrian networks used to verify explicit fuzzy attachment. The imported + * endpoint is about 0.56 meters from the base endpoint: close enough for the recommended 1-meter + * conflation radius, but still disconnected in an ordinary direct merge. + */ +export function createSyntheticConflationRoutingInputs(): { base: Osm; patch: Osm } { + const base = new Osm({ id: "synthetic-conflation-base" }); + const baseRefs = Array.from({ length: 41 }, (_, step) => { + const id = step === 0 ? 801 : step === 40 ? 802 : 8_000 + step; + base.nodes.addNode({ + id, + lon: -0.001 + (step * 0.001) / 40, + lat: 0, + ...(step === 40 ? { tags: { name: "Base endpoint" } } : {}), + }); + return id; + }); + for (let step = 0; step < baseRefs.length - 1; step++) { + base.ways.addWay({ + id: 810 + step, + refs: [baseRefs[step]!, baseRefs[step + 1]!], + tags: { highway: "footway", name: `Base sidewalk ${step + 1}` }, + }); + } + + const patch = new Osm({ id: "synthetic-conflation-patch" }); + const patchRefs = Array.from({ length: 41 }, (_, step) => { + const id = step === 0 ? 901 : step === 40 ? 902 : 9_000 + step; + patch.nodes.addNode({ + id, + lon: 0.000005 + (step * 0.000995) / 40, + lat: 0, + ...(step === 0 ? { tags: { name: "Imported endpoint", source: "synthetic survey" } } : {}), + }); + return id; + }); + for (let step = 0; step < patchRefs.length - 1; step++) { + patch.ways.addWay({ + id: 910 + step, + refs: [patchRefs[step]!, patchRefs[step + 1]!], + tags: { highway: "footway", name: `Imported sidewalk ${step + 1}` }, + }); + } + + return { base: complete(base), patch: complete(patch) }; +} + +/** Build ordinary, property-only, and network-attached results from the same PBF-decoded inputs. */ +export async function createSyntheticConflationRoutingVariants(): Promise<{ + ordinary: Osm; + propertyTransfer: Osm; + networkAttachment: Osm; +}> { + const inputs = createSyntheticConflationRoutingInputs(); + const [base, patch] = await Promise.all([ + roundTripRoutingOsm(inputs.base, "synthetic-conflation-base-pbf"), + roundTripRoutingOsm(inputs.patch, "synthetic-conflation-patch-pbf"), + ]); + const ordinary = await merge(base, patch, { directMerge: true }, () => undefined); + const propertyTransfer = await merge( + base, + patch, + { + directMerge: true, + conflation: { + propertyKeys: ["name"], + attachNetwork: false, + }, + }, + () => undefined, + ); + const networkAttachment = await merge( + base, + patch, + { + directMerge: true, + conflation: { + propertyKeys: [], + attachNetwork: true, + }, + }, + () => undefined, + ); + return { ordinary, propertyTransfer, networkAttachment }; +} diff --git a/packages/osmix/test/worker-registry.test.ts b/packages/osmix/test/worker-registry.test.ts index af726656..72731a6b 100644 --- a/packages/osmix/test/worker-registry.test.ts +++ b/packages/osmix/test/worker-registry.test.ts @@ -1,6 +1,6 @@ import { Osm } from "@osmix/core"; import { createMockBaseOsm, createMockPatchOsm } from "@osmix/core/mocks"; -import { describe, expect, it } from "vitest"; +import { describe, expect, it, vi } from "vitest"; import { OsmixWorker } from "../src/worker"; @@ -18,6 +18,55 @@ function withId(osm: Osm, id: string) { return new Osm({ ...osm.transferables(), id }); } +function createParallelFootway( + id: string, + nodeId: number, + wayId: number, + lat: number, + name: string, +) { + const osm = new Osm({ id }); + osm.nodes.addNode({ id: nodeId, lon: 0, lat }); + osm.nodes.addNode({ id: nodeId + 1, lon: 0.001, lat }); + osm.nodes.buildIndex(); + osm.ways.addWay({ + id: wayId, + refs: [nodeId, nodeId + 1], + tags: { highway: "footway", name }, + }); + osm.buildIndexes(); + osm.buildSpatialIndexes(); + return osm; +} + +function createMixedParallelNetwork( + id: string, + nodeId: number, + wayId: number, + offset: number, + namePrefix: string, +) { + const osm = new Osm({ id }); + osm.nodes.addNode({ id: nodeId, lon: 0, lat: offset }); + osm.nodes.addNode({ id: nodeId + 1, lon: 0.001, lat: offset }); + osm.nodes.addNode({ id: nodeId + 2, lon: 0, lat: 0.01 + offset }); + osm.nodes.addNode({ id: nodeId + 3, lon: 0.001, lat: 0.01 + offset }); + osm.nodes.buildIndex(); + osm.ways.addWay({ + id: wayId, + refs: [nodeId, nodeId + 1], + tags: { highway: "footway", name: `${namePrefix} path` }, + }); + osm.ways.addWay({ + id: wayId + 1, + refs: [nodeId + 2, nodeId + 3], + tags: { highway: "residential", name: `${namePrefix} street` }, + }); + osm.buildIndexes(); + osm.buildSpatialIndexes(); + return osm; +} + describe("OsmixWorker registries", () => { it("stores reserved and ordinary IDs without prototype collisions", () => { const worker = new TestWorker(); @@ -75,4 +124,313 @@ describe("OsmixWorker registries", () => { expect(worker.has("constructor")).toBe(true); expect(worker.has("toString")).toBe(true); }); + + it("validates initial decisions and filters explicitly for unmatched targets", () => { + const worker = new TestWorker(); + const base = withId(createMockBaseOsm(), "conflation-base"); + const patch = withId(createMockPatchOsm(), "conflation-patch"); + base.buildSpatialIndexes(); + patch.buildSpatialIndexes(); + worker.setOsm(base.id, base); + worker.setOsm(patch.id, patch); + + expect(() => + worker.discoverConflation(base.id, patch.id, { + propertyKeys: ["crossing"], + attachNetwork: true, + decisions: [{ candidateId: "node:missing->none", action: "reject" }], + }), + ).toThrow("Unknown conflation candidate: node:missing->none"); + + worker.discoverConflation(base.id, patch.id, { + propertyKeys: ["crossing"], + attachNetwork: true, + }); + worker.setConflationFilter(base.id, { targetId: null }); + const page = worker.getConflationPage(base.id, 0, 100); + + expect(page.totalCandidates).toBeGreaterThan(0); + expect(page.candidates.every((candidate) => candidate.targetId === null)).toBe(true); + + const candidateId = page.candidates[0]!.id; + const validDecision = { candidateId, action: "reject" as const }; + worker.setConflationDecisions(base.id, [validDecision]); + worker.generateConflationChangeset(base.id, { + directMerge: true, + deduplicateNodes: true, + deduplicateWays: true, + }); + const generatedChanges = worker.getChangesetPage(base.id, 0, 100).changes; + + expect(() => worker.setConflationDecisions(base.id, [validDecision, validDecision])).toThrow( + `Duplicate conflation decision for ${candidateId}`, + ); + expect(() => + worker.setConflationDecision(base.id, { + candidateId, + action: "invalid", + } as never), + ).toThrow(`Invalid conflation decision action for ${candidateId}`); + expect(() => + worker.setConflationDecision(base.id, { + candidateId, + action: "accept", + attachNetwork: "yes", + } as never), + ).toThrow(`Conflation attachNetwork must be a boolean for ${candidateId}`); + expect(() => worker.setConflationDecision(base.id, null as never)).toThrow( + "Conflation decision must be an object", + ); + expect(() => + worker.discoverConflation(base.id, patch.id, { + propertyKeys: ["crossing"], + attachNetwork: true, + decisions: [{ candidateId, action: "invalid" } as never], + }), + ).toThrow(`Invalid conflation decision action for ${candidateId}`); + expect(() => + worker.discoverConflation(base.id, patch.id, { + propertyKeys: ["crossing"], + attachNetwork: true, + decisions: null, + } as never), + ).toThrow("Conflation decisions must be an array"); + + worker.setConflationFilter(base.id, {}); + expect(worker.getConflationPage(base.id, 0, 100).candidates[0]?.decision).toEqual( + validDecision, + ); + expect(worker.getChangesetPage(base.id, 0, 100).changes).toEqual(generatedChanges); + }); + + it("applies filtered bulk decisions across pages and invalidates generated changes", () => { + const worker = new TestWorker(); + const base = createMixedParallelNetwork("bulk-base", 1, 10, 0, "Base"); + const patch = createMixedParallelNetwork("bulk-patch", 11, 20, 0.000004, "Imported"); + worker.setOsm(base.id, base); + worker.setOsm(patch.id, patch); + worker.discoverConflation(base.id, patch.id, { + propertyKeys: ["name"], + attachNetwork: false, + }); + worker.setConflationFilter(base.id, { entityType: "way" }); + + const firstPage = worker.getConflationPage(base.id, 0, 1); + expect(firstPage.totalPages).toBe(2); + expect(firstPage.bulkActions["transfer-properties"]).toMatchObject({ + filteredCandidates: 2, + eligibleCandidates: 2, + changedCandidates: 2, + }); + + worker.generateConflationChangeset(base.id, { + directMerge: true, + deduplicateNodes: true, + deduplicateWays: true, + }); + const accepted = worker.applyConflationBulkDecision(base.id, { + action: "transfer-properties", + filter: { entityType: "way" }, + }); + expect(accepted.decisions).toHaveLength(2); + expect(accepted.summary.accepted).toBe(2); + expect(() => worker.getChangesetPage(base.id, 0, 100)).toThrow("No active changeset"); + + worker.setConflationFilter(base.id, { status: "accepted" }); + expect(worker.getConflationPage(base.id, 0, 100).totalCandidates).toBe(2); + const rejected = worker.applyConflationBulkDecision(base.id, { + action: "reject", + filter: { status: "accepted" }, + }); + expect(rejected.preview).toMatchObject({ + filteredCandidates: 2, + changedCandidates: 2, + overriddenDecisions: 2, + }); + expect(rejected.summary.rejected).toBe(2); + expect(worker.getConflationPage(base.id, 0, 100).totalCandidates).toBe(0); + expect(() => + worker.applyConflationBulkDecision(base.id, { + action: "invalid", + filter: {}, + } as never), + ).toThrow("Invalid conflation bulk action"); + expect(worker.getConflationSummary(base.id).rejected).toBe(2); + }); + + it("generates diagnostics when way candidates have no network action", () => { + const worker = new TestWorker(); + const base = createParallelFootway("footway-base", 1, 10, 0, "Base path"); + const patch = createParallelFootway("footway-patch", 11, 20, 0.000004, "Imported path"); + worker.setOsm(base.id, base); + worker.setOsm(patch.id, patch); + + worker.discoverConflation(base.id, patch.id, { + propertyKeys: ["name"], + attachNetwork: false, + }); + const wayCandidates = worker + .getConflationPage(base.id, 0, 100) + .candidates.filter((candidate) => candidate.entityType === "way"); + expect(wayCandidates).toHaveLength(1); + expect(wayCandidates[0]?.networkAttachment).toBeNull(); + + const result = worker.generateConflationChangeset(base.id, { + directMerge: true, + deduplicateNodes: true, + deduplicateWays: true, + }); + expect(result.stats.totalChanges).toBeGreaterThan(0); + expect(result.routing.car.delta).toMatchObject({ + routableNodes: 0, + edges: 0, + components: 0, + }); + }); + + it("installs the validated conflation result without rebuilding it", () => { + const worker = new TestWorker(); + const base = createParallelFootway("materialized-base", 1, 10, 0, "Base path"); + const patch = createParallelFootway("materialized-patch", 11, 20, 0.000004, "Imported path"); + worker.setOsm(base.id, base); + worker.setOsm(patch.id, patch); + worker.discoverConflation(base.id, patch.id, { + propertyKeys: ["name"], + attachNetwork: false, + }); + const buildIndexes = vi.spyOn(Osm.prototype, "buildIndexes"); + + try { + worker.generateConflationChangeset(base.id, { + directMerge: true, + deduplicateNodes: true, + deduplicateWays: true, + }); + const buildsAfterGeneration = buildIndexes.mock.calls.length; + expect(buildsAfterGeneration).toBe(2); + + worker.applyChangesAndReplace(base.id); + + expect(buildIndexes).toHaveBeenCalledTimes(buildsAfterGeneration); + expect(worker.getOsm(base.id).ways.getById(10)?.tags?.["name"]).toBe("Imported path"); + } finally { + buildIndexes.mockRestore(); + } + }); + + it("returns defensive candidate, decision, filter, and summary snapshots", () => { + const worker = new TestWorker(); + const base = createParallelFootway("snapshot-base", 1, 10, 0, "Base path"); + const patch = createParallelFootway("snapshot-patch", 11, 20, 0.000004, "Imported path"); + worker.setOsm(base.id, base); + worker.setOsm(patch.id, patch); + const summary = worker.discoverConflation(base.id, patch.id, { + propertyKeys: ["name"], + attachNetwork: true, + }); + const initialPage = worker.getConflationPage(base.id, 0, 100); + const initialWay = initialPage.candidates.find((candidate) => candidate.entityType === "way")!; + const initialNode = initialPage.candidates.find( + (candidate) => candidate.entityType === "node", + )!; + worker.setConflationDecision(base.id, { + candidateId: initialWay.id, + action: "accept", + transferProperties: true, + }); + + const filter: Parameters<typeof worker.setConflationFilter>[1] = { status: undefined }; + worker.setConflationFilter(base.id, filter); + filter.status = "blocked"; + const page = worker.getConflationPage(base.id, 0, 100); + const way = page.candidates.find((candidate) => candidate.id === initialWay.id)!; + const node = page.candidates.find((candidate) => candidate.id === initialNode.id)!; + way.reasons.push("protected-tag"); + way.propertyTransfer.reasons.push("routing-property"); + way.evidence.sourceRoutingFamilies.push("motor-road"); + way.evidence.targetRoutingFamilies.push("motor-road"); + way.evidence.tagDiff[0]!.key = "corrupted"; + way.evidence.endpointDistancesMeters![0] = 999; + way.decision!.action = "reject"; + node.networkAttachment!.reasons.push("grade-conflict"); + node.evidence.patchWayIds!.push(999); + page.bulkActions.reject.changedCandidates = 0; + summary.total = 0; + + const freshPage = worker.getConflationPage(base.id, 0, 100); + const freshWay = freshPage.candidates.find((candidate) => candidate.id === initialWay.id)!; + const freshNode = freshPage.candidates.find((candidate) => candidate.id === initialNode.id)!; + expect(freshPage.candidates.every((candidate) => candidate.status === "automatic")).toBe(true); + expect(freshWay.reasons).not.toContain("protected-tag"); + expect(freshWay.propertyTransfer.reasons).not.toContain("routing-property"); + expect(freshWay.evidence.sourceRoutingFamilies).not.toContain("motor-road"); + expect(freshWay.evidence.targetRoutingFamilies).not.toContain("motor-road"); + expect(freshWay.evidence.tagDiff[0]?.key).toBe("name"); + expect(freshWay.evidence.endpointDistancesMeters?.[0]).toBeLessThan(1); + expect(freshWay.decision?.action).toBe("accept"); + expect(freshNode.networkAttachment?.reasons).not.toContain("grade-conflict"); + expect(freshNode.evidence.patchWayIds).not.toContain(999); + expect(freshPage.bulkActions.reject.changedCandidates).toBeGreaterThan(0); + expect(worker.getConflationSummary(base.id).total).toBeGreaterThan(0); + + worker.generateConflationChangeset(base.id, { + directMerge: true, + deduplicateNodes: true, + deduplicateWays: true, + }); + worker.applyChangesAndReplace(base.id); + expect(worker.getOsm(base.id).ways.getById(10)?.tags?.["name"]).toBe("Imported path"); + }); + + it("allows automatic pedestrian attachment only when the CAR graph is unchanged", () => { + const worker = new TestWorker(); + const base = createParallelFootway("walk-base", 1, 10, 0, "Base path"); + const patch = createParallelFootway("walk-patch", 11, 20, 0.000004, "Imported path"); + worker.setOsm(base.id, base); + worker.setOsm(patch.id, patch); + worker.discoverConflation(base.id, patch.id, { + propertyKeys: ["name"], + attachNetwork: true, + }); + const buildIndexes = vi.spyOn(Osm.prototype, "buildIndexes"); + + try { + const result = worker.generateConflationChangeset(base.id, { + directMerge: true, + deduplicateNodes: true, + deduplicateWays: true, + }); + + expect(buildIndexes).toHaveBeenCalledTimes(3); + expect(result.routing.car.delta).toMatchObject({ + routableNodes: 0, + edges: 0, + components: 0, + }); + expect(result.routing.walk.delta.components).toBeLessThan(0); + } finally { + buildIndexes.mockRestore(); + } + }); + + it("isolates the automatic WALK guard from unrelated CAR way property suppression", () => { + const worker = new TestWorker(); + const base = createMixedParallelNetwork("mixed-base", 1, 10, 0, "Base"); + const patch = createMixedParallelNetwork("mixed-patch", 11, 20, 0.000004, "Imported"); + worker.setOsm(base.id, base); + worker.setOsm(patch.id, patch); + worker.discoverConflation(base.id, patch.id, { + propertyKeys: ["name"], + attachNetwork: true, + }); + + const result = worker.generateConflationChangeset(base.id, { + directMerge: true, + deduplicateNodes: true, + deduplicateWays: true, + }); + + expect(result.routing.car.delta.edges).toBeLessThan(0); + expect(result.routing.walk.delta.components).toBeLessThan(0); + }); }); diff --git a/packages/router/README.md b/packages/router/README.md index e3c270c1..1e61de5b 100644 --- a/packages/router/README.md +++ b/packages/router/README.md @@ -5,6 +5,7 @@ ## Highlights - Builds a directed graph from OSM ways and nodes +- Respects explicit one-way tags and the one-way direction implied by `junction=roundabout` - Configurable highway type filtering - Multiple routing algorithms (Dijkstra, A\*, bidirectional search) - Support for both distance and time-based routing @@ -85,6 +86,10 @@ if (nearest) { - `filter` - Optional function `(tags?) => boolean` to select routable ways. Default: common vehicle highways. - `defaultSpeeds` - Optional speed limits (km/h) by highway type. +Way directionality is currently graph-wide: custom filters can select pedestrian ways, but they +do not disable `oneway` or implicit roundabout direction. Use a policy-aware router such as R5 +when authoritative pedestrian access and direction rules are required. + #### Serialization (Web Worker support) `RoutingGraph` can be serialized and transferred between Web Workers: diff --git a/packages/router/src/binary-heap.ts b/packages/router/src/binary-heap.ts index d9c45c1b..ff761a0d 100644 --- a/packages/router/src/binary-heap.ts +++ b/packages/router/src/binary-heap.ts @@ -115,12 +115,14 @@ export class BinaryHeap { const leftIndex = (index << 1) + 1; const rightIndex = leftIndex + 1; let smallest = index; + let smallestPriority = priority; - if (leftIndex < length && this.priorities[leftIndex]! < this.priorities[smallest]!) { + if (leftIndex < length && this.priorities[leftIndex]! < smallestPriority) { smallest = leftIndex; + smallestPriority = this.priorities[leftIndex]!; } - if (rightIndex < length && this.priorities[rightIndex]! < this.priorities[smallest]!) { + if (rightIndex < length && this.priorities[rightIndex]! < smallestPriority) { smallest = rightIndex; } diff --git a/packages/router/src/graph.ts b/packages/router/src/graph.ts index d5428315..cee2762a 100644 --- a/packages/router/src/graph.ts +++ b/packages/router/src/graph.ts @@ -127,8 +127,18 @@ export class RoutingGraph { const refs = osm.ways.getRefIds(wayIndex); if (refs.length < 2) continue; - // Create bidirectional edges between consecutive nodes (respecting one-way) - const oneway = tags?.["oneway"] === "yes" || tags?.["oneway"] === "1"; + // Create directed edges between consecutive nodes (respecting one-way direction). + const onewayTag = String(tags?.["oneway"] ?? "").toLowerCase(); + const explicitlyForward = onewayTag === "yes" || onewayTag === "1" || onewayTag === "true"; + const explicitlyReverse = onewayTag === "-1" || onewayTag === "reverse"; + const explicitlyBidirectional = + onewayTag === "no" || onewayTag === "0" || onewayTag === "false"; + // OSM roundabouts are one-way by implication unless explicitly overridden. + const direction = explicitlyReverse + ? "reverse" + : explicitlyForward || (tags?.["junction"] === "roundabout" && !explicitlyBidirectional) + ? "forward" + : "both"; const speedKph = getSpeedLimit(tags, defaultSpeeds); const speedMps = (speedKph * 1_000) / 60 / 60; const nodes = refs.map((ref) => osm.nodes.ids.getIndexFromId(ref)); @@ -142,17 +152,17 @@ export class RoutingGraph { const distanceM = haversineDistance(fromCoord, targetCoord); const time = distanceM / speedMps; - // Forward edge - addEdgeToNode(nodeIndex, { - targetNodeIndex: targetNodeIndex, - wayIndex, - distance: distanceM, - time, - }); - this.edgeCount++; - - // Reverse edge (unless one-way) - if (!oneway) { + if (direction !== "reverse") { + addEdgeToNode(nodeIndex, { + targetNodeIndex, + wayIndex, + distance: distanceM, + time, + }); + this.edgeCount++; + } + + if (direction !== "forward") { addEdgeToNode(targetNodeIndex, { targetNodeIndex: nodeIndex, wayIndex, diff --git a/packages/router/src/index.ts b/packages/router/src/index.ts index 2e1093b3..1ccd2ddd 100644 --- a/packages/router/src/index.ts +++ b/packages/router/src/index.ts @@ -9,7 +9,7 @@ * - **Graph construction**: Build routing graphs from OSM ways with highway filtering. * - **Multiple algorithms**: Dijkstra, A*, and bidirectional search. * - **Time-based routing**: Uses maxspeed tags and default speeds by highway type. - * - **One-way support**: Respects oneway=yes/1 tags. + * - **One-way support**: Respects explicit one-way tags and implicit roundabout direction. * - **Snapping**: Find nearest routable node from arbitrary coordinates. * * @example diff --git a/packages/router/test/binary-heap.test.ts b/packages/router/test/binary-heap.test.ts new file mode 100644 index 00000000..c7813d36 --- /dev/null +++ b/packages/router/test/binary-heap.test.ts @@ -0,0 +1,27 @@ +import { describe, expect, it } from "vitest"; + +import { BinaryHeap } from "../src/binary-heap.ts"; + +describe("BinaryHeap", () => { + it("preserves priority order when a replacement must move down multiple levels", () => { + const heap = new BinaryHeap(); + for (let priority = 1; priority <= 7; priority++) { + heap.push(priority, priority); + } + + const popped: number[] = []; + while (heap.size > 0) popped.push(heap.pop()!); + + expect(popped).toEqual([1, 2, 3, 4, 5, 6, 7]); + }); + + it("preserves ordering after decreasing an existing item's priority", () => { + const heap = new BinaryHeap(); + heap.push(1, 10); + heap.push(2, 20); + heap.push(3, 30); + heap.push(3, 5); + + expect([heap.pop(), heap.pop(), heap.pop()]).toEqual([3, 1, 2]); + }); +}); diff --git a/packages/router/test/router.test.ts b/packages/router/test/router.test.ts index 9eebbaf0..a2b19988 100644 --- a/packages/router/test/router.test.ts +++ b/packages/router/test/router.test.ts @@ -165,6 +165,59 @@ describe("Router", () => { } }); + it("treats roundabouts as one-way unless explicitly overridden", () => { + const createRoundabout = (oneway?: string) => { + const osm = new Osm({ id: `roundabout-${oneway ?? "implicit"}` }); + osm.nodes.addNode({ id: 1, lat: 0, lon: 0 }); + osm.nodes.addNode({ id: 2, lat: 0, lon: 0.001 }); + osm.nodes.addNode({ id: 3, lat: 0.001, lon: 0.001 }); + osm.ways.addWay({ + id: 10, + refs: [1, 2, 3, 1], + tags: { + highway: "residential", + junction: "roundabout", + ...(oneway === undefined ? {} : { oneway }), + }, + }); + osm.buildIndexes(); + osm.buildSpatialIndexes(); + return osm; + }; + + const implicit = createRoundabout(); + const implicitRoute = new Router(implicit, buildGraph(implicit)).route(1, 0); + expect(implicitRoute?.map((segment) => implicit.nodes.ids.at(segment.nodeIndex))).toEqual([ + 2, 3, 1, + ]); + + const overridden = createRoundabout("no"); + const overriddenRoute = new Router(overridden, buildGraph(overridden)).route(1, 0); + expect(overriddenRoute?.map((segment) => overridden.nodes.ids.at(segment.nodeIndex))).toEqual([ + 2, 1, + ]); + }); + + it("creates reverse-only edges for oneway=-1", () => { + const osm = new Osm({ id: "reverse-oneway" }); + osm.nodes.addNode({ id: 1, lat: 0, lon: 0 }); + osm.nodes.addNode({ id: 2, lat: 0, lon: 0.001 }); + osm.nodes.addNode({ id: 3, lat: 0, lon: 0.002 }); + osm.ways.addWay({ + id: 10, + refs: [1, 2, 3], + tags: { highway: "residential", oneway: "-1" }, + }); + osm.buildIndexes(); + osm.buildSpatialIndexes(); + const router = new Router(osm, buildGraph(osm)); + + expect(router.route(0, 2)).toBeNull(); + expect(router.route(2, 0)?.map((segment) => osm.nodes.ids.at(segment.nodeIndex))).toEqual([ + 3, 2, 1, + ]); + }); + it("should handle same start and end node", () => { const osm = createTestOsm(); const graph = buildGraph(osm); @@ -241,8 +294,8 @@ describe("Router with Monaco PBF", () => { const graph = buildGraph(monacoOsm); const router = new Router(monacoOsm, graph); expect(router).toBeDefined(); - // 11,414 total edges in the graph (bidirectional edges counted separately) - expect(graph.edges).toBe(11_414); + // Directed edges are counted separately; roundabout and reverse-oneway edges are directional. + expect(graph.edges).toBe(10_831); }); it("should find routes between points in Monaco", () => { diff --git a/packages/shortbread/src/feature-index.test.ts b/packages/shortbread/src/feature-index.test.ts index ea03be1b..0650b83d 100644 --- a/packages/shortbread/src/feature-index.test.ts +++ b/packages/shortbread/src/feature-index.test.ts @@ -147,7 +147,7 @@ describe("ShortbreadFeatureIndex", () => { expect(new Uint8Array(indexedTile)).toEqual(new Uint8Array(unindexedTile)); expect(decodedTileSnapshot(indexedTile)).toEqual(decodedTileSnapshot(unindexedTile)); } - }); + }, 15_000); it("suppresses a member area only when its classified relation supplies the geometry", () => { const osm = new Osm();