diff --git a/.gitignore b/.gitignore index d80e42e8..5b69bc8e 100644 --- a/.gitignore +++ b/.gitignore @@ -32,3 +32,8 @@ grimoires/ # Superpowers implementation plans are local scratch artifacts, never commit them. /docs/superpowers/plans/ /docs/superpowers/plans/**/*.md + +# PR follow-up code-review notes are local scratch, keep them untracked. +/docs/pr-*-follow-up-*code-review*.md +# GA cleanup backlog is a local working note, keep it untracked. +/docs/ga-cleanup-backlog.md diff --git a/docs/superpowers/specs/ 2026-08-04-nodejs-external-modules-design.md b/docs/superpowers/specs/2026-08-04-nodejs-external-modules-design.md similarity index 100% rename from docs/superpowers/specs/ 2026-08-04-nodejs-external-modules-design.md rename to docs/superpowers/specs/2026-08-04-nodejs-external-modules-design.md diff --git a/docs/superpowers/specs/2026-08-07-native-lib-multi-engine-design.md b/docs/superpowers/specs/2026-08-07-native-lib-multi-engine-design.md new file mode 100644 index 00000000..5c031910 --- /dev/null +++ b/docs/superpowers/specs/2026-08-07-native-lib-multi-engine-design.md @@ -0,0 +1,504 @@ +# Design: Multiple Isolated DataWeave Engines per Process (native-lib, Node) + +**Date:** 2026-08-07 (consolidated 2026-08-25) +**Status:** Approved and implemented on `w-23692110-multi-engine-design` (PR #157) +**Tracks:** GUS [W-23692110](https://gus.my.salesforce.com/lightning/r/ADM_Work__c/a07EE00002gS7SOYA0/view) — "Native-lib: support multiple DataWeave engine instances with independent module resolvers" +**Related:** [docs/superpowers/specs/2026-08-04-nodejs-external-modules-design.md](./2026-08-04-nodejs-external-modules-design.md) (the design during which this limitation was discovered) + +> **About this document.** This is the single, consolidated design for the multi-engine Node +> binding. It describes the **final state** of the feature as shipped on PR #157. The core +> feature (object-level engines behind opaque handles) is unchanged from the original design; +> the substantial addition is the **concurrency & lifecycle model** (§6), which was hardened +> across a long series of code reviews. Those hardening decisions are folded into the relevant +> sections here rather than kept as separate per-round documents; a provenance map for git +> archaeology lives in the [Appendix](#appendix-hardening-provenance). The product-facing +> `DataWeave` class is pre-GA, so several internal contracts (async `cleanup()`, the removed +> `*_with_resolver` C ABI) changed during hardening without a compatibility ceremony. + +## 1. Goal + +Let multiple `DataWeave` instances coexist in one Node process, each with its own module +resolver and script cache, so that different resolvers never collide. Before this change the +second `new DataWeave({ resolveModule })` in a process silently kept the first instance's +resolver. The isolation must hold with instances living in different Worker threads and being +created, run, and torn down concurrently, without leaking native resources or wedging the +shared GraalVM isolate. + +## 2. Background + +`native-lib`'s `ScriptRuntime` (`native-lib/src/main/java/org/mule/weave/lib/ScriptRuntime.java`) +was a `static final` singleton holding one `engine` and a **write-once** `static volatile resolver`. +`setResolver` refused to run a second time per process (logged a warning and returned). Every +`@CEntryPoint` in `NativeLib.java` routed through `ScriptRuntime.getInstance()`. So two +`DataWeave` instances in one process could not have independent module sets — whichever called a +resolver-backed `run()` first won. + +**This is not a GraalVM constraint.** `native-cli`'s `NativeRuntime` +(`native-cli/src/main/scala/org/mule/weave/dwnative/NativeRuntime.scala:50-60`) already builds one +independent `DataWeaveScriptingEngine` + `CompositeWeaveResourceResolver` per instance — there is +no shared static state there. GraalVM Java statics are scoped per-isolate, which is also why the +Python binding (one GraalVM isolate per `DataWeave()` instance) already gets resolver isolation +"for free." The limitation was specific to `native-lib`'s deliberate Java static singleton plus +the Node C addon's global resolver bridge. + +## 3. Scope + +**In scope:** +- `native-lib` Java layer: turn `ScriptRuntime` from a static singleton into a handle-addressable + registry of instances, each with its own engine + resolver. +- Node C addon (`native-lib/node/src/addon.c`): per-handle resolver bridge state instead of one + process-global bridge, plus the concurrency & lifecycle machinery in §6. +- Node TypeScript layer (`ffi.ts`, `dataweave.ts`, `stream.ts`, `reader.ts`): each `DataWeave` + instance owns an engine handle for its whole lifecycle. + +**Out of scope:** +- Python binding changes. Python already achieves isolation via one isolate per instance; + unifying it onto the same handle-based API is a follow-up. +- Separate GraalVM isolates per engine — rejected as the isolation mechanism (see §5). +- Solving streaming/transform + **custom-module** resolution across the background-thread + boundary. Streaming against a resolver-backed engine still fails closed (returns "not found") + for custom modules reached from a background worker thread; built-in modules continue to + resolve normally in all cases. This is a pre-existing, documented hazard, not introduced here. + +## 4. Definitions + +- **Isolate** — the single process-wide GraalVM isolate. All engines share it. Its lifetime is + governed by `g_ref_count` (§6.1). +- **Engine** — a `ScriptRuntime` Java object (own resolver + compiled-script cache) addressed by + an opaque `long long` handle. Many engines per isolate. +- **Init reference** — the `g_ref_count` unit an env acquires on each `initialize()` and releases + on the matching `cleanup()` (or on env death). Distinct from an engine handle. +- **Op** — one in-flight `run()`/`runStreaming()`/`runTransform()` native call. +- **Owner env / owner thread** — the `napi_env` (and its JS thread) that created a given engine + or init reference. `napi_env`/`napi_ref`/`napi_deferred`/`napi_threadsafe_function` are + thread-affine; env-affine calls only ever happen on the owner thread. + +## 5. Alternatives Considered (isolation mechanism) + +**Separate GraalVM isolates per engine (rejected).** The most complete isolation (own heap, own +JIT, own Java statics), and what Python does per-instance. Rejected for Node because `addon.c` +assumed exactly one isolate as global state; supporting N isolates means restructuring all of +that into per-handle structs, and isolate teardown is fragile (`graal_tear_down_isolate` blocks +until every attached thread reaches a safepoint). It is unnecessarily heavy for the actual need: +independent module resolution and script caching, not full JVM-level sandboxing. + +**Chosen: object-level engines in one shared isolate.** Multiple `ScriptRuntime` Java objects, +each with its own resolver and compiled-script cache, all in the single existing GraalVM isolate, +addressed by an opaque handle. Mirrors what `native-cli` already does and requires no change to +isolate lifecycle management for the *feature* — though it does require the careful +reference-and-teardown coordination in §6, because now the isolate is shared by independently +created and destroyed engines across threads. + +## 6. Concurrency & Lifecycle Model + +This section is the heart of the design. It governs how the shared isolate, per-engine registry +entries, and in-flight ops coordinate so that no thread ever attaches to, executes on, or +resolves a module against a torn-down isolate or a freed engine record, and no native resource +leaks — under concurrent creation, execution, abandonment (env death without `cleanup()`), and +teardown across Worker threads. + +All shared C state is read and written **only under `g_mutex`**, with two documented exceptions: +the cheap top-of-function `!g_initialized` fast-path read (a benign optimization; the +authoritative check is under the lock), and the lock-free `g_isolate` NULL-check that narrows a +window before a guarded re-check. + +### 6.1 The reference-ownership invariant + +The isolate lives while any env holds an init reference. The governing invariant is: + +> **`g_ref_count` == Σ `init_refs` over all live per-env records.** + +`g_ref_count` is a derived total, not a bare global that any code path may drive to zero. +Reference accounting is **per `napi_env`**, tracked in a `g_mutex`-guarded linked list of +`env_init_rec_t { napi_env env; int init_refs; next; }`: + +- **`initialize()`** acquires one init reference *on the calling env's record* (find-or-create the + record, `init_refs++`, `g_ref_count++`, both under the same lock). The record registers exactly + one env-death hook (`env_init_cleanup`) on first creation. +- **`cleanup()`** releases one reference **only if the calling env owns one** (`init_refs > 0`). + A `cleanup()` with no matching `initialize()` on that env, or a double-`cleanup()`, is a no-op + that resolves immediately — it must never steal another env's reference and tear the isolate + down under a live user. +- **Env death** (`env_init_cleanup`, an env-cleanup hook) releases *all* of that env's remaining + references at once, from a single env-scoped decision point. This is what reclaims an abandoned + Worker that exited without calling `cleanup()`. +- **`destroyEngine` never releases an init reference** — engines and init references have distinct + lifetimes (Java registry entry vs. isolate). The product `doCleanup()` calls `destroyEngine` + then `ffi.cleanup()`; the latter is the sole release. + +Because every release is keyed on a specific env's balance, an abandoned env-A can only reach +`g_ref_count == 0` when no other env holds a reference — so it can never tear down the isolate +under a live env-B. This closes both the cross-env abandonment UAF and the symmetric +over-`cleanup()` UAF. + +The three `g_ref_count` mutators after this design are: the three `initialize()` acquire sites +(adoption / already-initialized fast path / create path), `release_isolate_ref_locked` (the +`cleanup()` path), and `env_init_cleanup` (env death, via the bounded multi-release helper +`isolate_ref_release_n_locked(n)`, which makes the reached-zero teardown decision *at most once* +regardless of how many references it drops). + +### 6.2 The teardown state machine + +When a release drops `g_ref_count` to 0, the isolate must be torn down — but only after every +in-flight op has drained, because `graal_tear_down_isolate` blocks until every GraalVM-attached +worker thread detaches, and those workers deliver chunks via a `napi_threadsafe_function` that +needs the JS event loop to run. A naïve synchronous join-on-teardown from the JS thread +therefore **deadlocks**: JS thread waits for teardown → teardown waits for the worker to detach → +the worker waits for the JS thread to run its chunk callback. + +The resolution is a `g_active_ops` counter (all in-flight ops, every engine, every thread) plus a +tri-state machine, all under `g_mutex`: + +``` +TEARDOWN_NONE no teardown queued or running. +TEARDOWN_PENDING_WAIT a reached-zero release queued a teardown; a detached waiter thread is + blocked on `while (g_active_ops > 0 && !g_teardown_cancelled)`. The + isolate is STILL LIVE here — a fresh initialize() may ADOPT it. +TEARDOWN_TEARING_DOWN the waiter passed the point of no return and is in + graal_tear_down_isolate(). Adoption is unsafe; initialize() blocks + (deadlock-free, because g_active_ops is already 0 — nothing depends on + the JS loop). +``` + +- **Reached-zero release, `g_active_ops == 0`:** synchronous fast path — spawn+join + `cleanup_thread_fn` inline (it attaches its own Graal thread, tears down, and reports + success only when `graal_tear_down_isolate` returns 0), then clear + `g_thread`/`g_isolate`/`g_initialized`/`g_ref_count`. +- **Reached-zero release, `g_active_ops > 0`:** set `TEARDOWN_PENDING_WAIT`, spawn the detached + waiter thread, return a pending promise. Each op's completion sentinel decrements `g_active_ops` + and broadcasts `g_teardown_cond`; when it reaches 0 the waiter publishes `TEARDOWN_TEARING_DOWN` + (under the lock, the point of no return) and tears down. +- **Adoption (the deadlock fix):** an `initialize()` arriving in `TEARDOWN_PENDING_WAIT` sets + `g_teardown_cancelled = true`, takes a fresh init reference, broadcasts, and returns — the + waiter re-checks the flag, tears down nothing, and resolves every queued `cleanup()` promise + anyway (from each caller's perspective the reference it dropped is gone, whether the isolate was + physically destroyed or adopted by a newcomer is immaterial). +- **Multiple concurrent `cleanup()` calls** waiting on the same teardown each append a node + `{env, deferred, tsfn}` to `g_teardown_waiters` — a *list*, because a second/third `cleanup()` + can arrive from a different Worker env, and each thread-affine deferred must be resolved via its + own env's tsfn on its own thread. + +**Teardown-failure retry signal.** If a reached-zero teardown cannot be carried out — waiter +alloc/spawn fails, `fn_attach_thread` fails, or `graal_tear_down_isolate` returns nonzero — the +isolate is left live with `g_ref_count == 0` and no owner. Rather than fabricate a phantom +reference (which would violate the §6.1 invariant and the resolved `cleanup()` promise's +contract), a `g_mutex`-guarded `g_teardown_needed` **retry signal** is armed. It is *not* a +reference (never added to any count). It is cleared when the isolate is actually torn down or +adopted. Retry runs at two natural, already-locked points: each op-completion drain (once +`g_active_ops` reaches 0), and the top of the next `napi_initialize` (before adoption, so a +pending teardown is honored rather than silently discarded). On a nonzero-return teardown the +helper threads also **detach** their local IsolateThread before exiting (the isolate is still +live; exiting attached would leave a phantom thread that blocks later retries). The documented, +accepted residual: if teardown fails *and* no later `initialize()` or op ever occurs, the isolate +lingers until process exit — benign (one process-lifetime isolate, no invariant violation), the +deliberate tradeoff for not adding event-loop-affine async retry infrastructure to this code. + +### 6.3 Per-engine records, admission pinning, and deferred destroy + +Every engine — resolver-backed **and** resolver-less — gets a per-engine record +(`engine_bridge_t`) at creation, linked into `g_bridges`, carrying `handle`, `in_flight`, +`destroy_pending`, `deferred_registry_remove`, and (for resolver-backed engines only) +`resolver_js`/`env`/`owner`/`results`. Resolver-less records leave those resolver fields +zero/NULL. `in_flight` (per-handle registry drain) and `g_active_ops` (global isolate teardown) +are **distinct counters**, never merged. + +**Admission pins the engine atomically.** Each of the three run paths reserves `g_active_ops` +*and* pins the engine (`in_flight++` via `bridge_begin_op_locked`) in the **same** critical +section as the lifecycle check, before any window a concurrent `destroyEngine` could use: + +```c +uv_mutex_lock(&g_mutex); +if (!g_initialized || (g_teardown_state != TEARDOWN_NONE && !g_teardown_cancelled)) { + uv_mutex_unlock(&g_mutex); + /* free partials */ napi_throw_error(env, NULL, "Not initialized. Call initialize() first."); + return NULL; +} +g_active_ops++; +w->bridge = bridge_begin_op_locked(handle); // NULL for unknown handle -> worker surfaces the envelope +uv_mutex_unlock(&g_mutex); +``` + +- **Streaming / transform** reserve *early* (arg extraction is cheap relative to the async op) and + release via the completion sentinel on the worker thread. Every early-return between admission + and worker spawn (conversion error, OOM, tsfn/promise-create failure, spawn failure) unwinds + **both** `g_active_ops` and the engine pin. +- **Synchronous `run()`** reserves *late* — immediately before `fn_attach_thread`, so the + reservation spans exactly the isolate-touching window (attach→detach) with only two unwind + sites (attach-failure and normal completion); the string mallocs and arg extraction don't touch + the isolate. +- **`createEngine` / `createEngineWithResolver`** likewise do their lifecycle check + a transient + `g_active_ops` reservation in one critical section, and additionally require that the **calling + env owns an init reference** (`init_refs > 0`) — an env that never initialized must not create + engines on the shared isolate. + +With the pin taken under the admission lock, a concurrent `destroyEngine` either runs entirely +before admission (the handle is already gone → worker surfaces `Unknown engine handle`, no freed +access) or entirely after (`in_flight > 0` → destroy defers). There is no interleaving where an +admitted op observes a freed bridge. + +**Deferred destroy.** `napi_destroy_engine`, under `g_mutex`: if `in_flight > 0`, set +`destroy_pending` and defer the Java-registry removal (`fn_destroy_engine`); the last op to drain +performs it on completion. If `in_flight == 0`, remove now. `fn_destroy_engine` is called +**exactly once** per handle (immediate xor deferred, never both), and it attaches its own fresh +Graal thread so it is safe to call from the completion sentinel or directly. + +**The registry-removal step is itself teardown-guarded.** Removing the Java registry entry +touches the isolate (`fn_attach_thread(g_isolate, …)`), so it is split into +`bridge_finalize_registry` — which takes its **own** transient `g_active_ops` reservation, gated +on `g_teardown_state != TEARDOWN_TEARING_DOWN && g_isolate != NULL` in the *same* critical section +as the increment — and `bridge_finalize_free` (napi_ref deletion, still resolver-gated and on the +owner thread; result-buffer free; `free`). This closes the race where a deferred finalize could +attach to an isolate the waiter is destroying, without re-opening the completion-path +coordination: the op's own `g_active_ops--` stays on the worker thread; the finalize takes a fresh +short-lived reservation only around the attach, makes no env-affine or JS-loop-dependent call, and +never holds it across a JS callback (so it cannot re-introduce the §6.2 deadlock). + +**Env cleanup hooks reclaim abandoned engines.** Every engine registers a +`napi_add_env_cleanup_hook` at creation (checked for failure — creation is all-or-nothing; on +hook-registration failure the record is unlinked, its registry entry removed, its init reference +released, and the create throws with no usable handle escaping). When the owner env dies without +`destroyEngine`, the hook removes the Java registry entry and frees the record. Because every +engine now carries an env-affine hook, the **owner-thread `destroyEngine` guard fires for any +record** (not only resolver-backed ones): `napi_remove_env_cleanup_hook` is valid only on the +owner env, so an engine is destroyable only from its creating thread. Node runs env-cleanup hooks +LIFO, and the per-env init-record hook is registered on the *first* `initialize()` (before any +engine) — so at env death every per-engine `bridge_env_cleanup` runs (isolate still alive) before +`env_init_cleanup` releases the isolate reference(s). Ordering preserved. + +### 6.4 JS instance lifecycle + +`DataWeave` models three states — `"uninitialized" | "ready" | "cleaning-up"` — not a boolean, +because a boolean cannot represent the window during which `cleanup()` has started but +`ffi.cleanup()` has not settled: + +- **`initialize()`** — `ready` → no-op; `cleaning-up` → **throws** `DataWeaveError("Cannot + initialize while cleanup is in progress; await cleanup() first.")`; `uninitialized` → does the + load/create-engine work, `state = "ready"` on success. On engine-creation failure *after* + `ffi.initialize()` succeeded, the rollback release is modeled as pending state: `state` goes + `cleaning-up` and `cleanupPromise` is assigned the `ffi.cleanup()` rollback (with a + `.catch(() => {})` so an un-awaited rollback never becomes an unhandledRejection), so a + concurrent `initialize()` is deterministically rejected instead of racing a fresh isolate + against the in-flight release. `initialize()` and `run()` stay **synchronous** (an async + signature would be an API break). +- **`run()` / `runStreaming()` / `runTransform()`** — gated by `ensureReady()`: throw + `DataWeaveError` unless `state === "ready"`, so the internal `engineHandle === null` cleanup + window is unreachable by any public method (defense-in-depth behind the C admission check). + `runTransform` additionally re-checks `ensureReady()` **after** `await createChunkReader(input)` + (async input pre-buffering can span arbitrary time; the instance may be cleaned up during it) so + a misused instance gets a synchronous `DataWeaveError` rather than a resolved `Unknown engine + handle` envelope. +- **`cleanup()` / `doCleanup()`** — set `state = "cleaning-up"` synchronously *before* + `ffi.destroyEngine` / `ffi.cleanup` (the key ordering). It always runs `await ffi.cleanup()` + even if `destroyEngine()` throws (a real path — wrong-thread destruction throws synchronously), + so a throwing destroy cannot strand this env's init reference; the destroy error is re-thrown + after the release. `engineHandle` is cleared regardless so a retry cannot double-destroy. + Overlapping calls coalesce on `this.cleanupPromise` (one native teardown). `cleanup()` returns + `Promise`. + +**Module-level convenience API** (`run`/`cleanup`) drives a lazily-created singleton: +- `getGlobalInstance()` initializes a **local candidate** and publishes `globalInstance` only after + `initialize()` succeeds — a failed first init leaves the singleton null so the next call retries + cleanly, instead of poisoning it into permanent "not initialized". +- Process exit hooks (`beforeExit` async-drains, `exit` best-effort sync) are registered **once + per process** (module-scoped `exitHooksRegistered`, never reset), not per singleton, so + init→cleanup→reinit cycles don't accumulate listeners. `exit` is documented as best-effort: + Node does not emit it for termination signals (SIGTERM/SIGKILL) or all fatal modes; callers + needing guaranteed graceful shutdown register and await their own signal handlers. +- Module-level `cleanup()` coalesces overlapping calls via a module-scoped `cleanupPromise` (it + nulls `globalInstance` synchronously so new work builds a fresh instance, but overlapping + `cleanup()`s await the same drain and resolve only when native teardown finishes). + +### 6.5 Robustness of native allocation and streaming + +- **OOM safety.** Every allocation in the streaming/transform setup, worker, and callback paths + (`calloc`/`malloc`/`strdup`/`memcpy`, and every `napi_create_string_utf8`/ + `napi_create_threadsafe_function`/`napi_create_promise`) is NULL/status-checked before use. + Setup-phase failures throw a synchronous `napi_throw_error(env, NULL, "OOM")` (matching + `napi_run_script_engine`) and unwind `g_active_ops` + the engine pin with no double-free + (`calloc`-zeroed `w` makes the free-set `free(NULL)`-safe). Worker-thread OOM produces a + **terminal error JSON result** (a static `{"success":false,"error":"Out of memory"}` string when + the copy itself failed, flagged so it is never `free()`d), never a hung promise. +- **Argument validation.** Every FFI-facing entrypoint checks the status of every + `napi_get_value_*` conversion (handle `int64`, string size-probes and fills, `napi_typeof` for + nullable args) and throws before using the converted value, so a raw addon caller cannot turn a + malformed argument into an uninitialized native input. `inputCharset` is nullable + (`string | null | undefined`); any other type is rejected rather than silently coerced. +- **Stream error propagation.** `streamFromNative` handles **both** settlement branches of the + native `start()` promise: on rejection it records the error, marks completion, and wakes every + parked `next()` consumer (otherwise the generator hangs forever and the rejection is unhandled), + then re-throws after draining any chunks that arrived first. Rejection is tracked by a dedicated + `startRejected` boolean, not a value sentinel, so `Promise.reject(undefined)` propagates + correctly. + +## 7. Architecture (layer map) + +### Layer 1 — Java (`native-lib/src/main/java/org/mule/weave/lib/`) + +- **`ScriptRuntime.java`** — from static singleton to per-instance + a + `ConcurrentHashMap` registry with `register`/`get`/`destroy` and an + `AtomicLong` handle allocator. The resolver is bound once at construction (immutable for the + instance's lifetime); the `static setResolver` write-once mutation is removed. + `compositeResolver()` / `createModuleComponentsFactory()` become instance methods. + `getInstance()` is **kept** returning a lazily-created default (ClassLoader-only, handle-less) + instance so the resolver-less legacy entrypoints used by Python are untouched. +- **`CallbackWeaveResourceResolver.java`** — stores a `PointerBase ctx` alongside the callback, + forwarded on every `callback.invoke(...)`; constructor `(ResolveModuleCallback, PointerBase ctx)`. +- **`NativeCallbacks.java`** — `ResolveModuleCallback` gains a `ctx` parameter + (`invoke(IsolateThread, PointerBase ctx, CCharPointer modulePath)`), mirroring the existing + `WriteCallback`/`ReadCallback` ctx idiom. This is what lets one shared native callback dispatch + to the correct per-handle JS resolver on the C side. +- **`NativeLib.java`** — adds handle-based lifecycle + execution entrypoints (`create_engine`, + `create_engine_with_resolver`, `destroy_engine`, `run_script_engine`, + `run_script_callback_engine`, `run_script_input_output_callback_engine`) resolving via + `ScriptRuntime.get(handle)`. The legacy singleton entrypoints (`run_script`, + `run_script_callback`, `run_script_input_output_callback`) are **preserved unchanged** for + Python. The old `*_with_resolver` entrypoints are **removed** (see §9). + +### Layer 2 — C addon (`native-lib/node/src/addon.c`) + +- Per-handle resolver bridge state in `g_bridges` (§6.3) instead of a process-global bridge. +- **Resolver dispatch:** `createEngineWithResolver` passes the bridge record's address as the + `ctx`; when Java invokes `resolve_module_callback(thread, ctx, path)`, C casts `ctx` back to the + bridge and calls its JS resolver **synchronously on the JS thread** (no + `napi_threadsafe_function` — the create call runs synchronously on the calling JS thread, so the + original deadlock rationale still holds). A per-handle `owner`-thread guard fails closed to "not + found" if `resolve_module_callback` is reached from a non-owner thread (e.g. a streaming worker). +- All of §6's machinery: `g_active_ops`, the `TEARDOWN_*` state machine, `g_teardown_cancelled`, + `g_teardown_needed`, the per-env `g_env_recs` list, the `g_bridges` list, admission pinning, and + the split finalize. +- N-API methods: `createEngine`, `createEngineWithResolver`, `destroyEngine`, and handle-taking + `runScriptEngine`, `runScriptStreamingEngine`, `runScriptTransformEngine`. + +### Layer 3 — Node TypeScript (`native-lib/node/src/`) + +- **`ffi.ts`** — `createEngine`, `createEngineWithResolver`, `destroyEngine`, and handle-taking + `runScriptEngine`, `runScriptStreamingEngine`, `runScriptTransformEngine`. `runWithResolver` + removed. +- **`dataweave.ts`** — `DataWeave` owns a `private engineHandle`, the three-state lifecycle + machine, and the module-level singleton/exit-hook/coalescing logic (§6.4). `initialize()` calls + `ffi.createEngineWithResolver(this.resolveModule)` or `ffi.createEngine()`; run methods route + through the handle-based FFI (one code path per method, parameterized by handle); + `cleanup()` calls `ffi.destroyEngine` then `ffi.cleanup`. +- **`stream.ts`** — `streamFromNative` error propagation (§6.5). **`reader.ts`** — + `createChunkReader` pre-buffers async inputs (the native read callback is synchronous and cannot + await), which is why `runTransform` re-checks readiness after it. + +## 8. Data Flow + +``` +new DataWeave({ resolveModule: A }).initialize() + → ffi.initialize() // env init record for this env: init_refs 0→1, g_ref_count++ + → ffi.createEngineWithResolver(A) + → addon.c: allocate bridge_A { env, ref to A, owner=thisThread, in_flight:0 }; register env hook + → Java: new CallbackWeaveResourceResolver(callback, ctx=&bridge_A); + new ScriptRuntime(resolver) → handle_A = register(rt) + → handle_A stored as this.engineHandle + +dwA.run(script importing "custom/lib.dwl") + → ffi.runScriptEngine(handle_A, script, inputs) + → addon.c: admission (g_active_ops++, in_flight++ on bridge_A) → attach → fn_run_script_engine + → Java: ScriptRuntime.get(handle_A).run(...) + composite resolver: ClassLoader miss → callback.invoke(thread, ctx=&bridge_A, "custom/lib.dwl") + → C: resolve_module_callback casts ctx→bridge_A; thread==owner? yes → call resolver A synchronously + → result flows back, script compiles; on completion: in_flight--, g_active_ops-- + +new DataWeave({ resolveModule: B }).initialize() → handle_B, bridge_B (independent resolver + owner) +dwB.run(...) → resolves via resolver B only; A's cache untouched; no cross-talk +``` + +## 9. Error Handling & Backward Compatibility + +- **Module not found / resolver throws:** resolver returns `null` → composite resolver falls + through → standard DataWeave "unable to resolve module" error (unchanged, scoped per-handle). +- **Wrong-thread resolver invocation:** per-handle `owner` check fails closed to "not found" + rather than touching `napi_env` cross-thread. +- **Invalid/unknown/destroyed handle:** `ScriptRuntime.get(handle)` returns null → the entrypoint + returns `{"success":false,"error":"Unknown engine handle"}` (resolved for async ops, returned as + the JSON string for sync `run()`), never an NPE. +- **Admission / argument / allocation failures:** synchronous `napi_throw_error` (generic Error); + worker-thread OOM → terminal error JSON. Never `napi_reject_deferred` (absent from `addon.c`). +- **Python binding:** zero changes — it never called the removed `*_with_resolver` entrypoints and + continues on `getInstance()`. +- **Node, resolver-less / single-resolver usage:** behaves identically; the new code path is a + functional superset. +- **Intended breaking changes (pre-GA, no shims):** the dwlib C ABI drops the exported + `run_script_with_resolver` / `run_script_callback_with_resolver` / + `run_script_input_output_callback_with_resolver` entrypoints and replaces them with the + `*_engine` set, and adds a `ctx` parameter to `ResolveModuleCallback`. dwlib is consumed by this + repo's own Python and Node bindings in lockstep. `DataWeave.cleanup()` changes from `void` to + `Promise`. These are documented in the PR, not shimmed. + +## 10. Testing Strategy + +- **Java unit** (`native-lib:test`): two `ScriptRuntime` instances with different in-memory + resolvers each resolve only their own module; `destroy()` removes an instance. (The + `@CEntryPoint` methods can't be driven from a hosted JVM — GraalVM word types don't box — so + handle-based entrypoint coverage lives at the Node integration layer.) +- **Node integration** (`native-lib:nodeTest`, real addon, `vi.mock` of `ffi` forbidden): the core + W-23692110 regression (two independent resolvers in one process); unknown/destroyed-handle + envelopes for all three run paths; the deadlock regression (active stream + `cleanup()` + + concurrent `run()` resolves within a bounded timeout); same-instance lifecycle + (init/run/transform during the cleanup window); ref-count-proxy teardown assertions (a + subsequent raw engine call throwing `/not initialized/` proves the isolate reached zero refs); + and `worker_threads` Worker lifecycle — resolver-backed and resolver-less engines in a Worker, + per-Worker resolver binding, **normal Worker exit without `cleanup()`** (the abandonment / + init-reference-release proof: N Workers each `initialize()` + create N≥3 engines and exit; the + main thread's engine must survive and final teardown must reach exactly zero), + `Worker.terminate()` mid-life, and explicit in-Worker `cleanup()`. +- **Unit** (`ffi` mocked, no dwlib): `DataWeave.initialize()` ref-count/rollback safety; module + singleton poisoning recovery; module + instance `cleanup()` coalescing; `stream.ts` rejection + propagation (parked consumer wakes and throws; buffered-then-reject drains first); `runTransform` + post-pre-buffer re-check; `doCleanup()` releasing the init reference even when `destroyEngine` + throws. +- **Documented posture on non-forceable paths.** Allocator/N-API fault injection and exact + cross-thread teardown interleavings are **not deterministically forceable** from JS/vitest (no + addon-boundary fault-injection hook — deliberately not added, YAGNI/test-only surface). Their + correctness rests on the C-level invariants in §6, verified by code reasoning and adversarial + review; the Worker tests are best-effort probabilistic guards (green on fixed code, cannot + false-fail on it). This is a standing, documented decision. +- **Native image build** (`native-lib:nativeCompile`) stays green. + +## 11. Follow-Up Work + +- **Python binding parity:** port the handle-based `create_engine`/`run_script_engine` API to the + Python binding so both bindings share one mental model (child GUS item under W-23692110). The + broad Python-binding modernization currently riding along in this PR is acknowledged as a + scope-bundling and deferred to its own follow-up PR rather than split mid-review. +- **Streaming/transform + custom-module resolution** across the background-thread boundary remains + a separate, not-yet-scoped effort (unrelated to the singleton fix). + +## References + +| Item | Location | +|------|----------| +| GUS ticket | W-23692110 | +| Singleton root cause | `native-lib/src/main/java/org/mule/weave/lib/ScriptRuntime.java` | +| CLI's per-instance pattern (proof it's not a GraalVM constraint) | `native-cli/src/main/scala/org/mule/weave/dwnative/NativeRuntime.scala:50-60` | +| WriteCallback/ReadCallback ctx idiom | `native-lib/src/main/java/org/mule/weave/lib/NativeCallbacks.java` | +| Concurrency & lifecycle machinery | `native-lib/node/src/addon.c` | +| JS lifecycle / singleton / exit hooks | `native-lib/node/src/dataweave.ts` | +| Stream error propagation | `native-lib/node/src/stream.ts` | +| Node binding API + lifecycle docs | `native-lib/node/README.md`, `native-lib/node/docs/external-modules.md` | +| Original external-modules design | `docs/superpowers/specs/2026-08-04-nodejs-external-modules-design.md` | + +## Appendix: Hardening provenance + +The concurrency & lifecycle model (§6) converged over a series of code-review rounds; each round's +decisions are folded into the sections above. This map exists only for git archaeology — the +per-round design documents were consolidated into this file. + +| Round(s) | Area folded into | Decision | +|----------|------------------|----------| +| Feature (08-07) | §1–§5, §7–§9 | Object-level engines behind opaque handles; per-handle resolver bridge; ABI redesign. | +| 5 (08-11) | §6.2 | `cleanup()`-during-active-stream deadlock → async teardown + waiter thread + `TEARDOWN_*` adoption. | +| 6 (08-14) | §6.3, §6.4 | JS three-state lifecycle; atomic streaming/transform admission under `g_mutex`; handle-read validation. | +| 7 (08-18 ffi-sweep) | §6.3, §6.5, §9 | Atomic admission for sync `run()`; uniform `napi_get_value_*` status checks; docs await `cleanup()`. | +| 8 (08-18 oom-setup) | §6.5 | OOM-safe streaming/transform setup allocations. | +| 9 (08-18 engine/worker-oom) | §6.3, §6.5 | Deferred registry removal for all engines; worker/callback OOM → terminal result; N-API-create checks. | +| 10 (08-19 dangling-ctx) | §6.3, §6.4 | Env-cleanup removes the Java registry entry (`deferred_registry_remove`); shutdown-doc accuracy. | +| 11 (08-19 engine-pin) | §6.1, §6.3, §6.4 | Env hook + owner-guard for every engine; admission-time engine pin in all 3 paths; register-once exit hooks. | +| 12 (08-19 worker-ref-leak) | §6.1, §6.3, §6.4 | Init-reference release on abandoned env; teardown-guarded split finalize; module `cleanup()` coalescing; `runTransform` re-check; all-or-nothing engine creation. | +| 13 (08-20 per-env init) | §6.1 | Per-`napi_env` init-reference ownership; `g_ref_count == Σ init_refs`. | +| 14 (08-21 review5) | §6.2, §6.3 | Engine-creation admission requires an owned init reference; `g_teardown_needed` retry flag; `doCleanup()` releases the ref even when destroy throws. | +| 15 (08-21 review6) | §6.2, §6.4, §6.5 | Singleton-poisoning fix; stream rejection propagation; teardown return-code checks; init-driven stranded-teardown retry. | +| 16 (08-24 review7) | §6.2, §6.4, §6.5, §9 | Detach on failed teardown; init-hook-failure retry arming; observable init rollback; `Promise.reject(undefined)` fix; lifecycle-doc accuracy. | diff --git a/native-lib/README.md b/native-lib/README.md index 1498898f..550bc306 100644 --- a/native-lib/README.md +++ b/native-lib/README.md @@ -450,14 +450,21 @@ import { DataWeave } from "@dataweave/native"; const dw = new DataWeave(); dw.initialize(); - -const r1 = dw.run("2 + 2"); -const r2 = dw.run("x + y", { x: 10, y: 32 }); - -console.log(r1.getString()); // "4" -console.log(r2.getString()); // "42" - -dw.cleanup(); +try { + const r1 = dw.run("2 + 2"); + const r2 = dw.run("x + y", { x: 10, y: 32 }); + + console.log(r1.getString()); // "4" + console.log(r2.getString()); // "42" +} finally { + // cleanup() returns a Promise; await it. When this releases the FINAL shared + // native reference in the process, it drains any in-flight streaming/transform + // op and completes isolate teardown before resolving (so a subsequent + // initialize() does not race a still-tearing-down isolate). When other + // initialized instances remain, it resolves as soon as this instance is + // released, leaving the shared isolate live for them. + await dw.cleanup(); +} ``` ### 5) Error handling @@ -635,11 +642,20 @@ for await (const chunk of gen) { ### 9) Cleanup -The module registers a `process.on('exit')` handler to clean up automatically. For explicit control: +The module registers two process hooks to clean up automatically: `beforeExit` +(async — it awaits cleanup so an in-flight streaming/transform op drains before +the process exits normally) and `exit` (a synchronous best-effort fallback for +`process.exit()` and uncaught exceptions, which cannot await the drain). Neither +hook fires on `SIGTERM`/`SIGINT`/`SIGKILL`, so install your own signal handler +that awaits `cleanup()` if you need a graceful drain on termination. For explicit +control: ```typescript import { cleanup } from "@dataweave/native"; -// When done with all DataWeave operations -cleanup(); +// When done with all DataWeave operations. cleanup() returns a Promise; await it. +// Draining in-flight streaming/transform work and tearing down the isolate happen +// only when this releases the final shared native reference; if other initialized +// instances remain, it resolves as soon as this instance is released. +await cleanup(); ``` diff --git a/native-lib/node/README.md b/native-lib/node/README.md index 3f135c41..e7225b4a 100644 --- a/native-lib/node/README.md +++ b/native-lib/node/README.md @@ -128,13 +128,17 @@ const generator = runStreaming( '%dw 2.0\noutput application/json\n---\n[1, 2, 3, 4, 5]' ); -for await (const chunk of generator) { - console.log('Chunk:', chunk.toString()); +// Iterate manually with next() to capture the terminal return value. A +// `for await` loop consumes the generator's return value internally, so a later +// generator.return() would yield { value: undefined } -- drive next() yourself +// and read the metadata off the terminal { done: true, value: StreamingResult }. +let meta; +while (true) { + const { value, done } = await generator.next(); + if (done) { meta = value; break; } + console.log('Chunk:', value.toString()); } - -// Generator return value contains metadata: -const meta = await generator.return(); -console.log('MIME type:', meta.value.mimeType); +console.log('MIME type:', meta.mimeType); ``` **Parameters:** @@ -156,12 +160,21 @@ Execute a DataWeave script with streaming input and output (bidirectional stream ```javascript import { runTransform } from '@dataweave/native'; -import { createReadStream } from 'fs'; +import { readFileSync } from 'fs'; + +// The native read callback is synchronous, so an ASYNC input iterable (e.g. +// fs.createReadStream) is fully pre-buffered into memory before the transform +// starts. A SYNCHRONOUS iterable is instead consumed on demand -- one chunk at a +// time -- so the transform makes no extra full copy of the input (it does NOT by +// itself bound total memory: a source like readFileSync still holds the whole +// input). (See "Sync vs async input and memory" below.) +function* chunked(buf, size = 65536) { + for (let i = 0; i < buf.length; i += size) yield buf.subarray(i, i + size); +} -// Transform a large CSV file to JSON without loading it all into memory const generator = runTransform( '%dw 2.0\noutput application/json\n---\npayload', - createReadStream('large-file.csv'), + chunked(readFileSync('large-file.csv')), { inputName: 'payload', mimeType: 'application/csv', @@ -175,28 +188,39 @@ for await (const chunk of generator) { } ``` +> **Sync vs async input and memory.** The native read callback runs synchronously +> on the JS thread. **Synchronous** iterables (arrays, generators) are consumed +> on demand — the transform holds only one chunk at a time and makes no extra +> full copy of the input. This bounds the transform's *added* memory, not total +> memory: if the source itself already holds the whole input (e.g. `readFileSync`), +> that memory is still resident. **Async** iterables (e.g. `fs.createReadStream()`) +> are **fully pre-buffered** into memory before the transform starts, because their +> `.next()` returns a Promise that cannot be awaited inside the synchronous +> callback. For large inputs, prefer a synchronous generator so the transform adds +> no second copy. + **Parameters:** - `script` (string): DataWeave script - `input` (AsyncIterable | Iterable): Streaming input data - `opts` (object, optional): Options - `inputName` (string): Name of input variable (default: "payload") - `mimeType` (string): Input MIME type (default: "application/json") - - `charset` (string | null): Input character encoding + - `charset` (string, optional): Input character encoding - `inputs` (object): Additional input variables **Yields:** `Buffer` chunks as they're produced **Returns:** `StreamingResult` -#### `cleanup(): void` +#### `cleanup(): Promise` -Clean up the global DataWeave runtime instance. Called automatically on process exit. +Clean up the global DataWeave runtime instance. Called automatically on process shutdown via two hooks: `beforeExit` awaits it, so a streaming/transform operation still in flight drains gracefully before the process exits normally; `exit` is a synchronous last-ditch fallback for `process.exit()` and uncaught exceptions — cases where `beforeExit` never fires — and cannot await the drain. Neither hook fires on `SIGTERM`, `SIGINT`, or `SIGKILL` (Node does not emit `exit` for signals), so install your own signal handler that calls `cleanup()` if you need a graceful drain on termination. Called manually, it releases this instance's reference to the native runtime; the shared native isolate is torn down only when the **last** initialized instance in the process is released. When this call releases that final reference, it resolves once native teardown has actually finished, waiting for any still-in-flight streaming/transform operation to drain first; otherwise (other instances remain initialized) it resolves as soon as this instance is released, without draining process-wide work. ```javascript import { cleanup } from '@dataweave/native'; // Manual cleanup (usually not needed) -cleanup(); +await cleanup(); ``` ### Class-Based API @@ -213,13 +237,13 @@ try { const result = dw.run('2 + 2'); console.log(result.getString()); } finally { - dw.cleanup(); + await dw.cleanup(); } ``` **Methods:** - `initialize()`: Initialize the native library -- `cleanup()`: Release native resources +- `cleanup(): Promise`: Release this instance's native resources. When it releases the last initialized instance in the process, it resolves once the shared isolate has finished tearing down (draining any in-flight streaming/transform op first); otherwise it resolves as soon as this instance is released, leaving the isolate live for other instances. - `run(script, inputs?, opts?)`: Same as module-level `run()` - `runStreaming(script, inputs?)`: Same as module-level `runStreaming()` - `runTransform(script, input, opts?)`: Same as module-level `runTransform()` @@ -231,6 +255,7 @@ DataWeave scripts can import external modules using the `resolveModule` option. ```typescript import { DataWeave, composeResolvers, modulesFromDirectory, modulesFromJars } from '@dataweave/native'; +// Inside an async function (uses `await` for modulesFromJars and cleanup()). const dw = new DataWeave({ resolveModule: composeResolvers( modulesFromDirectory('./my-modules'), @@ -238,17 +263,21 @@ const dw = new DataWeave({ ) }); dw.initialize(); - -const result = dw.run(` - %dw 2.0 - import org::company::utils - output application/json - --- - utils::doSomething() -`); - -if (result.success) { - console.log(result.getString()); +try { + const result = dw.run(` + %dw 2.0 + import org::company::utils + output application/json + --- + utils::doSomething() + `); + + if (result.success) { + console.log(result.getString()); + } +} finally { + // Release the engine and resolver closure when done. + await dw.cleanup(); } ``` @@ -366,7 +395,7 @@ console.log(result.getString()); // "300" ```javascript import { runTransform } from '@dataweave/native'; -import { createReadStream, createWriteStream } from 'fs'; +import { readFileSync, createWriteStream } from 'fs'; const script = ` %dw 2.0 @@ -375,9 +404,18 @@ output application/json payload filter $.amount > 1000 `; +// A synchronous generator is consumed on demand: the transform does not make a +// second full copy of the input. Note readFileSync still holds the whole file in +// memory, so this bounds the transform's *added* memory, not total memory -- the +// native read callback is synchronous, so there is no fully-streaming-from-disk +// path (an async createReadStream would instead be pre-buffered in full first). +function* chunked(buf, size = 65536) { + for (let i = 0; i < buf.length; i += size) yield buf.subarray(i, i + size); +} + const generator = runTransform( script, - createReadStream('large-transactions.csv'), + chunked(readFileSync('large-transactions.csv')), { mimeType: 'application/csv' } ); @@ -422,12 +460,17 @@ try { ```javascript try { const generator = runStreaming('invalid syntax'); - for await (const chunk of generator) { - // Process chunks + // Drive next() manually so the terminal { done: true, value: StreamingResult } + // is captured; a `for await` loop would consume it and a later + // generator.return() would give { value: undefined }. + let meta; + while (true) { + const { value, done } = await generator.next(); + if (done) { meta = value; break; } + // Process chunk `value` } - const meta = await generator.return(); - if (!meta.value.success) { - console.error('Streaming error:', meta.value.error); + if (!meta.success) { + console.error('Streaming error:', meta.error); } } catch (err) { console.error('Native error:', err); @@ -440,20 +483,20 @@ The Node.js binding uses **N-API** (Node-API) for C addon integration: - **Thread-safe**: N-API calls are serialized on the Node.js event loop - **Async operations**: Streaming operations yield control to the event loop between chunks -- **No blocking**: Long-running scripts execute on the native side without blocking the event loop +- **No event-loop blocking for streaming**: `runStreaming`/`runTransform` execute on a background worker and yield to the event loop between chunks. Note the **synchronous** `run()` runs native work directly on the calling JS thread and *does* block it until the script completes — use the streaming methods for long-running work you cannot block on. **Important:** Do not share a single `DataWeave` instance across Worker threads. Use the module-level functions (which use a global singleton) or create separate instances per thread. -**Custom module resolvers and Worker threads:** the native layer installs at -most one resolver callback for the whole process lifetime, and it is bound to -the Worker (main thread or a `worker_threads` Worker) that registered it -first — see [External Modules: Multiple Resolvers](docs/external-modules.md#multiple-resolvers-in-one-process). +**Custom module resolvers and Worker threads:** each resolver-backed +`DataWeave` instance's native engine is bound to the thread that created it +(main thread or a `worker_threads` Worker) — see +[External Modules: Multiple Independent Engines](docs/external-modules.md#multiple-independent-engines). Custom-module resolution attempted from any *other* thread is not routed to -that thread's own `resolveModule` callback; it silently falls back to -built-in modules only (custom module paths resolve as "not found" rather than -crashing or hanging). If you need per-Worker custom modules, resolve them on -the thread that first constructs a resolver-backed `DataWeave` instance, or -avoid resolver-backed instances in worker pools altogether. +that engine's `resolveModule` callback; it silently falls back to built-in +modules only (custom module paths resolve as "not found" rather than +crashing or hanging). If you need custom modules on multiple Workers, +construct and use a separate resolver-backed `DataWeave` instance on each +Worker, created on that Worker itself. ## Platform Support @@ -541,12 +584,12 @@ Tests use **Vitest** and cover: - **Buffered execution** (`run`): Best for small scripts with sub-MB outputs - **Streaming execution** (`runStreaming`): Best for large outputs (MB+), reduces memory footprint -- **Bidirectional streaming** (`runTransform`): Best for large inputs and outputs, constant memory usage +- **Bidirectional streaming** (`runTransform`): Best for large outputs; input memory is bounded only with a **synchronous** input iterable (async streams are pre-buffered — see the `runTransform` memory note above) Benchmark (1MB JSON transformation): - `run()`: ~50ms, 2MB peak memory - `runStreaming()`: ~55ms, 500KB peak memory -- `runTransform()`: ~60ms, 256KB peak memory (streaming input) +- `runTransform()`: ~60ms, 256KB peak memory (synchronous input iterable; an async stream is pre-buffered, so peak memory scales with input size) ## See Also diff --git a/native-lib/node/docs/external-modules.md b/native-lib/node/docs/external-modules.md index 6b67ae91..45014a63 100644 --- a/native-lib/node/docs/external-modules.md +++ b/native-lib/node/docs/external-modules.md @@ -7,26 +7,32 @@ DataWeave scripts can import external modules using the `resolveModule` option. ```typescript import { DataWeave, modulesFromMap } from '@dataweave/native'; +// Inside an async function so `await dw.cleanup()` is available. const dw = new DataWeave({ resolveModule: modulesFromMap({ 'org/company/lib.dwl': '%dw 2.0\nfun greet(n) = "Hello " ++ n', }), }); dw.initialize(); - -const result = dw.run(` - %dw 2.0 - import org::company::lib - output application/json - --- - lib::greet("World") -`); -console.log(result.getString()); // "Hello World" +try { + const result = dw.run(` + %dw 2.0 + import org::company::lib + output application/json + --- + lib::greet("World") + `); + console.log(result.getString()); // "Hello World" +} finally { + // Release the engine and the resolver closure; an uncleaned instance retains + // both (see the lifecycle notes below). + await dw.cleanup(); +} ``` **Important:** The module-level convenience functions (`run()`, `runStreaming()`, `runTransform()` exported directly from `@dataweave/native`) operate on a lazily-initialized singleton that takes no constructor options and therefore cannot be configured with `resolveModule` — you **must** construct your own `DataWeave` instance to use external modules, as shown above. -Additionally, external module resolution is currently supported only through `.run()` (the synchronous API). `.runStreaming()` and `.runTransform()` do not yet support external modules and will only have access to built-in modules. +Additionally, external module resolution is currently supported only through `.run()` (the synchronous API). For a resolver-backed engine, `.runStreaming()` and `.runTransform()` execute on a background thread and cannot invoke that engine's `resolveModule` callback — they always resolve only built-in modules, and any custom-module import fails closed (module "not found") rather than crashing or hanging. ## Resolver Factories @@ -37,13 +43,18 @@ In-memory map of module paths to source code: ```typescript import { DataWeave, modulesFromMap } from '@dataweave/native'; +// Inside an async function so `await dw.cleanup()` is available. const dw = new DataWeave({ resolveModule: modulesFromMap({ 'org/test/lib.dwl': '%dw 2.0\nfun foo() = 42', }), }); dw.initialize(); -const result = dw.run('import org::test::lib\n%dw 2.0\n---\nlib::foo()'); +try { + const result = dw.run('%dw 2.0\nimport org::test::lib\n---\nlib::foo()'); +} finally { + await dw.cleanup(); +} ``` Best for: Small, in-memory module sets; testing and development. @@ -55,13 +66,18 @@ Read modules from a directory tree on disk: ```typescript import { DataWeave, modulesFromDirectory } from '@dataweave/native'; +// Inside an async function so `await dw.cleanup()` is available. const dw = new DataWeave({ resolveModule: modulesFromDirectory('./my-modules'), }); dw.initialize(); -// Resolves "org/test/lib.dwl" → reads "./my-modules/org/test/lib.dwl" -const result = dw.run('import org::test::lib\n%dw 2.0\n---\nlib::foo()'); +try { + // Resolves "org/test/lib.dwl" → reads "./my-modules/org/test/lib.dwl" + const result = dw.run('%dw 2.0\nimport org::test::lib\n---\nlib::foo()'); +} finally { + await dw.cleanup(); +} ``` Best for: Development and file-based module repositories. @@ -83,7 +99,11 @@ const dw = new DataWeave({ resolveModule: resolver, }); dw.initialize(); -const result = dw.run('import org::mule::weave::core::Strings\n%dw 2.0\n---\nStrings::capitalize("hello")'); +try { + const result = dw.run('%dw 2.0\nimport org::mule::weave::core::Strings\n---\nStrings::capitalize("hello")'); +} finally { + await dw.cleanup(); +} ``` **Note:** `modulesFromJars()` returns a `Promise` because JAR extraction must complete first. The returned resolver itself is synchronous and can be used repeatedly. @@ -94,6 +114,8 @@ Best for: Packaged dependencies and distributed libraries. Combine multiple resolvers with fallback chain (tries each in order, returns first match): +*Abbreviated fragment — see the first example for the required `try/finally { await dw.cleanup() }` lifecycle.* + ```typescript import { DataWeave, composeResolvers, modulesFromMap, modulesFromDirectory, modulesFromJars } from '@dataweave/native'; @@ -113,7 +135,7 @@ Best for: Layered resolution with fallbacks (overrides, shared libraries, vendor ## How It Works -- **One resolver per process**: The native engine maintains a single resolver per process lifetime. Only the first resolver registered is used; subsequent `DataWeave` instances with different resolvers will silently reuse the first one. +- **Independent engines**: each `DataWeave` instance owns its own native engine, resolver, and script cache; instances with different resolvers coexist with no cross-talk. - **Resolution at compile time**: The resolver is invoked during script compilation, not per execution. - **Synchronous resolution**: The resolver callback must be synchronous (no `async`/`await`, no Promise return). - **Built-in modules**: Built-in modules (CompositeResolver) are always available and work alongside custom resolvers. @@ -125,6 +147,7 @@ Best for: Layered resolution with fallbacks (overrides, shared libraries, vendor When a module cannot be resolved: ```typescript +// Inside an async function so `await dw.cleanup()` is available. const dw = new DataWeave({ resolveModule: modulesFromMap({ // Only 'org/test/lib.dwl' is available @@ -132,15 +155,19 @@ const dw = new DataWeave({ }); dw.initialize(); -const result = dw.run(` - %dw 2.0 - import org::missing::module // Not found - --- - missing::something() -`); +try { + const result = dw.run(` + %dw 2.0 + import org::missing::module // Not found + --- + missing::something() + `); -if (!result.success) { - console.error(result.error); // "Unable to resolve module with identifier ..." + if (!result.success) { + console.error(result.error); // "Unable to resolve module with identifier ..." + } +} finally { + await dw.cleanup(); } ``` @@ -151,92 +178,89 @@ The resolver returns `null`, and the engine reports a compile-time error. When the resolver encounters file system errors (unreadable files, permission denied, etc.), the resolver throws an error. This error is caught internally by the native layer and the callback returns `null` — indistinguishable from "module not found" to the DataWeave compiler: ```typescript +// Inside an async function so `await dw.cleanup()` is available. const dw = new DataWeave({ resolveModule: modulesFromDirectory('./my-modules'), }); dw.initialize(); -const result = dw.run(` - %dw 2.0 - import org::test::lib - --- - lib::foo() -`); - -if (!result.success) { - // result.error is the same generic message as "module not found": - console.error(result.error); // "Unable to resolve module with identifier ..." - // The actual error details (permissions, encoding, etc.) are not available - // in the result object; see "Debugging" below for how to surface them. +try { + const result = dw.run(` + %dw 2.0 + import org::test::lib + --- + lib::foo() + `); + + if (!result.success) { + // result.error is the same generic message as "module not found": + console.error(result.error); // "Unable to resolve module with identifier ..." + // The actual error details (permissions, encoding, etc.) are not available + // in the result object; see "Debugging" below for how to surface them. + } +} finally { + await dw.cleanup(); } ``` **Debugging:** By default, a resolver failure logs only a fixed, content-free diagnostic line to stderr — the actual exception message and stack are suppressed, since they can carry resolver-controlled data (module source, credentials, filesystem paths). To see the detailed message and stack for diagnosing a failing resolver (e.g., directory does not exist, file unreadable due to permissions), set `DATAWEAVE_RESOLVER_DEBUG=1` in the process environment before running. Only enable this in a trusted debugging context, since the detailed output may expose sensitive resolver-controlled data. -### Multiple Resolvers in One Process +### Multiple Independent Engines -If you construct multiple `DataWeave` instances with different resolvers in the same process: +Each `DataWeave` instance owns its own native engine, resolver, and script +cache. You can construct as many resolver-backed instances as you want in the +same process — each one only ever resolves its own modules, with no +cross-talk between instances: ```typescript -const dw1 = new DataWeave({ - resolveModule: modulesFromMap({ 'a.dwl': '...' }), -}); -dw1.initialize(); - -const dw2 = new DataWeave({ - resolveModule: modulesFromMap({ 'b.dwl': '...' }), -}); -dw2.initialize(); // Only loads/ref-counts the native library — does NOT register a resolver - -dw1.run('...'); // First resolver-backed run() in the process: installs dw1's resolver -dw2.run('...'); // Logs warning, silently reuses dw1's resolver instead of dw2's +async function example() { + const dw1 = new DataWeave({ + resolveModule: modulesFromMap({ 'a.dwl': '...' }), + }); + dw1.initialize(); -// Both dw1 and dw2 use dw1's resolver (only 'a.dwl' is available) -``` + const dw2 = new DataWeave({ + resolveModule: modulesFromMap({ 'b.dwl': '...' }), + }); + dw2.initialize(); -**The rule is "first resolver-backed `run()` wins," not "first `initialize()` wins."** -`initialize()` only loads and ref-counts the native library; the resolver -itself is registered lazily, on whichever instance's `run()` executes first -with a resolver configured. If `dw2.run()` happens to execute before -`dw1.run()` — even though `dw1.initialize()` ran first — `dw2`'s resolver -wins instead. - -**Workaround:** Use `composeResolvers()` to combine all modules into a single resolver: - -```typescript -const resolver = composeResolvers( - modulesFromMap({ 'a.dwl': '...' }), - modulesFromMap({ 'b.dwl': '...' }) -); - -const dw1 = new DataWeave({ resolveModule: resolver }); -dw1.initialize(); - -const dw2 = new DataWeave({ resolveModule: resolver }); -dw2.initialize(); // Both use the same resolver + try { + dw1.run('...'); // Only 'a.dwl' is available to dw1 + dw2.run('...'); // Only 'b.dwl' is available to dw2 — dw1's modules are not visible here + } finally { + await dw1.cleanup(); + await dw2.cleanup(); + } +} ``` -**Worker threads:** the same one-resolver-per-process rule applies across -`worker_threads` Workers, not just across instances on one thread. The -resolver callback is additionally bound to the specific thread that first -registered it. A resolver-backed `DataWeave` constructed and initialized on a -Worker other than the one that registered the process's resolver will not -have its `resolveModule` invoked at all — custom module paths resolve as "not -found" (falling back to built-ins only) rather than crashing. There is -currently no supported way to run distinct custom-module resolvers on -different Workers in the same process; either resolve modules on the thread -that owns the process's resolver, or avoid resolver-backed instances in -worker pools. - -**Concurrent resolver-backed runs across Workers are unsupported and -memory-unsafe.** Beyond the "not found" fallback described above, calling a -resolver-backed `run()` concurrently from more than one Worker is not just -unsupported behavior — it is a memory-safety hazard. The native layer tracks -in-flight resolver results in unsynchronized, process-global state, and one -Worker's cleanup can free memory another Worker's concurrent call is still -using. Restrict resolver-backed execution to a single thread (or fully -serialize resolver-backed calls across Workers) until a future release -isolates per-instance engine state. +**`cleanup()` is required for every instance.** Each `DataWeave` instance's +engine is tracked in a native registry keyed by handle. `cleanup()` destroys +the engine and removes its registry entry; an instance that is never +`cleanup()`'d keeps its engine (and the JS `resolveModule` closure it holds a +reference to) alive for the lifetime of the process, even if the `DataWeave` +object itself is garbage-collected on the JS side. Always `cleanup()` in a +`finally` block, as shown throughout this document. + +`composeResolvers()` is not a workaround for any resolver-sharing limitation +— each engine already has its own resolver. It's simply a layering tool for +building one resolver out of several fallback sources (overrides, then a +shared directory, then vendor JARs); see [composeResolvers](#composeresolvers) +above. + +**Worker threads and thread ownership:** each resolver-backed engine is bound +to the thread that created it (the thread that called `new DataWeave(...)` +and `initialize()` with a `resolveModule` configured). Only that thread's +synchronous `run()` calls can invoke the engine's `resolveModule` callback. +`runStreaming()` and `runTransform()` execute on a background thread even +when called from the owner thread, so they can never invoke that engine's +resolver — nor can `run()` calls made from any other `worker_threads` Worker. +In all of these cases the engine fails closed: custom module paths resolve as +"not found" (falling back to built-ins only) rather than crashing or hanging. +There is no supported way to invoke one engine's resolver from a thread other +than the one that created it; if you need custom modules on multiple +Workers, construct and use a separate resolver-backed `DataWeave` instance +on each Worker. ## Security / Trust Model @@ -268,6 +292,8 @@ Future releases may add `npm run dw-deps` for automatic resolution. Check your p Then pass JAR paths to `modulesFromJars()`: +*Abbreviated fragment — see the first example for the required `try/finally { await dw.cleanup() }` lifecycle.* + ```typescript const resolver = await modulesFromJars([ './libs/dw-lib-1.0.jar', @@ -319,7 +345,7 @@ async function main() { console.error('Error:', result.error); } } finally { - dw.cleanup(); + await dw.cleanup(); } } diff --git a/native-lib/node/src/addon.c b/native-lib/node/src/addon.c index 5aa31535..2861a740 100644 --- a/native-lib/node/src/addon.c +++ b/native-lib/node/src/addon.c @@ -3,6 +3,7 @@ #include #include #include +#include // GraalVM function pointer types typedef int (*graal_create_isolate_fn)(void*, void**, void**); @@ -13,21 +14,19 @@ typedef void* (*run_script_fn)(void*, const char*, const char*); typedef void (*free_cstring_fn)(void*, void*); typedef int (*write_callback_t)(void* ctx, const char* buf, int len); typedef int (*read_callback_t)(void* ctx, char* buf, int buf_size); -typedef char* (*resolve_module_callback_t)(void* thread, const char* module_path); +typedef char* (*resolve_module_callback_t)(void* thread, void* ctx, const char* module_path); typedef void* (*run_script_callback_fn)(void*, const char*, const char*, write_callback_t, void*); typedef void* (*run_script_input_output_callback_fn)(void*, const char*, const char*, const char*, const char*, const char*, read_callback_t, write_callback_t, void*); -// Resolver-aware entrypoint types -// NOTE: run_script_with_resolver has no mimeType parameter on the native side -// (NativeLib.runScriptWithResolver(thread, script, inputsJson, resolverCallback) -// delegates to ScriptRuntime.run(script, inputsJson), which infers/hardcodes -// output mime type internally). The JS-facing mimeType argument is accepted -// for API symmetry with other entrypoints but is NOT forwarded across the FFI -// boundary — passing it here would misalign the native call's argument -// registers and corrupt the callback function pointer. -typedef char* (*run_script_with_resolver_fn)(void*, const char*, const char*, resolve_module_callback_t); -typedef void* (*run_script_callback_with_resolver_fn)(void*, const char*, const char*, const char*, write_callback_t, void*, resolve_module_callback_t); -typedef void* (*run_script_input_output_callback_with_resolver_fn)(void*, const char*, const char*, const char*, const char*, const char*, read_callback_t, write_callback_t, void*, resolve_module_callback_t); +// Per-engine entrypoint types. Handles are Java long values and MUST be C +// long long everywhere (plain long is 32-bit on Windows LLP64 and would +// truncate a 64-bit handle). +typedef long long (*create_engine_fn)(void*); +typedef long long (*create_engine_with_resolver_fn)(void*, resolve_module_callback_t, void*); +typedef void (*destroy_engine_fn)(void*, long long); +typedef void* (*run_script_engine_fn)(void*, long long, const char*, const char*); +typedef void* (*run_script_callback_engine_fn)(void*, long long, const char*, const char*, write_callback_t, void*); +typedef void* (*run_script_input_output_callback_engine_fn)(void*, long long, const char*, const char*, const char*, const char*, const char*, read_callback_t, write_callback_t, void*); // Global state static uv_lib_t g_lib; @@ -54,14 +53,28 @@ static free_cstring_fn fn_free_cstring = NULL; static run_script_callback_fn fn_run_script_callback = NULL; static run_script_input_output_callback_fn fn_run_script_input_output_callback = NULL; -// Resolver-aware entrypoints -static run_script_with_resolver_fn fn_run_script_with_resolver = NULL; -static run_script_callback_with_resolver_fn fn_run_script_callback_with_resolver = NULL; -static run_script_input_output_callback_with_resolver_fn fn_run_script_input_output_callback_with_resolver = NULL; +// Per-engine entrypoints +static create_engine_fn fn_create_engine = NULL; +static create_engine_with_resolver_fn fn_create_engine_with_resolver = NULL; +static destroy_engine_fn fn_destroy_engine = NULL; +static run_script_engine_fn fn_run_script_engine = NULL; +static run_script_callback_engine_fn fn_run_script_callback_engine = NULL; +static run_script_input_output_callback_engine_fn fn_run_script_input_output_callback_engine = NULL; + +// A single run may trigger resolve_module_callback multiple times (one script +// can import several modules). Native copies each returned buffer immediately, +// but the copy is made *after* our callback returns — we don't get a per-call +// "done freeing" signal, only "the whole run finished". So track every buffer +// allocated during one run and free them all once the native call returns. +typedef struct resolver_result_node { + char* buf; + struct resolver_result_node* next; +} resolver_result_node_t; -// Resolver bridge state (one resolver per process). +// Per-engine resolver bridge: one node per resolver-backed engine, passed to +// Java as the callback ctx word and forwarded back to resolve_module_callback. // -// Unlike the streaming/transform entrypoints, runWithResolver's native call +// Unlike the streaming/transform entrypoints, runScriptEngine's native call // executes synchronously on the very thread that invoked it from JS — no // background uv_thread is spawned. So when native code calls back into // resolve_module_callback(), we are already on the correct (JS) thread and @@ -70,51 +83,373 @@ static run_script_input_output_callback_with_resolver_fn fn_run_script_input_out // caller on a condition variable until it's serviced — but if the caller // *is* the JS thread, it can never service its own queued item, causing a // deadlock (a real bug fixed in this codebase — see Task 11 report). -static napi_env g_resolver_env = NULL; -static napi_ref g_resolver_ref = NULL; - -// The OS thread that first installed the resolver (see napi_run_with_resolver -// below). ScriptRuntime's engine is a process-wide singleton, so once a -// resolver is installed, resolve_module_callback() can be reached from ANY -// entrypoint that later compiles a script against that shared engine — -// including runScriptStreaming/runScriptTransform, whose native calls run on -// a background uv_thread (see streaming_thread_fn/transform_thread_fn), not -// the JS thread. napi_env/napi_ref are thread-affine; calling into them from -// a thread other than the one that created them is undefined behavior. We -// record the owning thread here so resolve_module_callback can detect the -// mismatch and fail closed (return "not found") instead of crashing. -static uv_thread_t g_resolver_thread; - -// A single runWithResolver call may trigger resolve_module_callback multiple -// times (one script can import several modules). Native copies each -// returned buffer immediately, but the copy is made *after* our callback -// returns — we don't get a per-call "done freeing" signal, only "the whole -// run finished". So track every buffer allocated during one call and free -// them all once fn_run_script_with_resolver returns. -typedef struct resolver_result_node { - char* buf; - struct resolver_result_node* next; -} resolver_result_node_t; -static resolver_result_node_t* g_resolver_results = NULL; - -static void resolver_results_track(char* buf) { - if (buf == NULL) return; +// +// napi_env/napi_ref are thread-affine; each bridge records the JS thread that +// created it (owner) so resolve_module_callback can detect a mismatch — e.g. a +// streamed/transform custom-module lookup arriving on the background uv_thread +// — and fail closed (return "not found") instead of crashing. +typedef struct engine_bridge { + long long handle; + napi_env env; + napi_ref resolver_js; // NULL => resolver-less engine (no bridge created) + uv_thread_t owner; // JS thread that created and must run this engine + resolver_result_node_t* results; // buffers to free after each run on this engine + // Lifecycle accounting, mutated only under g_mutex. A streaming/transform op + // runs the native call on a background uv_thread that can still call back into + // resolve_module_callback with this bridge as ctx, so the bridge must outlive + // every in-flight op. in_flight counts ops that can still dereference this + // bridge; destroy_pending marks that destroyEngine ran while in_flight > 0 and + // freeing was deferred to the last op draining on the owner thread. + int in_flight; + bool destroy_pending; + // True when a destroy (via destroyEngine OR the env cleanup hook) was + // deferred because in_flight > 0; gates the deferred fn_destroy_engine + // registry removal in bridge_end_op. round-9 (#1) introduced this for the + // destroyEngine path; round-10 (#1) extended it to bridge_env_cleanup, which + // must ALSO remove the Java registry entry when its free is deferred -- + // otherwise a resolver-backed engine's ScriptRuntime is left registered with + // a CallbackWeaveResourceResolver whose ctx points at the freed bridge (UAF). + bool deferred_registry_remove; + struct engine_bridge* next; +} engine_bridge_t; +static engine_bridge_t* g_bridges = NULL; // linked list, guarded by g_mutex + +// One record per napi_env that has ever taken an init reference (via +// initialize()). init_refs is that env's net initialize()-minus-cleanup() +// balance. Created lazily on the env's first initialize(); registers exactly +// one env-death hook (env_init_cleanup) at creation; freed by that hook when +// its env dies (after releasing every reference the env still holds). All +// fields mutated ONLY under g_mutex. +// +// INVARIANT: g_ref_count == sum of init_refs over all records in g_env_recs. +// This is the round-13 (#5) fix: the isolate's reference count is owned per +// env, so an abandoned env (or a raw multi-engine-per-initialize() consumer) +// can only release the references IT holds -- it can never drive g_ref_count +// to zero and tear the isolate down while ANOTHER env's engines are live. +typedef struct env_init_rec { + napi_env env; + int init_refs; + struct env_init_rec* next; +} env_init_rec_t; +static env_init_rec_t* g_env_recs = NULL; // linked list, guarded by g_mutex + +// --- Teardown-vs-active-ops coordination (deadlock fix) --- +// +// napi_cleanup's last-release path used to synchronously join a thread that +// calls graal_tear_down_isolate(), which blocks until every GraalVM-attached +// thread detaches. A runStreaming()/runTransform() background worker stays +// attached and can be mid-delivery in napi_call_threadsafe_function(..., +// napi_tsfn_blocking), which needs the JS thread to run its callback -- but +// the JS thread is the one blocked in the join. g_active_ops tracks every +// in-flight streaming/transform op (resolver-backed or not, since teardown +// blocks on ANY attached worker) so napi_cleanup can wait for them to drain +// on a dedicated thread instead of blocking the calling JS thread. +static int g_active_ops = 0; +// Teardown lifecycle, all transitions under g_mutex: +// NONE -> no teardown queued or in progress. +// PENDING_WAIT -> napi_cleanup Case 5 queued a teardown; the waiter thread is +// blocked waiting for g_active_ops to drain. The isolate is +// STILL LIVE and un-torn-down here, so a fresh initialize() +// may ADOPT it (cancel the teardown) instead of blocking the +// JS thread -- this is the round-5 deadlock fix. +// TEARING_DOWN -> the waiter has passed the point of no return and is calling +// graal_tear_down_isolate(). Adoption is unsafe; initialize() +// must block here, which is deadlock-free because g_active_ops +// is already 0 (nothing depends on the JS event loop). +typedef enum { + TEARDOWN_NONE = 0, + TEARDOWN_PENDING_WAIT, + TEARDOWN_TEARING_DOWN, +} teardown_state_t; +static teardown_state_t g_teardown_state = TEARDOWN_NONE; +// Set by an adopting initialize() to tell the waiter thread to abort its +// queued teardown and leave the live isolate intact. Read/reset by the waiter. +static bool g_teardown_cancelled = false; +// Round-14 (#2/#3): set under g_mutex when a reached-zero teardown could NOT be +// carried out (teardown-waiter alloc/spawn failed, or cleanup_thread_fn attach +// failed) and the isolate was therefore left LIVE with g_ref_count == 0 and no +// pending teardown. This is a RETRY SIGNAL, not an ownership reference: +// g_ref_count stays 0, so the invariant g_ref_count == sum(init_refs) is +// unaffected. It is cleared when the isolate is (a) actually torn down by a +// retry, or (b) adopted by a later initialize() (a new owner wants it kept). +// While set with g_active_ops > 0, the op-completion drain point retries the +// teardown once ops reach 0 (retry_stranded_teardown_locked). +static bool g_teardown_needed = false; +static uv_cond_t g_teardown_cond; + +// One node per cleanup() call that arrived while a teardown was already +// pending. napi_env/napi_deferred/napi_threadsafe_function are thread-affine, +// so a second cleanup() call from a different Worker's env cannot have its +// promise resolved via another env's tsfn -- each waiting caller gets its own +// node, created on its own env, resolved by the waiter thread on completion. +typedef struct teardown_waiter { + napi_env env; + napi_deferred deferred; + napi_threadsafe_function tsfn; + struct teardown_waiter* next; +} teardown_waiter_t; +static teardown_waiter_t* g_teardown_waiters = NULL; // linked list, guarded by g_mutex + +// Returns true if the buffer is now tracked (or there was nothing to track). +// Returns false only when a buffer was supplied but the tracking node could +// not be allocated — in that case the caller owns `buf` again and MUST free +// it itself, since it will never be reachable from b->results. +static bool resolver_results_track(engine_bridge_t* b, char* buf) { + if (b == NULL || buf == NULL) return true; resolver_result_node_t* node = (resolver_result_node_t*)malloc(sizeof(resolver_result_node_t)); - if (node == NULL) return; // Leak the buffer rather than crash; best-effort tracking. + if (node == NULL) return false; // OOM: caller must free buf to avoid leaking it untracked. node->buf = buf; - node->next = g_resolver_results; - g_resolver_results = node; + node->next = b->results; + b->results = node; + return true; } -static void resolver_results_free_all(void) { - resolver_result_node_t* node = g_resolver_results; +static void resolver_results_free_all(engine_bridge_t* b) { + if (b == NULL) return; + resolver_result_node_t* node = b->results; while (node != NULL) { resolver_result_node_t* next = node->next; free(node->buf); free(node); node = next; } - g_resolver_results = NULL; + b->results = NULL; +} + +// Call under g_mutex. +static engine_bridge_t* bridge_find(long long handle) { + for (engine_bridge_t* b = g_bridges; b != NULL; b = b->next) { + if (b->handle == handle) return b; + } + return NULL; +} + +// Find this env's init record, or NULL. Caller MUST hold g_mutex. +static env_init_rec_t* env_init_rec_find_locked(napi_env env) { + for (env_init_rec_t* r = g_env_recs; r != NULL; r = r->next) { + if (r->env == env) return r; + } + return NULL; +} + +// Find-or-create this env's init record and increment its init_refs. Sets +// *is_new = true iff a record was just allocated (the caller must then register +// the env-death hook on its own thread). Returns the record, or NULL only on +// calloc failure (caller must NOT bump g_ref_count in that case). Caller MUST +// hold g_mutex. +static env_init_rec_t* env_init_rec_acquire_locked(napi_env env, bool* is_new) { + *is_new = false; + env_init_rec_t* r = env_init_rec_find_locked(env); + if (r == NULL) { + r = (env_init_rec_t*)calloc(1, sizeof(env_init_rec_t)); + if (r == NULL) return NULL; + r->env = env; + r->init_refs = 0; + r->next = g_env_recs; + g_env_recs = r; + *is_new = true; + } + r->init_refs++; + return r; +} + +// Sum of live per-env init references. Caller holds g_mutex. Establishes the +// value g_ref_count must equal (invariant g_ref_count == sum of init_refs); used +// to restore g_ref_count coherently when a deferred teardown cannot be spawned. +static int env_init_refs_total_locked(void) { + int total = 0; + for (env_init_rec_t* r = g_env_recs; r != NULL; r = r->next) total += r->init_refs; + return total; +} + +// Fully dispose of a bridge: delete its napi_ref (if the owning env is still +// alive), free tracked result buffers, free the struct. napi_ref/napi_env are +// thread-affine, so napi_delete_reference MUST run on the bridge's owner +// thread (the JS/Worker thread that created it) while that env is still +// alive -- `env_still_alive` must be false whenever the caller knows the +// owning env is tearing down/dead (e.g. the env == NULL sentinel path in +// call_js_write/call_js_transform_write), even though b->env itself is never +// cleared and stays non-NULL. When env_still_alive is false the napi_ref is +// simply skipped -- Node auto-reclaims refs when their env is destroyed, so +// nothing leaks. The bridge must already be unlinked from g_bridges. Do NOT +// hold g_mutex across this call — it invokes N-API. Callers that freed a +// bridge *early* (destroyEngine / streaming completion) must first drop the +// env cleanup hook via napi_remove_env_cleanup_hook so Node never invokes it +// on freed memory; the hook path itself (bridge_env_cleanup) must not remove +// itself and calls this directly. +// `do_registry_remove` is true when the caller must remove the Java registry +// entry (fn_destroy_engine) for this handle before freeing the record: the +// immediate destroyEngine path, or the deferred drain of either destroyEngine +// (round-9 #1) or the env cleanup hook (round-10 #1). fn_destroy_engine is +// called at most once per handle because destroyEngine and bridge_env_cleanup +// are mutually exclusive (destroyEngine removes the hook). It runs on whichever +// thread finalizes (the owner JS thread from the completion sentinel, +// destroyEngine's thread, or the env-cleanup hook thread); fn_destroy_engine +// attaches its own isolate thread, so it is not JS-thread-affine. Must be +// called WITHOUT g_mutex held (it enters GraalVM and, for env_still_alive, +// calls N-API). +// #3 (round 12): the isolate-touching registry removal. Takes a TRANSIENT +// g_active_ops reservation so graal_tear_down_isolate() cannot run across the +// attach. The teardown-state check and the g_active_ops++ are ONE critical +// section: no teardown path can interleave between "isolate is live" and +// "reservation taken". Callable from any thread NOT holding g_mutex. +static void bridge_finalize_registry(engine_bridge_t* b) { + if (b == NULL || fn_destroy_engine == NULL) return; + uv_mutex_lock(&g_mutex); + // If the waiter already committed to physical teardown (TEARING_DOWN) or the + // isolate is already gone, the Java registry died/dies with it -- nothing to + // remove, and attaching would race graal_tear_down_isolate. Skip. Because + // the waiter publishes TEARING_DOWN (and Case 4 holds g_mutex across its + // g_active_ops==0 check + teardown) under this same lock, this check plus the + // increment below cannot be split by a teardown. + if (g_teardown_state == TEARDOWN_TEARING_DOWN || g_isolate == NULL) { + uv_mutex_unlock(&g_mutex); + return; + } + g_active_ops++; // pins the live isolate against teardown for this attach + uv_mutex_unlock(&g_mutex); + + void* thread = NULL; + if (fn_attach_thread(g_isolate, &thread) == 0) { + fn_destroy_engine(thread, b->handle); + fn_detach_thread(thread); + } + + // Verbatim g_active_ops release pattern. + uv_mutex_lock(&g_mutex); + g_active_ops--; + uv_cond_broadcast(&g_teardown_cond); + uv_mutex_unlock(&g_mutex); +} + +// The non-isolate finalize phase: delete the resolver napi_ref (owner JS thread +// only, and only while its env is alive -- resolver-gated), free tracked result +// buffers, free the record. Touches no GraalVM isolate state, so it is safe to +// run after the g_active_ops reservation above is released. +static void bridge_finalize_free(engine_bridge_t* b, bool env_still_alive) { + if (b == NULL) return; + if (env_still_alive && b->resolver_js != NULL && b->env != NULL) { + napi_delete_reference(b->env, b->resolver_js); + } + resolver_results_free_all(b); + free(b); +} + +// Thin wrapper preserving the original signature and every call site. Registry +// removal (if requested) runs first under its transient reservation, then the +// record is freed. +static void bridge_finalize(engine_bridge_t* b, bool env_still_alive, bool do_registry_remove) { + if (b == NULL) return; + if (do_registry_remove) bridge_finalize_registry(b); + bridge_finalize_free(b, env_still_alive); +} + +// Env cleanup hook (F2): registered per resolver-backed bridge at creation via +// napi_add_env_cleanup_hook, so each Worker/main env disposes its OWN bridges on +// its OWN thread when that env tears down — instead of napi_cleanup deleting +// refs from whichever thread happens to release the last DataWeave instance, +// which is undefined behavior for thread-affine napi_env/napi_ref. Runs on the +// owner thread with the env still alive, which is exactly where napi_ref deletion +// is legal. +static void bridge_env_cleanup(void* arg) { + engine_bridge_t* b = (engine_bridge_t*)arg; + if (b == NULL) return; + + uv_mutex_lock(&g_mutex); + // Unlink from g_bridges if still present (destroyEngine may have already + // unlinked it while deferring a free — see below). + engine_bridge_t** pp = &g_bridges; + while (*pp != NULL) { + if (*pp == b) { *pp = b->next; break; } + pp = &(*pp)->next; + } + // An in-flight streaming/transform op holds a live threadsafe function that + // keeps this env's event loop alive, so the env should never tear down while + // in_flight > 0. Guard defensively anyway: mark destroy_pending and let the + // op's completion path drain and finalize it (do NOT finalize here, the op's + // background thread could still dereference this bridge). + if (b->in_flight > 0) { + b->destroy_pending = true; + // round-10 (#1): the draining op must ALSO remove the Java registry + // entry (like destroyEngine's deferred path), or the resolver engine's + // ScriptRuntime is left registered with a resolver ctx pointing at the + // freed bridge. Set the deferred-registry-removal flag here. + b->deferred_registry_remove = true; + uv_mutex_unlock(&g_mutex); + return; + } + // in_flight == 0: finalize now. The abandoned engine's init reference is + // NOT released here (round-13 #5) -- it is released by the env-death hook + // (env_init_cleanup) when this env dies, which owns the whole per-env + // balance. There is nothing left to do under the lock before unlocking in + // this branch. bridge_finalize_registry inside finalize checks teardown + // state under g_mutex, so a torn-down/TEARING_DOWN isolate makes the + // registry removal a correct no-op (the Java registry died with the + // isolate). + uv_mutex_unlock(&g_mutex); + + // We are inside Node's invocation of this hook, so we must not (and need not) + // call napi_remove_env_cleanup_hook for ourselves here. The env is still + // alive here -- that is the whole point of this hook's design (see above) -- + // so the napi_ref deletion in bridge_finalize is legal. + // round-10 (#1): remove the Java registry entry too (do_registry_remove=true). + // This hook only ever fires for a resolver-backed engine that was never + // passed to destroyEngine (destroyEngine removes this hook), so its + // initialize() ref was never released either -> the isolate is still live + // and fn_destroy_engine's fresh-thread attach is legal (bridge_finalize + // guards on g_isolate for the main-env-after-isolate-teardown corner). Not + // removing it would leave a CallbackWeaveResourceResolver whose ctx is the + // freed bridge -> UAF on a later invocation of this handle. + bridge_finalize(b, /*env_still_alive=*/true, /*do_registry_remove=*/true); +} + +// Increment this engine's in_flight while g_mutex is ALREADY held. Used by the +// run/streaming/transform admission paths so the per-engine pin is taken in the +// SAME critical section as the g_active_ops reservation and the lifecycle check +// -- closing the round-11 window where a concurrent destroyEngine could observe +// in_flight == 0 and free the bridge under an already-admitted op. Returns the +// record, or NULL for an unknown handle (nothing to pin; the worker/native call +// surfaces "Unknown engine handle"). Caller MUST hold g_mutex. +static engine_bridge_t* bridge_begin_op_locked(long long handle) { + engine_bridge_t* b = bridge_find(handle); + if (b != NULL) b->in_flight++; + return b; +} + +// A streaming/transform/run op marks one op in flight on the engine's record so +// the record (and, for resolver-backed engines, its napi_ref) cannot be freed +// while the background uv_thread runs -- and, since round-9 (#1), so that +// destroyEngine defers the Java registry removal until this op drains. Every +// engine (resolver-backed or resolver-less) now has a record, so +// bridge_begin_op_locked returns a non-NULL pointer for any known handle; the +// completion sentinel MUST call bridge_end_op on it to balance in_flight and +// run any deferred destroy. Returns NULL only for an unknown handle (nothing to +// protect, no bridge_end_op needed). The returned pointer is stable for the +// op's lifetime because in_flight > 0 blocks both destroyEngine and the env +// cleanup hook from freeing the record. Since round-11 (#2), every call site +// takes the pin atomically with its g_mutex-guarded admission check via +// bridge_begin_op_locked directly (no self-locking wrapper) -- see +// napi_run_script_streaming_engine / napi_run_script_transform_engine. + +// End a streaming/transform op. Runs on the owner (JS) thread from the completion +// sentinel. If destroyEngine (or the env cleanup hook) ran while this op was in +// flight, it deferred the free — already unlinked from g_bridges — so the last op +// to drain finalizes the bridge here, on the legal (owner) thread. `env_still_alive` +// must be false when the caller is running the env == NULL sentinel path (the +// owning env is tearing down/dead), so a finalize triggered from here does not +// call napi_delete_reference on a dead env. +static void bridge_end_op(engine_bridge_t* b, bool env_still_alive) { + if (b == NULL) return; + uv_mutex_lock(&g_mutex); + b->in_flight--; + bool finalize = (b->destroy_pending && b->in_flight == 0); + bool remove_registry = finalize && b->deferred_registry_remove; + uv_mutex_unlock(&g_mutex); + // remove_registry is true when either destroyEngine (round-9 #1) or the env + // cleanup hook (round-10 #1) deferred the registry removal while this op was + // in flight; the draining op performs it exactly once here. bridge_finalize + // guards the call on g_isolate, so a teardown that raced ahead is a no-op. + if (finalize) bridge_finalize(b, env_still_alive, /*do_registry_remove=*/remove_registry); } // --- Initialization --- @@ -145,15 +480,16 @@ static void init_thread_fn(void* arg) { uv_dlsym(&g_lib, "run_script_callback", (void**)&fn_run_script_callback); uv_dlsym(&g_lib, "run_script_input_output_callback", (void**)&fn_run_script_input_output_callback); - // Load resolver-aware entrypoints (optional - newer symbols) - uv_dlsym(&g_lib, "run_script_with_resolver", (void**)&fn_run_script_with_resolver); - // fn_run_script_callback_with_resolver / fn_run_script_input_output_callback_with_resolver - // are resolved here but intentionally never called from this file. Wiring them into - // runScriptStreaming/runScriptTransform would put the resolver callback on a background - // uv_thread, which is unsafe for the same reason resolve_module_callback() above guards - // against cross-thread napi calls — do not wire these up without solving that hazard first. - uv_dlsym(&g_lib, "run_script_callback_with_resolver", (void**)&fn_run_script_callback_with_resolver); - uv_dlsym(&g_lib, "run_script_input_output_callback_with_resolver", (void**)&fn_run_script_input_output_callback_with_resolver); + // Load per-engine entrypoints. Every initialize() call creates an engine via + // create_engine/create_engine_with_resolver (see dataweave.ts), so these are + // load-time required, not optional, even though they are newer than the + // legacy singleton symbols above. + uv_dlsym(&g_lib, "create_engine", (void**)&fn_create_engine); + uv_dlsym(&g_lib, "create_engine_with_resolver", (void**)&fn_create_engine_with_resolver); + uv_dlsym(&g_lib, "destroy_engine", (void**)&fn_destroy_engine); + uv_dlsym(&g_lib, "run_script_engine", (void**)&fn_run_script_engine); + uv_dlsym(&g_lib, "run_script_callback_engine", (void**)&fn_run_script_callback_engine); + uv_dlsym(&g_lib, "run_script_input_output_callback_engine", (void**)&fn_run_script_input_output_callback_engine); if (!fn_create_isolate || !fn_run_script || !fn_free_cstring) { snprintf(args->error, sizeof(args->error), "Missing required symbols in library"); @@ -161,6 +497,21 @@ static void init_thread_fn(void* arg) { return; } + // Fail fast, with a clear message, if the loaded dwlib predates the + // per-engine ABI (W-23692110). Without this check, the library would load + // "successfully" here and every initialize() call would still fail later + // deep inside createEngine()/createEngineWithResolver() with a confusing + // "not available in native library" error instead of this one. + if (!fn_create_engine || !fn_create_engine_with_resolver || !fn_destroy_engine || + !fn_run_script_engine || !fn_run_script_callback_engine || + !fn_run_script_input_output_callback_engine) { + snprintf(args->error, sizeof(args->error), + "dwlib is missing required per-engine symbols (expected in dwlib " + "built with W-23692110 or later) - rebuild/upgrade the native library"); + args->result = -2; + return; + } + void* boot_thread = NULL; rc = fn_create_isolate(NULL, &g_isolate, &boot_thread); if (rc != 0) { @@ -184,6 +535,50 @@ static void init_thread_fn(void* arg) { args->result = 0; } +// Forward declaration: the env-death hook that reclaims an abandoned env's +// init references. Defined below (round-13 #5); registered here (in +// env_init_acquire_and_hook) because napi_add_env_cleanup_hook is only legal +// while the env is alive on its own JS thread, which napi_initialize is. +static void env_init_cleanup(void* arg); // defined below (round-13 #5) + +// Acquire one init reference for `env` under g_mutex, registering the env-death +// hook on first use. Returns true on success (caller then does g_ref_count++); +// on failure the caller must NOT bump g_ref_count -- it unlocks and throws. +// Caller MUST hold g_mutex; this function keeps it held on success and on the +// calloc-failure return. On hook-registration failure it rolls back the +// just-acquired init_refs (freeing the record if it drops to 0) so no orphan +// record without a death hook survives. +static bool env_init_acquire_and_hook(napi_env env) { + bool is_new = false; + env_init_rec_t* rec = env_init_rec_acquire_locked(env, &is_new); + if (rec == NULL) return false; // calloc failed + if (is_new) { + napi_status hs = napi_add_env_cleanup_hook(env, env_init_cleanup, rec); + if (hs != napi_ok) { + // Roll back: this record has no death hook, so its references would + // never be reclaimed. Drop the one we just took; free if now empty. + rec->init_refs--; + if (rec->init_refs == 0) { + env_init_rec_t** pp = &g_env_recs; + while (*pp != NULL) { if (*pp == rec) { *pp = rec->next; break; } pp = &(*pp)->next; } + free(rec); + } + return false; + } + } + return true; +} + +// Forward declaration: tears down g_isolate on a dedicated attached thread. +// Defined below; used here (napi_initialize's create-path acquire-failure +// recovery) and further down by isolate_ref_release_n_locked. +static void cleanup_thread_fn(void* arg); + +// Forward declaration: retries a stranded teardown (round-14 #2/#3). Defined +// further below; used by the streaming/transform op-completion drain points, +// which run earlier in this file than the definition. +static void retry_stranded_teardown_locked(void); + static napi_value napi_initialize(napi_env env, napi_callback_info info) { size_t argc = 1; napi_value argv[1]; @@ -199,8 +594,82 @@ static napi_value napi_initialize(napi_env env, napi_callback_info info) { napi_get_value_string_utf8(env, argv[0], lib_path, sizeof(lib_path), &len); uv_mutex_lock(&g_mutex); + + // A prior last-release could not tear the isolate down and armed the retry + // signal (review #6 #3/#4). Because retries otherwise fire only at op + // completion (the streaming/transform drains), a zero-op stranded isolate + // would never be reclaimed and the adoption/fast paths below would silently + // discard the pending teardown (review #6 #5). Drive the pending teardown to + // completion here first: on success g_isolate/g_initialized are cleared and we + // build a fresh isolate below; on repeated failure the live isolate is adopted + // by the fast path (safe -- the teardown was resource reclamation, not a + // malfunction). No-ops cheaply when nothing is stranded (flag clear -> return). + retry_stranded_teardown_locked(); + + // After the retry above, a PERSISTENTLY failing teardown leaves the isolate + // live but unusable: g_isolate != NULL, g_initialized == 0, and + // g_teardown_state == TEARDOWN_NONE (no teardown thread exists). The wait loop + // below would treat `g_isolate != NULL && !g_initialized` as "a teardown is in + // flight" and block on uv_cond_wait -- but nothing remains to broadcast + // g_teardown_cond, so it would hang forever holding g_mutex and freeze every + // future initialize()/cleanup() (review #8 #1). This state is not recoverable + // by waiting; fail deterministically instead. g_teardown_needed stays armed so + // a later op-completion drain can still reclaim the isolate; we neither clear + // it nor touch g_ref_count (still 0 == sum(init_refs), invariant intact). + if (g_isolate != NULL && !g_initialized && g_teardown_state == TEARDOWN_NONE) { + uv_mutex_unlock(&g_mutex); + napi_throw_error(env, NULL, + "DataWeave native runtime is stranded: a prior isolate " + "teardown failed and could not be reclaimed"); + return NULL; + } + + // If a teardown from a prior cleanup() is still draining (the isolate is + // being torn down on the waiter thread from Task 2), do not race a fresh + // graal_create_isolate against it -- wait until the isolate is fully gone + // before proceeding. This is a narrow, rare path (re-initializing mid-drain), + // not a fast path, so a blocking wait here is acceptable and matches this + // function's existing fully-synchronous contract -- except in + // TEARDOWN_PENDING_WAIT (see below), where blocking would deadlock. + while (g_teardown_state != TEARDOWN_NONE || (g_isolate != NULL && !g_initialized)) { + if (g_teardown_state == TEARDOWN_PENDING_WAIT) { + // A teardown is queued but the waiter has NOT begun physical teardown + // (that transition to TEARING_DOWN happens under this same g_mutex), so + // g_isolate/g_initialized are still valid. Blocking here would freeze the + // JS event loop that an active streaming/transform worker needs in order + // to drain g_active_ops -- the waiter would then wait forever and this + // wait would never end (the P1 deadlock). Instead, ADOPT the live isolate: + // cancel the queued teardown, take a fresh ref, and wake the waiter so it + // aborts without tearing down. g_initialized is already 1, so fall through + // to the ref-count path below is unnecessary -- return directly. + if (!env_init_acquire_and_hook(env)) { + uv_mutex_unlock(&g_mutex); + napi_throw_error(env, NULL, "Failed to allocate/register env init record"); + return NULL; + } + g_teardown_cancelled = true; + g_ref_count++; + g_teardown_needed = false; // round-14: a new owner wants the isolate kept + uv_cond_broadcast(&g_teardown_cond); + uv_mutex_unlock(&g_mutex); + return NULL; + } + // TEARDOWN_TEARING_DOWN (or a transient g_isolate!=NULL && !g_initialized): + // g_active_ops has already reached 0, so nothing depends on the JS event + // loop -- this blocking wait is deadlock-free and preserves the original + // "don't race graal_create_isolate against graal_tear_down_isolate" + // guarantee that round 3's Task 3 added. + uv_cond_wait(&g_teardown_cond, &g_mutex); + } + if (g_initialized) { + if (!env_init_acquire_and_hook(env)) { + uv_mutex_unlock(&g_mutex); + napi_throw_error(env, NULL, "Failed to allocate/register env init record"); + return NULL; + } g_ref_count++; + g_teardown_needed = false; // round-14: a new owner wants the isolate kept uv_mutex_unlock(&g_mutex); return NULL; } @@ -214,7 +683,12 @@ static napi_value napi_initialize(napi_env env, napi_callback_info info) { uv_thread_options_t opts; opts.flags = UV_THREAD_HAS_STACK_SIZE; opts.stack_size = 16 * 1024 * 1024; - uv_thread_create_ex(&tid, &opts, init_thread_fn, &args); + int spawn_rc = uv_thread_create_ex(&tid, &opts, init_thread_fn, &args); + if (spawn_rc != 0) { + uv_mutex_unlock(&g_mutex); + napi_throw_error(env, NULL, "Failed to spawn initialization thread"); + return NULL; + } uv_thread_join(&tid); if (args.result != 0) { @@ -223,8 +697,74 @@ static napi_value napi_initialize(napi_env env, napi_callback_info info) { return NULL; } + if (!env_init_acquire_and_hook(env)) { + // init_thread_fn already built the isolate (g_isolate != NULL) but we have + // not yet set g_initialized = 1. If we just unlock and throw here, we leave + // g_isolate != NULL && g_initialized == 0 -- the exact condition the wait + // loop above (`g_isolate != NULL && !g_initialized`) treats as "a teardown + // is in flight". With g_teardown_state == TEARDOWN_NONE that loop cannot + // take the TEARDOWN_PENDING_WAIT adoption branch, so it falls into + // uv_cond_wait(&g_teardown_cond, ...) with nothing left to ever broadcast -- + // every subsequent initialize() on any env hangs forever. Every sibling + // error path (args.result != 0 above, and the spawn-failure path before it) + // leaves g_isolate == NULL instead, which is the recoverable state. Tear + // the just-built isolate back down before throwing so we restore that same + // recoverable g_isolate == NULL state. + // + // g_ref_count is still 0 here (we never got past this check to bump it), + // and env_init_acquire_and_hook leaves no orphan record behind on failure + // (calloc failure never created one; hook-registration failure rolls its + // own record back) -- so the invariant g_ref_count == sum(init_refs) holds + // with both sides at 0 both before and after this block. + uv_thread_t cleanup_tid; + uv_thread_options_t cleanup_opts; + cleanup_opts.flags = UV_THREAD_HAS_STACK_SIZE; + cleanup_opts.stack_size = 2 * 1024 * 1024; + int torn_down = 0; + int cleanup_spawn_rc = uv_thread_create_ex(&cleanup_tid, &cleanup_opts, cleanup_thread_fn, &torn_down); + if (cleanup_spawn_rc == 0) { + uv_thread_join(&cleanup_tid); + } + if (torn_down) { + // Teardown ran (or there was nothing to tear down) -- clear the globals + // so the next initialize() sees a clean slate. g_ref_count is already 0. + g_thread = NULL; + g_isolate = NULL; + g_initialized = 0; + } else { + // Spawn failed, or cleanup_thread_fn's attach/teardown to the isolate + // failed. The isolate is genuinely still alive with g_initialized == 0. + // Without a retry signal the next initialize() would reach the wait loop's + // `g_isolate != NULL && !g_initialized` condition with TEARDOWN_NONE (so no + // adoption branch) and block on uv_cond_wait forever -- nothing left to + // broadcast (review #7 #2). Arm the stranded-teardown retry so the + // retry_stranded_teardown_locked() at the top of the next napi_initialize + // reclaims the isolate (teardown succeeds -> fresh build). This path leaves + // g_initialized == 0, so -- unlike the release-path twin in + // isolate_ref_release_n_locked, which leaves g_initialized == 1 and is + // adopted by the g_initialized-gated fast path -- recovery here relies on + // the retry actually tearing down: it recovers the realistic TRANSIENT + // failure, but a truly PERSISTENT graal_tear_down_isolate failure would + // re-arm and retry each time and ultimately leave the isolate stranded + // until process exit (best-effort degradation, not a wedge of new work). + // g_ref_count is still 0 here, so g_teardown_needed (a retry SIGNAL, not a + // reference) keeps the invariant g_ref_count == sum(init_refs) intact. + // Mirrors the twin arm in teardown_waiter_thread_fn. + g_teardown_needed = true; + } + uv_mutex_unlock(&g_mutex); + napi_throw_error(env, NULL, "Failed to allocate/register env init record"); + return NULL; + } g_initialized = 1; g_ref_count++; + // Round-14: defensive clear. A brand-new isolate can never carry a stale + // stranded-teardown signal for itself (a new graal_create_isolate only runs + // when g_isolate == NULL, so this path cannot reuse a surviving stranded + // isolate) -- but clear it here anyway at the single create-path success + // point so no later drain retries a teardown against the isolate this + // initialize() just created and now owns. + g_teardown_needed = false; uv_mutex_unlock(&g_mutex); return NULL; } @@ -293,7 +833,13 @@ static napi_value dw_napi_run_script(napi_env env, napi_callback_info info) { uv_thread_options_t opts; opts.flags = UV_THREAD_HAS_STACK_SIZE; opts.stack_size = 2 * 1024 * 1024; - uv_thread_create_ex(&tid, &opts, run_script_thread_fn, &call_args); + int spawn_rc = uv_thread_create_ex(&tid, &opts, run_script_thread_fn, &call_args); + if (spawn_rc != 0) { + free(script); + free(inputs); + napi_throw_error(env, NULL, "Failed to spawn script execution thread"); + return NULL; + } uv_thread_join(&tid); free(script); @@ -311,6 +857,13 @@ static napi_value dw_napi_run_script(napi_env env, napi_callback_info info) { // --- Streaming output --- +// Round-9 (#2): static terminal-error JSON used when a worker thread cannot +// even strdup its result string (OOM). It is a file-scope constant, never +// heap-allocated, so any code path that would free a sentinel/chunk buffer +// must first check `buf != OOM_JSON` -- freeing a static pointer is UB. The +// wording matches the existing terse worker error style ("Empty response"). +static const char OOM_JSON[] = "{\"success\":false,\"error\":\"Out of memory\"}"; + // chunk_data with len == -1 is a sentinel indicating completion (buf holds meta JSON) struct chunk_data { char* buf; @@ -321,31 +874,63 @@ struct streaming_work { uv_thread_t tid; napi_threadsafe_function tsfn; napi_deferred deferred; + long long handle; char* script; char* inputs_json; + // The engine's record whose in_flight count this op holds. Since round-9 (#1) + // every engine has a record, so this is non-NULL for any known handle (NULL only + // for an unknown handle). The completion sentinel calls bridge_end_op on it to + // balance in_flight and run any deferred destroy (F1). + engine_bridge_t* bridge; }; static void call_js_write(napi_env env, napi_value js_callback, void* context, void* data) { - if (env == NULL || data == NULL) return; + // data == NULL: nothing was queued, nothing to free or finalize. + if (data == NULL) return; struct chunk_data* chunk = (struct chunk_data*)data; struct streaming_work* w = (struct streaming_work*)context; if (chunk->len == -1) { - napi_value result; - napi_create_string_utf8(env, chunk->buf, strlen(chunk->buf), &result); - napi_resolve_deferred(env, w->deferred, result); + // Completion sentinel. env == NULL means the environment is tearing down + // (e.g. a Worker terminating mid-op): we must not call any napi value or + // JS-calling API (napi_create_string_utf8/napi_resolve_deferred need a + // live env), but we must still perform every bit of native finalization + // -- join the worker, release the tsfn, drop the bridge in-flight hold, + // and free every heap field -- exactly once. Skipping this on env == NULL + // would leak `w` and could strand a bridge marked for deferred destruction + // indefinitely. + if (env != NULL) { + napi_value result; + napi_create_string_utf8(env, chunk->buf, strlen(chunk->buf), &result); + napi_resolve_deferred(env, w->deferred, result); + } - free(chunk->buf); + if (chunk->buf != OOM_JSON) free(chunk->buf); free(chunk); free(w->script); free(w->inputs_json); uv_thread_join(&w->tid); napi_release_threadsafe_function(w->tsfn, napi_tsfn_release); + // Drop the in-flight hold last, on this owner thread: if destroyEngine ran + // during the op it deferred the free to here (F1). After this the bridge may + // be freed, so touch nothing on it afterward. env == NULL means this env is + // dead/tearing down -- tell bridge_end_op (and any bridge_finalize it + // triggers) not to touch the napi_ref, since b->env is this same dead env. + bridge_end_op(w->bridge, /*env_still_alive=*/env != NULL); free(w); return; } + // Non-sentinel data chunk. If env == NULL the environment is gone and we + // cannot deliver it to JS; free it and return without touching `w` (its + // finalization happens only on the sentinel, above). + if (env == NULL) { + free(chunk->buf); + free(chunk); + return; + } + napi_value buffer; void* buf_data; napi_create_buffer_copy(env, chunk->len, chunk->buf, &buf_data, &buffer); @@ -360,8 +945,14 @@ static void call_js_write(napi_env env, napi_value js_callback, void* context, v static int streaming_write_cb(void* ctx, const char* buf, int len) { napi_threadsafe_function tsfn = (napi_threadsafe_function)ctx; + // Round-9 (#2): OOM here must not deref NULL / memcpy into NULL. Returning -1 + // aborts the native run cleanly (write-callback contract: non-zero stops the + // DataWeave run); the worker then still produces a terminal meta_result and + // sentinel, so the op resolves. struct chunk_data* chunk = malloc(sizeof(struct chunk_data)); + if (chunk == NULL) return -1; chunk->buf = malloc(len); + if (chunk->buf == NULL) { free(chunk); return -1; } memcpy(chunk->buf, buf, len); chunk->len = len; @@ -380,70 +971,273 @@ static void streaming_thread_fn(void* arg) { void* worker_thread = NULL; int rc = fn_attach_thread(g_isolate, &worker_thread); + // Round-9 (#2): strdup can fail under OOM. meta_result must still be a valid + // C string so the sentinel path below can deliver a terminal result -- fall + // back to the OOM_JSON static (which must never be freed; see the guarded + // frees below and in call_js_write). char* meta_result = NULL; if (rc != 0) { char err[256]; snprintf(err, sizeof(err), "{\"success\":false,\"error\":\"Failed to attach thread (code %d)\"}", rc); meta_result = strdup(err); + if (meta_result == NULL) meta_result = (char*)OOM_JSON; } else { - void* result_ptr = fn_run_script_callback( - worker_thread, w->script, w->inputs_json, streaming_write_cb, (void*)w->tsfn + void* result_ptr = fn_run_script_callback_engine( + worker_thread, w->handle, w->script, w->inputs_json, streaming_write_cb, (void*)w->tsfn ); if (result_ptr) { meta_result = strdup((const char*)result_ptr); + if (meta_result == NULL) meta_result = (char*)OOM_JSON; fn_free_cstring(worker_thread, result_ptr); } else { meta_result = strdup("{\"success\":false,\"error\":\"Empty response\"}"); + if (meta_result == NULL) meta_result = (char*)OOM_JSON; } fn_detach_thread(worker_thread); } + // Decrement here, once this thread has fully detached from the isolate -- + // not in call_js_write's completion branch. call_js_write only runs when + // the JS thread's event loop turns, and napi_initialize's pending-teardown + // wait (Task 3) can block that same event loop indefinitely; decrementing + // from the JS-thread callback made the two waits circular. Decrementing + // here ties g_active_ops to the actual invariant isolate teardown needs + // (no GraalVM-attached thread remains), independent of the event loop. + uv_mutex_lock(&g_mutex); + g_active_ops--; + uv_cond_broadcast(&g_teardown_cond); + // Round-14 (#2/#3): if a prior last-release could not tear the isolate down + // and left it stranded (g_teardown_needed), retry now that this op has drained. + retry_stranded_teardown_locked(); + uv_mutex_unlock(&g_mutex); + + // Round-9 (#2): if even the sentinel struct cannot be allocated, we cannot + // enqueue a completion -- run the SAME native finalize the env-dead + // (napi_closing) branch below runs, so g_active_ops (already decremented + // above) plus the bridge in-flight hold and w are released and nothing is + // stranded. This is the "sentinel malloc NULL -> skip enqueue + unwind like + // the env-dead sentinel branch" path. struct chunk_data* sentinel = malloc(sizeof(struct chunk_data)); + if (sentinel == NULL) { + if (meta_result != OOM_JSON) free(meta_result); + free(w->script); + free(w->inputs_json); + bridge_end_op(w->bridge, /*env_still_alive=*/false); + free(w); + return; + } sentinel->buf = meta_result; sentinel->len = -1; - napi_call_threadsafe_function(w->tsfn, sentinel, napi_tsfn_blocking); + napi_status enq = napi_call_threadsafe_function(w->tsfn, sentinel, napi_tsfn_blocking); + if (enq != napi_ok) { + // The env is tearing down (napi_closing): the sentinel was dropped and + // call_js_write will never run, so finalize here instead -- the exact same + // native cleanup as call_js_write's sentinel branch, minus the things + // that are illegal, impossible, or already done on this worker thread: + // - no napi value / deferred call (env is dead; those are env-affine) + // - no uv_thread_join(&w->tid): we ARE w->tid; a thread cannot join + // itself. The handle goes unreaped -- an unavoidable, negligible leak + // during a Worker teardown that is already discarding this env. + // - no napi_release_threadsafe_function(w->tsfn, ...): this tsfn was + // created with initial_thread_count = 1 and this worker is its sole + // producer, so Node's internal thread_count for it is exactly 1 on + // entry to this Push call. Node's ThreadSafeFunction::Push (the + // implementation behind napi_call_threadsafe_function) decrements + // thread_count for the calling thread BEFORE returning napi_closing, + // and -- if that decrement brings thread_count to 0 while the + // internal state is already kClosed -- Push runs `delete this` on + // the tsfn right there. So receiving napi_closing here already IS + // this thread's discharge of the tsfn (matches the doc's "destroyed + // when every thread ... has called napi_release_threadsafe_function() + // or has received a return status of napi_closing"); calling release + // again afterward would be a double-discharge and, whenever Push + // already deleted the object, a use-after-free. Omit it. + // End the bridge op with env_still_alive=false so bridge_finalize skips + // the thread-affine napi_delete_reference (Node auto-reclaims the ref + // when the dead env is destroyed). + if (sentinel->buf != OOM_JSON) free(sentinel->buf); + free(sentinel); + free(w->script); + free(w->inputs_json); + bridge_end_op(w->bridge, /*env_still_alive=*/false); + free(w); + } } -static napi_value napi_run_script_streaming(napi_env env, napi_callback_info info) { +static napi_value napi_run_script_streaming_engine(napi_env env, napi_callback_info info) { if (!g_initialized) { napi_throw_error(env, NULL, "Not initialized. Call initialize() first."); return NULL; } - if (!fn_run_script_callback) { - napi_throw_error(env, NULL, "run_script_callback not available in native library"); + if (!fn_run_script_callback_engine) { + napi_throw_error(env, NULL, "run_script_callback_engine not available in native library"); return NULL; } - size_t argc = 3; - napi_value argv[3]; + size_t argc = 4; + napi_value argv[4]; napi_get_cb_info(env, info, &argc, argv, NULL, NULL); - if (argc < 3) { - napi_throw_error(env, NULL, "runScriptStreaming requires (script, inputsJson, chunkCallback)"); + if (argc < 4) { + napi_throw_error(env, NULL, "runScriptStreamingEngine requires (handle, script, inputsJson, chunkCallback)"); + return NULL; + } + + // Validate the handle before admission (round-6 #1, defense-in-depth): a + // non-integer handle must be rejected before g_active_ops is ever reserved, + // so there is nothing to unwind here -- simpler than reserving first and + // unwinding on failure. + int64_t handle64; + if (napi_get_value_int64(env, argv[0], &handle64) != napi_ok) { + napi_throw_error(env, NULL, "runScriptStreamingEngine: handle must be an integer"); return NULL; } + // Atomic admission: check lifecycle state and reserve the op in ONE critical + // section, before allocating any work/tsfn/promise/bridge. Reading + // g_initialized outside the lock and reserving g_active_ops later (the old + // shape) let a second Worker's napi_cleanup Case-4 tear the isolate down in + // the gap, so a freshly spawned worker attached to a dead isolate (round-6 + // #2). Rejecting on g_teardown_state != TEARDOWN_NONE also refuses new ops + // once a teardown is queued/underway. Admit an ADOPTED isolate: + // napi_initialize's adoption branch sets g_teardown_cancelled = true on a + // still-live PENDING_WAIT isolate but does not reset g_teardown_state (only + // the async waiter does), so a merely-cancelled teardown must not reject + // here -- otherwise a valid post-adoption op throws "Not initialized". A + // genuine (non-cancelled) PENDING_WAIT or a committed TEARING_DOWN still + // rejects. + uv_mutex_lock(&g_mutex); + if (!g_initialized || (g_teardown_state != TEARDOWN_NONE && !g_teardown_cancelled)) { + uv_mutex_unlock(&g_mutex); + napi_throw_error(env, NULL, "Not initialized. Call initialize() first."); + return NULL; + } + g_active_ops++; + // Round-11 (#2): pin the engine in the SAME critical section as the + // g_active_ops reservation, before any window a concurrent destroyEngine + // could use. NULL for an unknown handle (the worker surfaces "Unknown engine + // handle"). Stashed on w->bridge once w is allocated; every early-return + // below releases it via bridge_end_op alongside g_active_ops. + engine_bridge_t* pinned = bridge_begin_op_locked((long long)handle64); + uv_mutex_unlock(&g_mutex); + + // Conversions run after the admission reservation above, so any throw here + // must release g_active_ops before returning (round-7 #2). size_t script_len, inputs_len; - napi_get_value_string_utf8(env, argv[0], NULL, 0, &script_len); - napi_get_value_string_utf8(env, argv[1], NULL, 0, &inputs_len); + if (napi_get_value_string_utf8(env, argv[1], NULL, 0, &script_len) != napi_ok) { + bridge_end_op(pinned, /*env_still_alive=*/true); + uv_mutex_lock(&g_mutex); g_active_ops--; uv_cond_broadcast(&g_teardown_cond); uv_mutex_unlock(&g_mutex); + napi_throw_error(env, NULL, "runScriptStreamingEngine: script must be a string"); + return NULL; + } + if (napi_get_value_string_utf8(env, argv[2], NULL, 0, &inputs_len) != napi_ok) { + bridge_end_op(pinned, /*env_still_alive=*/true); + uv_mutex_lock(&g_mutex); g_active_ops--; uv_cond_broadcast(&g_teardown_cond); uv_mutex_unlock(&g_mutex); + napi_throw_error(env, NULL, "runScriptStreamingEngine: inputsJson must be a string"); + return NULL; + } + // OOM safety (round-8): every allocation is NULL-checked before it is + // dereferenced, and every failure path releases the g_active_ops reservation + // taken above (mirroring napi_run_script_engine's "OOM" throw). Without this + // an allocation failure segfaults the host process AND strands g_active_ops. struct streaming_work* w = calloc(1, sizeof(struct streaming_work)); + if (w == NULL) { + // w is NULL -- do not touch w->script/w->inputs_json here. + bridge_end_op(pinned, /*env_still_alive=*/true); + uv_mutex_lock(&g_mutex); g_active_ops--; uv_cond_broadcast(&g_teardown_cond); uv_mutex_unlock(&g_mutex); + napi_throw_error(env, NULL, "OOM"); + return NULL; + } + w->handle = (long long)handle64; w->script = malloc(script_len + 1); w->inputs_json = malloc(inputs_len + 1); - napi_get_value_string_utf8(env, argv[0], w->script, script_len + 1, NULL); - napi_get_value_string_utf8(env, argv[1], w->inputs_json, inputs_len + 1, NULL); + if (w->script == NULL || w->inputs_json == NULL) { + free(w->script); free(w->inputs_json); free(w); + bridge_end_op(pinned, /*env_still_alive=*/true); + uv_mutex_lock(&g_mutex); g_active_ops--; uv_cond_broadcast(&g_teardown_cond); uv_mutex_unlock(&g_mutex); + napi_throw_error(env, NULL, "OOM"); + return NULL; + } + if (napi_get_value_string_utf8(env, argv[1], w->script, script_len + 1, NULL) != napi_ok || + napi_get_value_string_utf8(env, argv[2], w->inputs_json, inputs_len + 1, NULL) != napi_ok) { + free(w->script); free(w->inputs_json); free(w); + bridge_end_op(pinned, /*env_still_alive=*/true); + uv_mutex_lock(&g_mutex); g_active_ops--; uv_cond_broadcast(&g_teardown_cond); uv_mutex_unlock(&g_mutex); + napi_throw_error(env, NULL, "runScriptStreamingEngine: failed to read script/inputsJson"); + return NULL; + } + // Round-9 (#3, updated round-11 #2): the resource creations below run AFTER + // g_active_ops was reserved (and after w + its buffers were allocated), and + // the engine pin (`pinned`) was already taken at admission. A failed create + // must release both the pin (bridge_end_op) and g_active_ops (verbatim + // pattern), free any tsfn already created, free w + buffers, and throw -- + // otherwise the worker sees a zeroed w->tsfn/w->deferred (crash), the pin is + // stranded (blocks destroyEngine forever), or g_active_ops is stranded + // (teardown wedge). napi_value resource_name; - napi_create_string_utf8(env, "dwStreaming", NAPI_AUTO_LENGTH, &resource_name); - napi_create_threadsafe_function(env, argv[2], NULL, resource_name, 0, 1, NULL, NULL, w, call_js_write, &w->tsfn); + if (napi_create_string_utf8(env, "dwStreaming", NAPI_AUTO_LENGTH, &resource_name) != napi_ok) { + free(w->script); free(w->inputs_json); free(w); + bridge_end_op(pinned, /*env_still_alive=*/true); + uv_mutex_lock(&g_mutex); g_active_ops--; uv_cond_broadcast(&g_teardown_cond); uv_mutex_unlock(&g_mutex); + napi_throw_error(env, NULL, "runScriptStreamingEngine: failed to create resource name"); + return NULL; + } + if (napi_create_threadsafe_function(env, argv[3], NULL, resource_name, 0, 1, NULL, NULL, w, call_js_write, &w->tsfn) != napi_ok) { + free(w->script); free(w->inputs_json); free(w); + bridge_end_op(pinned, /*env_still_alive=*/true); + uv_mutex_lock(&g_mutex); g_active_ops--; uv_cond_broadcast(&g_teardown_cond); uv_mutex_unlock(&g_mutex); + napi_throw_error(env, NULL, "runScriptStreamingEngine: failed to create threadsafe function"); + return NULL; + } napi_value promise; - napi_create_promise(env, &w->deferred, &promise); + if (napi_create_promise(env, &w->deferred, &promise) != napi_ok) { + // The tsfn was created above; release it before freeing w (it holds w as + // its context). No worker exists yet, so this release is the sole discharge. + napi_release_threadsafe_function(w->tsfn, napi_tsfn_release); + free(w->script); free(w->inputs_json); free(w); + bridge_end_op(pinned, /*env_still_alive=*/true); + uv_mutex_lock(&g_mutex); g_active_ops--; uv_cond_broadcast(&g_teardown_cond); uv_mutex_unlock(&g_mutex); + napi_throw_error(env, NULL, "runScriptStreamingEngine: failed to create promise"); + return NULL; + } + + // Round-11 (#2): the pin was taken at admission (bridge_begin_op_locked) in + // the same critical section as g_active_ops, so a concurrent destroyEngine + // could never free this bridge under the admitted op. Just record it on w; + // the completion sentinel releases it via bridge_end_op. NULL for a + // resolver-less/unknown engine, handled everywhere as a no-op. + w->bridge = pinned; uv_thread_options_t opts; opts.flags = UV_THREAD_HAS_STACK_SIZE; opts.stack_size = 2 * 1024 * 1024; - uv_thread_create_ex(&w->tid, &opts, streaming_thread_fn, w); + int spawn_rc = uv_thread_create_ex(&w->tid, &opts, streaming_thread_fn, w); + + if (spawn_rc != 0) { + // The worker never ran, so nothing will ever decrement g_active_ops, + // release the bridge hold, or resolve the promise -- unwind everything + // committed above ourselves, in reverse order, mirroring call_js_write's + // completion branch (minus uv_thread_join: there is no thread to join). + uv_mutex_lock(&g_mutex); + g_active_ops--; + uv_cond_broadcast(&g_teardown_cond); + uv_mutex_unlock(&g_mutex); + + // Synchronous call on the JS thread -- env is live here. + bridge_end_op(w->bridge, /*env_still_alive=*/true); + napi_release_threadsafe_function(w->tsfn, napi_tsfn_release); + + napi_value result; + napi_create_string_utf8(env, "{\"success\":false,\"error\":\"Failed to spawn streaming worker thread\"}", NAPI_AUTO_LENGTH, &result); + napi_resolve_deferred(env, w->deferred, result); + + free(w->script); + free(w->inputs_json); + free(w); + } return promise; } @@ -455,11 +1249,17 @@ struct transform_work { napi_threadsafe_function read_tsfn; napi_threadsafe_function write_tsfn; napi_deferred deferred; + long long handle; char* script; char* inputs_json; char* input_name; char* input_mime_type; char* input_charset; + // The engine's record whose in_flight count this op holds. Since round-9 (#1) + // every engine has a record, so this is non-NULL for any known handle (NULL only + // for an unknown handle). The completion sentinel calls bridge_end_op on it to + // balance in_flight and run any deferred destroy (F1). + engine_bridge_t* bridge; }; struct read_request { @@ -472,66 +1272,78 @@ struct read_request { }; static void call_js_read(napi_env env, napi_value js_callback, void* context, void* data) { - if (env == NULL || data == NULL) return; + if (data == NULL) return; // nothing to signal struct read_request* req = (struct read_request*)data; - napi_value buf_size_val; - napi_create_int32(env, req->buffer_size, &buf_size_val); + if (env == NULL) { + // N-API can invoke a threadsafe-function callback with env == NULL when + // the environment is tearing down with items still queued (e.g. a Worker + // terminating mid-transform). transform_read_cb is synchronously blocked + // on req->cond waiting for this callback to signal it -- unlike + // call_js_write/call_js_transform_write, there is no sentinel-driven path + // that would otherwise unblock it. Treat this as a terminal read error so + // the blocked thread wakes up, detects the failure via bytes_read == -1, + // and the worker can detach from the isolate instead of hanging forever. + req->bytes_read = -1; + } else { + napi_value buf_size_val; + napi_create_int32(env, req->buffer_size, &buf_size_val); - napi_value global; - napi_get_global(env, &global); + napi_value global; + napi_get_global(env, &global); - napi_value result; - napi_status status = napi_call_function(env, global, js_callback, 1, &buf_size_val, &result); - - if (status == napi_ok && result != NULL) { - bool is_buffer; - napi_is_buffer(env, result, &is_buffer); - if (is_buffer) { - void* buf_data; - size_t buf_len; - napi_get_buffer_info(env, result, &buf_data, &buf_len); - int n = (int)buf_len < req->buffer_size ? (int)buf_len : req->buffer_size; - if (n > 0) memcpy(req->buffer, buf_data, n); - req->bytes_read = n; - } else { - req->bytes_read = 0; - } - } else { - // Clear pending exception to prevent propagation - if (status == napi_pending_exception) { - napi_value exception; - napi_get_and_clear_last_exception(env, &exception); - - // Extract and log exception details before discarding - napi_value message_prop, stack_prop; - char message_buf[512] = {0}; - char stack_buf[2048] = {0}; - size_t message_len = 0, stack_len = 0; - - // Try to get the message property - if (napi_get_named_property(env, exception, "message", &message_prop) == napi_ok) { - napi_get_value_string_utf8(env, message_prop, message_buf, sizeof(message_buf), &message_len); + napi_value result; + napi_status status = napi_call_function(env, global, js_callback, 1, &buf_size_val, &result); + + if (status == napi_ok && result != NULL) { + bool is_buffer; + napi_is_buffer(env, result, &is_buffer); + if (is_buffer) { + void* buf_data; + size_t buf_len; + napi_get_buffer_info(env, result, &buf_data, &buf_len); + int n = (int)buf_len < req->buffer_size ? (int)buf_len : req->buffer_size; + if (n > 0) memcpy(req->buffer, buf_data, n); + req->bytes_read = n; + } else { + req->bytes_read = 0; } + } else { + // Clear pending exception to prevent propagation + if (status == napi_pending_exception) { + napi_value exception; + napi_get_and_clear_last_exception(env, &exception); + + // Extract and log exception details before discarding + napi_value message_prop, stack_prop; + char message_buf[512] = {0}; + char stack_buf[2048] = {0}; + size_t message_len = 0, stack_len = 0; + + // Try to get the message property + if (napi_get_named_property(env, exception, "message", &message_prop) == napi_ok) { + napi_get_value_string_utf8(env, message_prop, message_buf, sizeof(message_buf), &message_len); + } - // Try to get the stack property - if (napi_get_named_property(env, exception, "stack", &stack_prop) == napi_ok) { - napi_get_value_string_utf8(env, stack_prop, stack_buf, sizeof(stack_buf), &stack_len); - } + // Try to get the stack property + if (napi_get_named_property(env, exception, "stack", &stack_prop) == napi_ok) { + napi_get_value_string_utf8(env, stack_prop, stack_buf, sizeof(stack_buf), &stack_len); + } - // Log the exception to stderr for diagnostics - fprintf(stderr, "[DataWeave Node addon] Read callback threw exception:\n"); - if (message_len > 0) { - fprintf(stderr, " Message: %s\n", message_buf); - } - if (stack_len > 0) { - fprintf(stderr, " Stack:\n%s\n", stack_buf); - } - if (message_len == 0 && stack_len == 0) { - fprintf(stderr, " (Unable to extract exception details)\n"); + // Log the exception to stderr for diagnostics + fprintf(stderr, "[DataWeave Node addon] Read callback threw exception:\n"); + if (message_len > 0) { + fprintf(stderr, " Message: %s\n", message_buf); + } + if (stack_len > 0) { + fprintf(stderr, " Stack:\n%s\n", stack_buf); + } + if (message_len == 0 && stack_len == 0) { + fprintf(stderr, " (Unable to extract exception details)\n"); + } } + req->bytes_read = -1; // Signal error } - req->bytes_read = -1; // Signal error } uv_mutex_lock(&req->mutex); @@ -572,8 +1384,12 @@ static int transform_read_cb(void* ctx, char* buf, int buf_size) { static int transform_write_cb(void* ctx, const char* buf, int len) { struct transform_work* w = (struct transform_work*)ctx; + // Round-9 (#2): OOM-safe, mirrors streaming_write_cb. Return -1 to abort the + // native run cleanly; the worker still delivers a terminal sentinel. struct chunk_data* chunk = malloc(sizeof(struct chunk_data)); + if (chunk == NULL) return -1; chunk->buf = malloc(len); + if (chunk->buf == NULL) { free(chunk); return -1; } memcpy(chunk->buf, buf, len); chunk->len = len; @@ -587,16 +1403,27 @@ static int transform_write_cb(void* ctx, const char* buf, int len) { } static void call_js_transform_write(napi_env env, napi_value js_callback, void* context, void* data) { - if (env == NULL || data == NULL) return; + // data == NULL: nothing was queued, nothing to free or finalize. + if (data == NULL) return; struct chunk_data* chunk = (struct chunk_data*)data; struct transform_work* w = (struct transform_work*)context; if (chunk->len == -1) { - napi_value result; - napi_create_string_utf8(env, chunk->buf, strlen(chunk->buf), &result); - napi_resolve_deferred(env, w->deferred, result); + // Completion sentinel. env == NULL means the environment is tearing down + // (e.g. a Worker terminating mid-op): we must not call any napi value or + // JS-calling API (napi_create_string_utf8/napi_resolve_deferred need a + // live env), but we must still perform every bit of native finalization + // -- join the worker, release both tsfns, drop the bridge in-flight hold, + // and free every heap field -- exactly once. Skipping this on env == NULL + // would leak `w` and could strand a bridge marked for deferred destruction + // indefinitely. + if (env != NULL) { + napi_value result; + napi_create_string_utf8(env, chunk->buf, strlen(chunk->buf), &result); + napi_resolve_deferred(env, w->deferred, result); + } - free(chunk->buf); + if (chunk->buf != OOM_JSON) free(chunk->buf); free(chunk); free(w->script); free(w->inputs_json); @@ -607,10 +1434,25 @@ static void call_js_transform_write(napi_env env, napi_value js_callback, void* uv_thread_join(&w->tid); napi_release_threadsafe_function(w->read_tsfn, napi_tsfn_release); napi_release_threadsafe_function(w->write_tsfn, napi_tsfn_release); + // Drop the in-flight hold last, on this owner thread: if destroyEngine ran + // during the op it deferred the free to here (F1). After this the bridge may + // be freed, so touch nothing on it afterward. env == NULL means this env is + // dead/tearing down -- tell bridge_end_op (and any bridge_finalize it + // triggers) not to touch the napi_ref, since b->env is this same dead env. + bridge_end_op(w->bridge, /*env_still_alive=*/env != NULL); free(w); return; } + // Non-sentinel data chunk. If env == NULL the environment is gone and we + // cannot deliver it to JS; free it and return without touching `w` (its + // finalization happens only on the sentinel, above). + if (env == NULL) { + free(chunk->buf); + free(chunk); + return; + } + napi_value buffer; void* buf_data; napi_create_buffer_copy(env, chunk->len, chunk->buf, &buf_data, &buffer); @@ -629,94 +1471,303 @@ static void transform_thread_fn(void* arg) { void* worker_thread = NULL; int rc = fn_attach_thread(g_isolate, &worker_thread); + // Round-9 (#2): strdup can fail under OOM; fall back to the OOM_JSON static + // so the sentinel below still delivers a terminal result. Mirrors + // streaming_thread_fn. char* meta_result = NULL; if (rc != 0) { char err[256]; snprintf(err, sizeof(err), "{\"success\":false,\"error\":\"Failed to attach thread (code %d)\"}", rc); meta_result = strdup(err); + if (meta_result == NULL) meta_result = (char*)OOM_JSON; } else { - void* result_ptr = fn_run_script_input_output_callback( - worker_thread, w->script, w->inputs_json, + void* result_ptr = fn_run_script_input_output_callback_engine( + worker_thread, w->handle, w->script, w->inputs_json, w->input_name, w->input_mime_type, w->input_charset, transform_read_cb, transform_write_cb, (void*)w ); if (result_ptr) { meta_result = strdup((const char*)result_ptr); + if (meta_result == NULL) meta_result = (char*)OOM_JSON; fn_free_cstring(worker_thread, result_ptr); } else { meta_result = strdup("{\"success\":false,\"error\":\"Empty response\"}"); + if (meta_result == NULL) meta_result = (char*)OOM_JSON; } fn_detach_thread(worker_thread); } + // See streaming_thread_fn's comment: decrement here (after detach), not in + // call_js_transform_write's completion branch, to avoid the same + // circular-wait deadlock against napi_initialize's pending-teardown wait. + uv_mutex_lock(&g_mutex); + g_active_ops--; + uv_cond_broadcast(&g_teardown_cond); + // Round-14 (#2/#3): retry a stranded teardown now that this op has drained. + retry_stranded_teardown_locked(); + uv_mutex_unlock(&g_mutex); + + // Round-9 (#2): sentinel malloc NULL -> skip enqueue and run the same native + // finalize as the env-dead branch below (release the bridge hold + free w and + // all fields), so g_active_ops (already decremented above) and the in-flight + // hold are released. No self-join, no env-affine napi call, no tsfn release + // (see the env-dead branch's citation for why releasing the tsfns here is + // unsafe). struct chunk_data* sentinel = malloc(sizeof(struct chunk_data)); + if (sentinel == NULL) { + if (meta_result != OOM_JSON) free(meta_result); + free(w->script); + free(w->inputs_json); + free(w->input_name); + free(w->input_mime_type); + free(w->input_charset); + bridge_end_op(w->bridge, /*env_still_alive=*/false); + free(w); + return; + } sentinel->buf = meta_result; sentinel->len = -1; - napi_call_threadsafe_function(w->write_tsfn, sentinel, napi_tsfn_blocking); + napi_status enq = napi_call_threadsafe_function(w->write_tsfn, sentinel, napi_tsfn_blocking); + if (enq != napi_ok) { + // See streaming_thread_fn: env tearing down, sentinel dropped, finalize + // here. No self-join, no env-affine napi call. + // + // Do NOT release write_tsfn: this worker is its sole producer + // (initial_thread_count = 1), so receiving napi_closing from this same + // Push call already decremented Node's internal thread_count for it to 0 + // and, if the tsfn's internal state was already kClosed, already ran + // `delete this` on it inside Push -- see streaming_thread_fn's comment + // for the full citation. Releasing it again here would be a + // double-discharge and potentially a use-after-free. + // + // Do NOT release read_tsfn either, even though this same worker is also + // its sole producer: whether *it* has already received napi_closing (and + // so already discharged/deleted itself the same way) depends on whether + // the script issued reads during teardown, which this code path has no + // way to know. We cannot prove read_tsfn's discharge state here, so -- + // consistent with the env == NULL dead-env handling elsewhere in this + // file -- we accept the small leak of an already-tearing-down tsfn + // rather than risk a use-after-free on an object whose state is unknown. + // + // End the bridge op with env_still_alive=false so bridge_finalize skips + // the thread-affine napi_delete_reference (Node auto-reclaims the ref + // when the dead env is destroyed). + if (sentinel->buf != OOM_JSON) free(sentinel->buf); + free(sentinel); + free(w->script); + free(w->inputs_json); + free(w->input_name); + free(w->input_mime_type); + free(w->input_charset); + bridge_end_op(w->bridge, /*env_still_alive=*/false); + free(w); + } } -static napi_value napi_run_script_transform(napi_env env, napi_callback_info info) { +static napi_value napi_run_script_transform_engine(napi_env env, napi_callback_info info) { if (!g_initialized) { napi_throw_error(env, NULL, "Not initialized. Call initialize() first."); return NULL; } - if (!fn_run_script_input_output_callback) { - napi_throw_error(env, NULL, "run_script_input_output_callback not available in native library"); + if (!fn_run_script_input_output_callback_engine) { + napi_throw_error(env, NULL, "run_script_input_output_callback_engine not available in native library"); return NULL; } - size_t argc = 7; - napi_value argv[7]; + size_t argc = 8; + napi_value argv[8]; napi_get_cb_info(env, info, &argc, argv, NULL, NULL); - if (argc < 7) { - napi_throw_error(env, NULL, "runScriptTransform requires 7 arguments"); + if (argc < 8) { + napi_throw_error(env, NULL, "runScriptTransformEngine requires 8 arguments"); + return NULL; + } + + // Validate the handle before admission (round-6 #1, defense-in-depth): a + // non-integer handle must be rejected before g_active_ops is ever reserved, + // so there is nothing to unwind here -- simpler than reserving first and + // unwinding on failure. Keep this consistent with + // napi_run_script_streaming_engine's ordering. + int64_t handle64; + if (napi_get_value_int64(env, argv[0], &handle64) != napi_ok) { + napi_throw_error(env, NULL, "runScriptTransformEngine: handle must be an integer"); return NULL; } + // Atomic admission (see napi_run_script_streaming_engine for the full + // rationale, round-6 #2): check lifecycle + reserve g_active_ops in one + // critical section, before any work/tsfn/promise/bridge is committed. + // Admit an ADOPTED isolate: napi_initialize's adoption branch sets + // g_teardown_cancelled = true on a still-live PENDING_WAIT isolate but does + // not reset g_teardown_state (only the async waiter does), so a + // merely-cancelled teardown must not reject here -- otherwise a valid + // post-adoption op throws "Not initialized". A genuine (non-cancelled) + // PENDING_WAIT or a committed TEARING_DOWN still rejects. + uv_mutex_lock(&g_mutex); + if (!g_initialized || (g_teardown_state != TEARDOWN_NONE && !g_teardown_cancelled)) { + uv_mutex_unlock(&g_mutex); + napi_throw_error(env, NULL, "Not initialized. Call initialize() first."); + return NULL; + } + g_active_ops++; + // Round-11 (#2): pin the engine in the SAME critical section as the + // g_active_ops reservation, before any window a concurrent destroyEngine + // could use. NULL for an unknown handle (the worker surfaces "Unknown engine + // handle"). Stashed on w->bridge once w is allocated; every early-return + // below releases it via bridge_end_op alongside g_active_ops. + engine_bridge_t* pinned = bridge_begin_op_locked((long long)handle64); + uv_mutex_unlock(&g_mutex); + + // Conversions run after the admission reservation above, so any throw here + // must free the partially-populated work struct AND release g_active_ops + // before returning (round-7 #2). calloc zeroed w, so free() on an unset + // field pointer is a safe free(NULL). TRANSFORM_FAIL centralizes the + // unwind. + // OOM safety (round-8): NULL-check the work struct before dereferencing it, + // releasing the g_active_ops reservation taken above. The per-field malloc + // checks below reuse TRANSFORM_FAIL (which frees all fields + w and unwinds); + // this standalone branch cannot use it (the macro dereferences w). struct transform_work* w = calloc(1, sizeof(struct transform_work)); + if (w == NULL) { + bridge_end_op(pinned, /*env_still_alive=*/true); + uv_mutex_lock(&g_mutex); g_active_ops--; uv_cond_broadcast(&g_teardown_cond); uv_mutex_unlock(&g_mutex); + napi_throw_error(env, NULL, "OOM"); + return NULL; + } size_t len; - - napi_get_value_string_utf8(env, argv[0], NULL, 0, &len); + w->handle = (long long)handle64; + + #define TRANSFORM_FAIL(msg) do { \ + bridge_end_op(pinned, /*env_still_alive=*/true); \ + free(w->script); free(w->inputs_json); free(w->input_name); \ + free(w->input_mime_type); free(w->input_charset); free(w); \ + uv_mutex_lock(&g_mutex); g_active_ops--; uv_cond_broadcast(&g_teardown_cond); uv_mutex_unlock(&g_mutex); \ + napi_throw_error(env, NULL, (msg)); \ + return NULL; \ + } while (0) + + if (napi_get_value_string_utf8(env, argv[1], NULL, 0, &len) != napi_ok) TRANSFORM_FAIL("runScriptTransformEngine: script must be a string"); w->script = malloc(len + 1); - napi_get_value_string_utf8(env, argv[0], w->script, len + 1, NULL); + if (w->script == NULL) TRANSFORM_FAIL("OOM"); + if (napi_get_value_string_utf8(env, argv[1], w->script, len + 1, NULL) != napi_ok) TRANSFORM_FAIL("runScriptTransformEngine: failed to read script"); - napi_get_value_string_utf8(env, argv[1], NULL, 0, &len); + if (napi_get_value_string_utf8(env, argv[2], NULL, 0, &len) != napi_ok) TRANSFORM_FAIL("runScriptTransformEngine: inputsJson must be a string"); w->inputs_json = malloc(len + 1); - napi_get_value_string_utf8(env, argv[1], w->inputs_json, len + 1, NULL); + if (w->inputs_json == NULL) TRANSFORM_FAIL("OOM"); + if (napi_get_value_string_utf8(env, argv[2], w->inputs_json, len + 1, NULL) != napi_ok) TRANSFORM_FAIL("runScriptTransformEngine: failed to read inputsJson"); - napi_get_value_string_utf8(env, argv[2], NULL, 0, &len); + if (napi_get_value_string_utf8(env, argv[3], NULL, 0, &len) != napi_ok) TRANSFORM_FAIL("runScriptTransformEngine: inputName must be a string"); w->input_name = malloc(len + 1); - napi_get_value_string_utf8(env, argv[2], w->input_name, len + 1, NULL); + if (w->input_name == NULL) TRANSFORM_FAIL("OOM"); + if (napi_get_value_string_utf8(env, argv[3], w->input_name, len + 1, NULL) != napi_ok) TRANSFORM_FAIL("runScriptTransformEngine: failed to read inputName"); - napi_get_value_string_utf8(env, argv[3], NULL, 0, &len); + if (napi_get_value_string_utf8(env, argv[4], NULL, 0, &len) != napi_ok) TRANSFORM_FAIL("runScriptTransformEngine: inputMimeType must be a string"); w->input_mime_type = malloc(len + 1); - napi_get_value_string_utf8(env, argv[3], w->input_mime_type, len + 1, NULL); + if (w->input_mime_type == NULL) TRANSFORM_FAIL("OOM"); + if (napi_get_value_string_utf8(env, argv[4], w->input_mime_type, len + 1, NULL) != napi_ok) TRANSFORM_FAIL("runScriptTransformEngine: failed to read inputMimeType"); napi_valuetype type; - napi_typeof(env, argv[4], &type); + if (napi_typeof(env, argv[5], &type) != napi_ok) TRANSFORM_FAIL("runScriptTransformEngine: invalid inputCharset argument"); if (type == napi_string) { - napi_get_value_string_utf8(env, argv[4], NULL, 0, &len); + if (napi_get_value_string_utf8(env, argv[5], NULL, 0, &len) != napi_ok) TRANSFORM_FAIL("runScriptTransformEngine: inputCharset must be a string"); w->input_charset = malloc(len + 1); - napi_get_value_string_utf8(env, argv[4], w->input_charset, len + 1, NULL); - } else { + if (w->input_charset == NULL) TRANSFORM_FAIL("OOM"); + if (napi_get_value_string_utf8(env, argv[5], w->input_charset, len + 1, NULL) != napi_ok) TRANSFORM_FAIL("runScriptTransformEngine: failed to read inputCharset"); + } else if (type == napi_null || type == napi_undefined) { + // inputCharset is nullable: null/undefined mean "no charset". This is the + // only non-string form the JS binding ever sends (dataweave.ts normalizes + // opts?.charset ?? null). w->input_charset = NULL; + } else { + // Any other type (object, number, boolean, ...) is a caller error, not + // "no charset". Fail closed like the four non-nullable string args above + // rather than silently coercing to NULL (review #9 #6). + TRANSFORM_FAIL("runScriptTransformEngine: inputCharset must be a string, null, or undefined"); } - + #undef TRANSFORM_FAIL + + // Round-9 (#3, updated round-11 #2): check each resource creation; on + // failure release the engine pin (`pinned`, taken at admission) via + // bridge_end_op, release g_active_ops (verbatim), release any tsfn already + // created, free w + all five string buffers, and throw. read_tsfn has no + // context (NULL); write_tsfn holds w as context, so release write_tsfn + // before freeing w if it was created. napi_value resource_name; - napi_create_string_utf8(env, "dwTransform", NAPI_AUTO_LENGTH, &resource_name); + if (napi_create_string_utf8(env, "dwTransform", NAPI_AUTO_LENGTH, &resource_name) != napi_ok) { + free(w->script); free(w->inputs_json); free(w->input_name); free(w->input_mime_type); free(w->input_charset); free(w); + bridge_end_op(pinned, /*env_still_alive=*/true); + uv_mutex_lock(&g_mutex); g_active_ops--; uv_cond_broadcast(&g_teardown_cond); uv_mutex_unlock(&g_mutex); + napi_throw_error(env, NULL, "runScriptTransformEngine: failed to create resource name"); + return NULL; + } - napi_create_threadsafe_function(env, argv[5], NULL, resource_name, 0, 1, NULL, NULL, NULL, call_js_read, &w->read_tsfn); - napi_create_threadsafe_function(env, argv[6], NULL, resource_name, 0, 1, NULL, NULL, w, call_js_transform_write, &w->write_tsfn); + if (napi_create_threadsafe_function(env, argv[6], NULL, resource_name, 0, 1, NULL, NULL, NULL, call_js_read, &w->read_tsfn) != napi_ok) { + free(w->script); free(w->inputs_json); free(w->input_name); free(w->input_mime_type); free(w->input_charset); free(w); + bridge_end_op(pinned, /*env_still_alive=*/true); + uv_mutex_lock(&g_mutex); g_active_ops--; uv_cond_broadcast(&g_teardown_cond); uv_mutex_unlock(&g_mutex); + napi_throw_error(env, NULL, "runScriptTransformEngine: failed to create read threadsafe function"); + return NULL; + } + if (napi_create_threadsafe_function(env, argv[7], NULL, resource_name, 0, 1, NULL, NULL, w, call_js_transform_write, &w->write_tsfn) != napi_ok) { + napi_release_threadsafe_function(w->read_tsfn, napi_tsfn_release); + free(w->script); free(w->inputs_json); free(w->input_name); free(w->input_mime_type); free(w->input_charset); free(w); + bridge_end_op(pinned, /*env_still_alive=*/true); + uv_mutex_lock(&g_mutex); g_active_ops--; uv_cond_broadcast(&g_teardown_cond); uv_mutex_unlock(&g_mutex); + napi_throw_error(env, NULL, "runScriptTransformEngine: failed to create write threadsafe function"); + return NULL; + } napi_value promise; - napi_create_promise(env, &w->deferred, &promise); + if (napi_create_promise(env, &w->deferred, &promise) != napi_ok) { + napi_release_threadsafe_function(w->read_tsfn, napi_tsfn_release); + napi_release_threadsafe_function(w->write_tsfn, napi_tsfn_release); + free(w->script); free(w->inputs_json); free(w->input_name); free(w->input_mime_type); free(w->input_charset); free(w); + bridge_end_op(pinned, /*env_still_alive=*/true); + uv_mutex_lock(&g_mutex); g_active_ops--; uv_cond_broadcast(&g_teardown_cond); uv_mutex_unlock(&g_mutex); + napi_throw_error(env, NULL, "runScriptTransformEngine: failed to create promise"); + return NULL; + } + + // Round-11 (#2): the pin was taken at admission (bridge_begin_op_locked) in + // the same critical section as g_active_ops, so a concurrent destroyEngine + // could never free this bridge under the admitted op. Just record it on w; + // the completion sentinel releases it via bridge_end_op. NULL for a + // resolver-less/unknown engine, handled everywhere as a no-op. + w->bridge = pinned; uv_thread_options_t opts; opts.flags = UV_THREAD_HAS_STACK_SIZE; opts.stack_size = 2 * 1024 * 1024; - uv_thread_create_ex(&w->tid, &opts, transform_thread_fn, w); + int spawn_rc = uv_thread_create_ex(&w->tid, &opts, transform_thread_fn, w); + + if (spawn_rc != 0) { + // The worker never ran, so nothing will ever decrement g_active_ops, + // release the bridge hold, or resolve the promise -- unwind everything + // committed above ourselves, in reverse order, mirroring + // call_js_transform_write's completion branch (minus uv_thread_join: + // there is no thread to join). + uv_mutex_lock(&g_mutex); + g_active_ops--; + uv_cond_broadcast(&g_teardown_cond); + uv_mutex_unlock(&g_mutex); + + // Synchronous call on the JS thread -- env is live here. + bridge_end_op(w->bridge, /*env_still_alive=*/true); + napi_release_threadsafe_function(w->read_tsfn, napi_tsfn_release); + napi_release_threadsafe_function(w->write_tsfn, napi_tsfn_release); + + napi_value result; + napi_create_string_utf8(env, "{\"success\":false,\"error\":\"Failed to spawn transform worker thread\"}", NAPI_AUTO_LENGTH, &result); + napi_resolve_deferred(env, w->deferred, result); + + free(w->script); + free(w->inputs_json); + free(w->input_name); + free(w->input_mime_type); + free(w->input_charset); + free(w); + } return promise; } @@ -724,35 +1775,36 @@ static napi_value napi_run_script_transform(napi_env env, napi_callback_info inf // --- Resolver callback bridge --- // Called by native code, synchronously, on the same JS thread that invoked -// runWithResolver (see the comment on g_resolver_env above for why this must -// NOT hop through napi_threadsafe_function). Calls the JS resolver directly -// and returns its result copied onto the heap; the caller (napi_run_with_resolver) -// frees it via g_resolver_last_result after the native side has copied it. -static char* resolve_module_callback(void* thread, const char* module_path) { +// runScriptEngine for a resolver-backed engine (see the comment on +// engine_bridge_t above for why this must NOT hop through +// napi_threadsafe_function). The ctx word is the engine's own engine_bridge_t*, +// passed to Java in create_engine_with_resolver and forwarded back here. Calls +// the JS resolver directly and returns its result copied onto the heap; the +// caller frees the tracked buffers after the native side has copied them. +static char* resolve_module_callback(void* thread, void* ctx, const char* module_path) { (void)thread; - if (g_resolver_env == NULL || g_resolver_ref == NULL) { - return NULL; // No resolver set + engine_bridge_t* bridge = (engine_bridge_t*)ctx; + if (bridge == NULL || bridge->env == NULL || bridge->resolver_js == NULL) { + return NULL; // No resolver for this engine } - // Guard against cross-thread napi calls. The engine that triggers this - // callback is a process-wide singleton shared by run()/runStreaming()/ - // runTransform(); streaming and transform execute their native call on a - // background uv_thread (streaming_thread_fn/transform_thread_fn), not the - // JS thread that registered g_resolver_env/g_resolver_ref. If we're not - // on the thread that owns this napi_env, calling napi_get_reference_value + // Guard against cross-thread napi calls. Streaming and transform execute + // their native call on a background uv_thread (streaming_thread_fn/ + // transform_thread_fn), not the JS thread that created this bridge. If we're + // not on the thread that owns this napi_env, calling napi_get_reference_value // or napi_call_function here is undefined behavior (typically a crash). // Fail closed instead: report "not found", which matches the documented // built-ins-only fallback for streaming/transform. uv_thread_t current = uv_thread_self(); - if (!uv_thread_equal(¤t, &g_resolver_thread)) { + if (!uv_thread_equal(¤t, &bridge->owner)) { return NULL; } - napi_env env = g_resolver_env; + napi_env env = bridge->env; napi_value js_callback; - if (napi_get_reference_value(env, g_resolver_ref, &js_callback) != napi_ok) { + if (napi_get_reference_value(env, bridge->resolver_js, &js_callback) != napi_ok) { return NULL; } @@ -849,136 +1901,467 @@ static char* resolve_module_callback(void* thread, const char* module_path) { } // null/undefined/other → not found (result_source stays NULL) - resolver_results_track(result_source); + if (!resolver_results_track(bridge, result_source)) { + // Tracking-node allocation failed (OOM): result_source would otherwise + // be an untracked buffer that nothing ever frees. Free it here and + // report "unresolved" instead of leaking it. + free(result_source); + return NULL; + } return result_source; // Native copies this immediately; we free the original after the call. } -// N-API method: runWithResolver -static napi_value napi_run_with_resolver(napi_env env, napi_callback_info info) { - if (!g_initialized) { +// --- Per-engine N-API methods --- + +// createEngine() -> number +static napi_value napi_create_engine(napi_env env, napi_callback_info info) { + (void)info; + if (!fn_create_engine) { napi_throw_error(env, NULL, "create_engine not available in native library"); return NULL; } + + // Round-14 (#1): admission in ONE g_mutex critical section (mirrors + // bridge_finalize_registry). Require (a) a live isolate not past the point + // of no return, (b) that THIS env owns an init reference (round-13 ownership + // model: an env with no reference must not create engines on the shared + // isolate -- it could otherwise attach to an isolate another env is tearing + // down), and (c) pin the isolate with a g_active_ops reservation so + // graal_tear_down_isolate() cannot run across the attach/create below. The + // check and the g_active_ops++ cannot be split by a teardown because every + // teardown transition and the g_active_ops==0 fast path also hold g_mutex. + uv_mutex_lock(&g_mutex); + env_init_rec_t* self = env_init_rec_find_locked(env); + if (!g_initialized || g_isolate == NULL || + g_teardown_state == TEARDOWN_TEARING_DOWN || + self == NULL || self->init_refs == 0) { + uv_mutex_unlock(&g_mutex); napi_throw_error(env, NULL, "Not initialized. Call initialize() first."); return NULL; } - if (!fn_run_script_with_resolver) { - napi_throw_error(env, NULL, "run_script_with_resolver not available in native library"); + g_active_ops++; // pins the live isolate against teardown across the attach + uv_mutex_unlock(&g_mutex); + + void* thread = NULL; + if (fn_attach_thread(g_isolate, &thread) != 0) { + uv_mutex_lock(&g_mutex); g_active_ops--; uv_cond_broadcast(&g_teardown_cond); uv_mutex_unlock(&g_mutex); + napi_throw_error(env, NULL, "Failed to attach thread"); return NULL; + } + long long handle = fn_create_engine(thread); + fn_detach_thread(thread); + // A GraalVM @CEntryPoint that throws on the Java side returns the return + // type's default value instead of propagating the exception — 0 for a + // long long. The real handle registry only ever hands out handles >= 1, so + // any handle <= 0 means construction failed; never hand that back to JS as + // if it were usable. + if (handle <= 0) { + uv_mutex_lock(&g_mutex); g_active_ops--; uv_cond_broadcast(&g_teardown_cond); uv_mutex_unlock(&g_mutex); + napi_throw_error(env, NULL, "create_engine returned an invalid handle"); return NULL; + } + + // Round-9 (#1): every engine -- resolver-backed or not -- gets a per-engine + // record so destroyEngine can defer the registry removal (fn_destroy_engine) + // until this engine's in-flight streaming/transform ops drain. A resolver-less + // record leaves resolver_js/results NULL. Round-11 (#1): it now ALSO registers + // an env cleanup hook (mirroring napi_create_engine_with_resolver), because + // without one a Worker that creates a resolver-less engine and exits without + // destroyEngine() would strand this record, the Java registry entry, and the + // native-lib reference. Round-12 (#2) closed the record/registry gap via + // bridge_finalize; round-13 (#5) moved ownership of the native-lib + // initialize() reference to the env itself (env_init_rec), released by the + // env-death hook env_init_cleanup, not per-engine. + // owner is recorded for symmetry but is NOT used to restrict destruction based + // on resolver state (see the owner guard in napi_destroy_engine, which now + // fires for any record). + engine_bridge_t* rec = (engine_bridge_t*)calloc(1, sizeof(engine_bridge_t)); + if (rec == NULL) { + // Roll back the engine we just created so we don't leak a registered but + // unrecorded handle. fn_destroy_engine attaches its own thread. + if (fn_destroy_engine) { + void* t2 = NULL; + if (fn_attach_thread(g_isolate, &t2) == 0) { fn_destroy_engine(t2, handle); fn_detach_thread(t2); } + } + uv_mutex_lock(&g_mutex); g_active_ops--; uv_cond_broadcast(&g_teardown_cond); uv_mutex_unlock(&g_mutex); + napi_throw_error(env, NULL, "Failed to allocate engine record"); + return NULL; + } + rec->handle = handle; + rec->owner = uv_thread_self(); + rec->env = env; + uv_mutex_lock(&g_mutex); rec->next = g_bridges; g_bridges = rec; uv_mutex_unlock(&g_mutex); + // Round-11 (#1): register an env cleanup hook for EVERY engine, not just + // resolver-backed ones. Without it, a Worker that creates a resolver-less + // engine and exits without destroyEngine() would strand this record, the Java + // ScriptRuntime registry entry, and the native-lib reference -- leaking + // engines and blocking isolate teardown across Worker churn. bridge_env_cleanup + // + bridge_finalize already handle a resolver-less record (resolver_js == NULL): + // skip the napi_ref delete, still unlink, remove the registry entry (round-10 + // do_registry_remove=true), and free. Round-13 (#5) moved ownership of the + // native-lib initialize() reference to the env itself (env_init_rec): this + // per-engine hook no longer touches g_ref_count -- the reference is released + // by the env-death hook env_init_cleanup (or by cleanup()), so an abandoned + // env releases exactly one reference regardless of how many engines it made. + // destroyEngine removes this hook before an early free so Node never invokes + // it on freed memory. + napi_status hook_st = napi_add_env_cleanup_hook(env, bridge_env_cleanup, rec); + if (hook_st != napi_ok) { + // Creation must be all-or-nothing (round-12 #6): without a cleanup hook a + // Worker that abandons this engine would strand the record and the Java + // registry entry. Unlink, remove the registry entry, free, and throw -- + // no usable handle escapes. The record was just linked on this thread + // with in_flight==0 and its handle was never returned to JS, so no op + // can be in flight against it. + // Do NOT release the init reference here (fix round 1): this throw + // propagates to initialize()'s TS catch (dataweave.ts), which sees + // libRefAcquired==true and calls ffi.cleanup() -- that is the ONE + // release for this creation's ref, matching every sibling + // creation-failure path (invalid-handle guard, alloc failure) that also + // leaves the release to the TS catch. Releasing natively here too would + // double-decrement g_ref_count -- masked in a single-instance process + // (the guard no-ops a second release at 0) but a live UAF hazard with a + // second engine instance still holding a reference. + uv_mutex_lock(&g_mutex); + engine_bridge_t** pp = &g_bridges; + while (*pp != NULL) { if (*pp == rec) { *pp = rec->next; break; } pp = &(*pp)->next; } + uv_mutex_unlock(&g_mutex); + bridge_finalize_registry(rec); + bridge_finalize_free(rec, /*env_still_alive=*/true); + uv_mutex_lock(&g_mutex); g_active_ops--; uv_cond_broadcast(&g_teardown_cond); uv_mutex_unlock(&g_mutex); + napi_throw_error(env, NULL, "Failed to register engine cleanup hook"); return NULL; } - size_t argc = 5; - napi_value args[5]; - napi_get_cb_info(env, info, &argc, args, NULL, NULL); + napi_value out; napi_create_int64(env, (int64_t)handle, &out); + uv_mutex_lock(&g_mutex); g_active_ops--; uv_cond_broadcast(&g_teardown_cond); uv_mutex_unlock(&g_mutex); + return out; +} - if (argc < 5) { - napi_throw_error(env, NULL, "Expected 5 arguments: script, inputs, mimeType, resolverCallback, isolate"); - return NULL; +// createEngineWithResolver(resolver) -> number +static napi_value napi_create_engine_with_resolver(napi_env env, napi_callback_info info) { + if (!fn_create_engine_with_resolver) { napi_throw_error(env, NULL, "create_engine_with_resolver not available in native library"); return NULL; } + size_t argc = 1; napi_value argv[1]; + napi_get_cb_info(env, info, &argc, argv, NULL, NULL); + if (argc < 1) { napi_throw_error(env, NULL, "createEngineWithResolver requires (resolverCallback)"); return NULL; } + + engine_bridge_t* bridge = (engine_bridge_t*)calloc(1, sizeof(engine_bridge_t)); + if (bridge == NULL) { napi_throw_error(env, NULL, "Failed to allocate engine bridge"); return NULL; } + if (napi_create_reference(env, argv[0], 1, &bridge->resolver_js) != napi_ok) { + free(bridge); napi_throw_error(env, NULL, "Failed to reference resolver callback"); return NULL; } + bridge->env = env; bridge->owner = uv_thread_self(); bridge->results = NULL; - // Extract script, inputs, mimeType - size_t script_len, inputs_len, mime_len; - napi_get_value_string_utf8(env, args[0], NULL, 0, &script_len); - napi_get_value_string_utf8(env, args[1], NULL, 0, &inputs_len); - napi_get_value_string_utf8(env, args[2], NULL, 0, &mime_len); + // Round-14 (#1): same admission block as napi_create_engine. Taken AFTER the + // bridge/resolver-ref allocation (those failures touch no isolate state and + // must not decrement a reservation not yet held) and BEFORE fn_attach_thread. + uv_mutex_lock(&g_mutex); + env_init_rec_t* self = env_init_rec_find_locked(env); + if (!g_initialized || g_isolate == NULL || + g_teardown_state == TEARDOWN_TEARING_DOWN || + self == NULL || self->init_refs == 0) { + uv_mutex_unlock(&g_mutex); + napi_delete_reference(env, bridge->resolver_js); free(bridge); + napi_throw_error(env, NULL, "Not initialized. Call initialize() first."); + return NULL; + } + g_active_ops++; // pins the live isolate against teardown across the attach + uv_mutex_unlock(&g_mutex); - char* script = (char*)malloc(script_len + 1); - char* inputs = (char*)malloc(inputs_len + 1); - char* mime_type = (char*)malloc(mime_len + 1); + void* thread = NULL; + if (fn_attach_thread(g_isolate, &thread) != 0) { + uv_mutex_lock(&g_mutex); g_active_ops--; uv_cond_broadcast(&g_teardown_cond); uv_mutex_unlock(&g_mutex); + napi_delete_reference(env, bridge->resolver_js); free(bridge); + napi_throw_error(env, NULL, "Failed to attach thread"); return NULL; + } + long long handle = fn_create_engine_with_resolver(thread, resolve_module_callback, (void*)bridge); + fn_detach_thread(thread); - if (script == NULL || inputs == NULL || mime_type == NULL) { - free(script); - free(inputs); - free(mime_type); - napi_throw_error(env, NULL, "Failed to allocate memory for arguments"); + // Same invalid-handle guard as napi_create_engine: a Java-side construction + // failure surfaces here as handle == 0 (GraalVM @CEntryPoint default-value + // semantics), and any handle <= 0 is never valid. Reject before this bridge + // is linked into g_bridges or a cleanup hook is registered for it — at this + // point neither has happened, so there's nothing to unlink/unhook. Still use + // bridge_finalize (not a manual napi_delete_reference+free) because the failed + // construction may have called resolve_module_callback (e.g. during eager + // module setup) before ultimately failing, which can have already populated + // bridge->results via resolver_results_track; bridge_finalize frees those + // tracked buffers too, so nothing is dropped on the floor. + if (handle <= 0) { + uv_mutex_lock(&g_mutex); g_active_ops--; uv_cond_broadcast(&g_teardown_cond); uv_mutex_unlock(&g_mutex); + // Synchronous call on the JS thread -- env is live here. + bridge_finalize(bridge, /*env_still_alive=*/true, /*do_registry_remove=*/false); + napi_throw_error(env, NULL, "create_engine_with_resolver returned an invalid handle"); return NULL; } - napi_get_value_string_utf8(env, args[0], script, script_len + 1, NULL); - napi_get_value_string_utf8(env, args[1], inputs, inputs_len + 1, NULL); - napi_get_value_string_utf8(env, args[2], mime_type, mime_len + 1, NULL); + bridge->handle = handle; + uv_mutex_lock(&g_mutex); bridge->next = g_bridges; g_bridges = bridge; uv_mutex_unlock(&g_mutex); + // Register a per-env cleanup hook so THIS Worker/main thread disposes this + // bridge's napi_ref on its own thread when its env tears down (F2). napi_cleanup + // no longer touches bridge refs. destroyEngine removes this hook before an + // early free so Node never calls it on freed memory. + napi_status hook_st = napi_add_env_cleanup_hook(env, bridge_env_cleanup, bridge); + if (hook_st != napi_ok) { + // Creation must be all-or-nothing (round-12 #6): without a cleanup hook a + // Worker that abandons this engine would strand the record and the Java + // registry entry. Unlink, remove the registry entry, free, and throw -- + // no usable handle escapes. The record was just linked on this thread + // with in_flight==0 and its handle was never returned to JS, so no op + // can be in flight against it. + // Do NOT release the init reference here (fix round 1): this throw + // propagates to initialize()'s TS catch (dataweave.ts), which sees + // libRefAcquired==true and calls ffi.cleanup() -- that is the ONE + // release for this creation's ref, matching every sibling + // creation-failure path (resolver invalid-handle guard uses + // bridge_finalize with do_registry_remove=false and also does NOT + // release) that also leaves the release to the TS catch. Releasing + // natively here too would double-decrement g_ref_count -- masked in a + // single-instance process (the guard no-ops a second release at 0) but + // a live UAF hazard with a second engine instance still holding a + // reference. + uv_mutex_lock(&g_mutex); + engine_bridge_t** pp = &g_bridges; + while (*pp != NULL) { if (*pp == bridge) { *pp = bridge->next; break; } pp = &(*pp)->next; } + uv_mutex_unlock(&g_mutex); + bridge_finalize_registry(bridge); + bridge_finalize_free(bridge, /*env_still_alive=*/true); + uv_mutex_lock(&g_mutex); g_active_ops--; uv_cond_broadcast(&g_teardown_cond); uv_mutex_unlock(&g_mutex); + napi_throw_error(env, NULL, "Failed to register engine cleanup hook"); + return NULL; + } + napi_value out; napi_create_int64(env, (int64_t)handle, &out); + uv_mutex_lock(&g_mutex); g_active_ops--; uv_cond_broadcast(&g_teardown_cond); uv_mutex_unlock(&g_mutex); + return out; +} - // Resolver is installed once per process lifetime. Subsequent calls with - // different resolver callbacks will reuse the first resolver, as enforced by - // ScriptRuntime.setResolver() on the native side (one resolver per engine). - // - // No thread-hop machinery is needed: fn_run_script_with_resolver() below - // runs on this very thread, so resolve_module_callback() (invoked from - // inside that call) can call directly back into JS via the stored - // napi_ref. See the comment on g_resolver_env for why napi_threadsafe_function - // must NOT be used here. +// destroyEngine(handle) -> void +static napi_value napi_destroy_engine(napi_env env, napi_callback_info info) { + if (!g_initialized) return NULL; + size_t argc = 1; napi_value argv[1]; + napi_get_cb_info(env, info, &argc, argv, NULL, NULL); + if (argc < 1) { napi_throw_error(env, NULL, "destroyEngine requires (handle)"); return NULL; } + int64_t handle64; + if (napi_get_value_int64(env, argv[0], &handle64) != napi_ok) { + napi_throw_error(env, NULL, "destroyEngine: handle must be an integer"); + return NULL; + } + long long handle = (long long)handle64; + + // F2: a resolver-backed engine's bridge owns thread-affine N-API state -- + // a napi_ref and an env cleanup hook, both created on the engine's owning + // JS thread. Deleting that ref (bridge_finalize) or removing that hook + // (napi_remove_env_cleanup_hook) from another Worker's thread is undefined + // behavior. Reject cross-thread destruction, mirroring the fail-closed + // owner check in resolve_module_callback; the owner env's cleanup hook + // disposes the bridge when that Worker tears down. We are on the owner + // thread past this point, so the env cannot be concurrently tearing down + // and the bridge stays stable between this check and the unlink below. + // Owner-thread guard: round-11 (#1) registers an env cleanup hook for EVERY + // engine (resolver-backed or not), so every record now carries env-affine + // N-API state -- napi_remove_env_cleanup_hook (called below before an early + // free) can only be invoked legally on the owner thread. The guard + // therefore fires for any record (owned != NULL), not just resolver-backed + // ones. bridge_finalize's napi_ref deletion stays resolver-gated + // (resolver_js != NULL && env != NULL) -- that part is unchanged. uv_mutex_lock(&g_mutex); - if (g_resolver_ref == NULL) { - napi_status status = napi_create_reference(env, args[3], 1, &g_resolver_ref); - if (status != napi_ok) { + engine_bridge_t* owned = bridge_find(handle); + if (owned != NULL) { + uv_thread_t self = uv_thread_self(); + if (!uv_thread_equal(&self, &owned->owner)) { uv_mutex_unlock(&g_mutex); - free(script); - free(inputs); - free(mime_type); - napi_throw_error(env, NULL, "Failed to reference resolver callback"); + napi_throw_error(env, NULL, + "destroyEngine must be called from the thread that created the engine"); return NULL; } - g_resolver_env = env; - g_resolver_thread = uv_thread_self(); } - // Note: subsequent calls reuse the first resolver for this process lifetime. + + // Round-9 (#1): unlink the record and decide, under the lock, whether the + // registry removal (fn_destroy_engine) and the record free must be DEFERRED. + // If an op is in flight, its worker may not yet have called + // ScriptRuntime.get(handle) (the first statement of the Java entrypoint) -- + // removing the registry entry now would make that lookup fail with + // "Unknown engine handle". So defer BOTH the registry removal and the free + // to the last op draining (bridge_end_op -> bridge_finalize with + // do_registry_remove=true), which runs on this same owner thread. When no op + // is in flight, remove the registry entry and finalize immediately, as + // before. Every engine now has a record, so `found` is non-NULL for both + // resolver-backed and resolver-less engines. + engine_bridge_t** pp = &g_bridges; engine_bridge_t* found = NULL; + while (*pp != NULL) { if ((*pp)->handle == handle) { found = *pp; *pp = found->next; break; } pp = &(*pp)->next; } + bool defer = false; + // deferred_registry_remove gates the deferred registry removal in + // bridge_end_op; set it together with destroy_pending here. + if (found != NULL && found->in_flight > 0) { found->destroy_pending = true; found->deferred_registry_remove = true; defer = true; } uv_mutex_unlock(&g_mutex); - // Need to attach thread for this call - void* thread = NULL; - int rc = fn_attach_thread(g_isolate, &thread); - if (rc != 0) { - free(script); - free(inputs); - free(mime_type); - napi_throw_error(env, NULL, "Failed to attach thread"); - return NULL; + if (found != NULL) { + // Drop the env cleanup hook. Round-11 (#1): every engine now registers + // one at creation (napi_create_engine / napi_create_engine_with_resolver), + // so this removal must run unconditionally, not just for resolver-backed + // engines. Whether we finalize now or defer, the free happens explicitly, + // so Node must never invoke the hook on this (soon-to-be or already) + // freed record. + napi_remove_env_cleanup_hook(env, bridge_env_cleanup, found); + if (!defer) { + // Not in flight: remove the registry entry AND finalize now, on this + // owner thread (env live). do_registry_remove=true folds the + // fn_destroy_engine call into bridge_finalize so it happens exactly + // once regardless of path. + bridge_finalize(found, /*env_still_alive=*/true, /*do_registry_remove=*/true); + } + // else: the draining op's bridge_end_op -> bridge_finalize performs both + // the registry removal and the free (see Step 5). + } else { + // No record found (should not happen now that every engine has one, but + // stay robust to a double-destroy or an unknown handle): fall back to the + // pre-round-9 behavior of removing the registry entry directly. + if (fn_destroy_engine) { + void* thread = NULL; + if (fn_attach_thread(g_isolate, &thread) == 0) { fn_destroy_engine(thread, handle); fn_detach_thread(thread); } + } } + return NULL; +} - // Call native with resolver callback. mime_type is accepted from JS for API - // symmetry but is not part of the native run_script_with_resolver signature - // (see run_script_with_resolver_fn typedef comment) — do not forward it. - char* result = fn_run_script_with_resolver( - thread, - script, - inputs, - resolve_module_callback - ); - - // Native has copied every resolver result returned during this call; free - // our copies now that it's done. - resolver_results_free_all(); - - // result (if non-NULL) is a GraalVM UnmanagedMemory.malloc'd buffer, like - // every other native result pointer in this file; it must be released via - // fn_free_cstring(), not libc free(), and while the isolate thread is - // still attached. Copy it to a libc-owned buffer first so we can build - // the JS string after detaching, matching the strdup + fn_free_cstring - // pattern used by run_script_thread_fn/streaming_thread_fn/transform_thread_fn. - char* result_copy = result ? strdup(result) : NULL; - if (result != NULL) { - fn_free_cstring(thread, result); +// runScriptEngine(handle, script, inputsJson) -> string +static napi_value napi_run_script_engine(napi_env env, napi_callback_info info) { + if (!g_initialized) { napi_throw_error(env, NULL, "Not initialized. Call initialize() first."); return NULL; } + if (!fn_run_script_engine) { napi_throw_error(env, NULL, "run_script_engine not available in native library"); return NULL; } + size_t argc = 3; napi_value argv[3]; + napi_get_cb_info(env, info, &argc, argv, NULL, NULL); + if (argc < 3) { napi_throw_error(env, NULL, "runScriptEngine requires (handle, script, inputsJson)"); return NULL; } + int64_t handle64; + if (napi_get_value_int64(env, argv[0], &handle64) != napi_ok) { + napi_throw_error(env, NULL, "runScriptEngine: handle must be an integer"); + return NULL; } + long long handle = (long long)handle64; - fn_detach_thread(thread); + size_t script_len, inputs_len; + if (napi_get_value_string_utf8(env, argv[1], NULL, 0, &script_len) != napi_ok) { + napi_throw_error(env, NULL, "runScriptEngine: script must be a string"); + return NULL; + } + if (napi_get_value_string_utf8(env, argv[2], NULL, 0, &inputs_len) != napi_ok) { + napi_throw_error(env, NULL, "runScriptEngine: inputsJson must be a string"); + return NULL; + } + char* script = (char*)malloc(script_len + 1); + char* inputs = (char*)malloc(inputs_len + 1); + if (script == NULL || inputs == NULL) { free(script); free(inputs); napi_throw_error(env, NULL, "OOM"); return NULL; } + if (napi_get_value_string_utf8(env, argv[1], script, script_len + 1, NULL) != napi_ok || + napi_get_value_string_utf8(env, argv[2], inputs, inputs_len + 1, NULL) != napi_ok) { + free(script); free(inputs); + napi_throw_error(env, NULL, "runScriptEngine: failed to read script/inputsJson"); + return NULL; + } - free(script); - free(inputs); - free(mime_type); + // Round-7 #1: reserve an active op across the isolate-touching window + // (attach -> run -> detach) so a concurrent Worker's last cleanup() + // (napi_cleanup Case 4) cannot observe g_active_ops == 0 and tear down + // g_isolate while this synchronous op is attaching to or executing in it. + // Reserve LATE (here, not at the top): the malloc/arg-extraction above do + // not touch the isolate, so the reservation only needs to span attach.. + // detach -- giving exactly two unwind sites (attach-failure and normal + // completion) instead of also unwinding the OOM path. Rejecting on + // g_teardown_state != TEARDOWN_NONE also refuses to start once a teardown + // is queued/underway. run() is fully synchronous on the JS thread, so the + // reserve and release both happen inline (no worker thread). Admit an + // ADOPTED isolate: napi_initialize's adoption branch sets + // g_teardown_cancelled = true on a still-live PENDING_WAIT isolate but + // does not reset g_teardown_state (only the async waiter does), so a + // merely-cancelled teardown must not reject here -- otherwise a valid + // post-adoption op throws "Not initialized". A genuine (non-cancelled) + // PENDING_WAIT or a committed TEARING_DOWN still rejects. + uv_mutex_lock(&g_mutex); + if (!g_initialized || (g_teardown_state != TEARDOWN_NONE && !g_teardown_cancelled)) { + uv_mutex_unlock(&g_mutex); + free(script); free(inputs); + napi_throw_error(env, NULL, "Not initialized. Call initialize() first."); + return NULL; + } + g_active_ops++; + // Round-11 (#3): pin the engine in the same critical section as the + // g_active_ops reservation so a concurrent destroyEngine cannot free the + // resolver bridge (still held by Java as the resolver ctx) while this + // synchronous op attaches to Graal or runs. NULL for a resolver-less/unknown + // handle -- bridge_end_op no-ops on NULL. Released in the attach-failure and + // completion paths below, alongside g_active_ops. + engine_bridge_t* bridge = bridge_begin_op_locked(handle); + uv_mutex_unlock(&g_mutex); - if (result_copy == NULL) { - napi_throw_error(env, NULL, "Script execution failed"); - return NULL; + void* thread = NULL; + if (fn_attach_thread(g_isolate, &thread) != 0) { + bridge_end_op(bridge, /*env_still_alive=*/true); + uv_mutex_lock(&g_mutex); g_active_ops--; uv_cond_broadcast(&g_teardown_cond); uv_mutex_unlock(&g_mutex); + free(script); free(inputs); + napi_throw_error(env, NULL, "Failed to attach thread"); + return NULL; } - napi_value result_str; - napi_create_string_utf8(env, result_copy, NAPI_AUTO_LENGTH, &result_str); - free(result_copy); + char* result = (char*)fn_run_script_engine(thread, handle, script, inputs); + + // The pin taken at admission kept this record alive across the run, so no + // second lookup is needed. resolver_results_free_all is a no-op for a + // resolver-less/unknown engine (bridge == NULL). + if (bridge != NULL) resolver_results_free_all(bridge); - return result_str; + char* result_copy = result ? strdup(result) : NULL; + if (result != NULL) fn_free_cstring(thread, result); + fn_detach_thread(thread); + free(script); free(inputs); + + // Round-11 (#3): release the per-engine pin (may finalize a destroy that a + // concurrent Worker deferred while this op held in_flight > 0), then release + // the global op reservation. env is live on this JS thread, so env_still_alive + // is true. Order: bridge_end_op before the g_active_ops release, mirroring + // streaming/transform completion. + bridge_end_op(bridge, /*env_still_alive=*/true); + uv_mutex_lock(&g_mutex); g_active_ops--; uv_cond_broadcast(&g_teardown_cond); uv_mutex_unlock(&g_mutex); + + napi_value out; + if (result_copy) { napi_create_string_utf8(env, result_copy, NAPI_AUTO_LENGTH, &out); free(result_copy); } + else { napi_create_string_utf8(env, "", 0, &out); } + return out; } // --- Cleanup (must run on a separate thread to avoid V8 signal handler conflict) --- +// Called on each waiter's own env/thread (via its own napi_threadsafe_function) +// once the waiter thread has finished isolate teardown. Resolves that specific +// caller's promise, then releases its tsfn and frees the node. `data` is +// unused (NULL) -- there is nothing to report beyond "done". +// +// napi_call_threadsafe_function(..., napi_tsfn_blocking) only ENQUEUES this +// callback for the target env's event loop to run later; it does not wait for +// it to actually execute. So the waiter node and its tsfn must stay alive +// until this callback runs and must be released/freed HERE, not by the +// thread that enqueued the call (teardown_waiter_thread_fn) -- freeing there +// right after the enqueueing call would be a use-after-free once this +// callback later dereferences `context`. Same ownership pattern as +// call_js_write/call_js_transform_write freeing their own work struct from +// inside their own completion branch. +static void call_js_teardown_done(napi_env env, napi_value js_callback, void* context, void* data) { + (void)js_callback; + (void)data; + teardown_waiter_t* waiter = (teardown_waiter_t*)context; + if (waiter == NULL) return; + + if (env != NULL) { + napi_value undefined; + napi_get_undefined(env, &undefined); + napi_resolve_deferred(env, waiter->deferred, undefined); + } + + napi_release_threadsafe_function(waiter->tsfn, napi_tsfn_release); + free(waiter); +} + +// `arg` is an int* out-param: the caller (napi_cleanup's case 4) must set it +// to 0 before spawning this thread and read it after uv_thread_join returns. +// Mirrors teardown_waiter_thread_fn's `torn_down` local exactly, so the +// caller can tell "isolate torn down / nothing to tear down" (safe to clear +// g_thread/g_isolate/g_initialized/g_ref_count) apart from "attach failed, +// isolate still alive" (must leave those globals set, or the isolate becomes +// unreachable and can never be torn down). static void cleanup_thread_fn(void* arg) { - (void)arg; + int* out_torn_down = (int*)arg; // graal_tear_down_isolate() must be passed the IsolateThread belonging to the // *calling* OS thread. g_thread was created by graal_create_isolate() on the // (now-exited, already-joined) init thread, so it is invalid here — passing it @@ -986,49 +2369,533 @@ static void cleanup_thread_fn(void* arg) { // StackOverflowError during teardown. Attach this cleanup thread to the isolate // to obtain a valid local IsolateThread, then tear down with that. if (!fn_tear_down_isolate || !fn_attach_thread || !g_isolate) { + // Nothing to tear down (no isolate / FFI unavailable) -- safe to clear. + *out_torn_down = 1; return; } void* local_thread = NULL; if (fn_attach_thread(g_isolate, &local_thread) != 0 || local_thread == NULL) { + // Attach failed -- the isolate is still alive. Leave *out_torn_down at 0 + // (its caller-initialized value) so the caller does NOT clear g_isolate, + // or it becomes unreachable and can never be torn down. return; } - fn_tear_down_isolate(local_thread); + // Check the teardown return code (0 == success). On nonzero the isolate is + // still live: leave *out_torn_down at 0 so the caller retains + // g_isolate/g_initialized/g_ref_count and (per its own logic) arms the retry, + // rather than orphaning a live isolate (review #6 #3). On that failure the + // isolate was NOT destroyed, so this thread is still attached to it -- detach + // before the helper thread exits, or the live isolate keeps a phantom + // attached thread that can make a later retry teardown block or fail (review + // #7 #1). On success the isolate is gone: do NOT detach (would be a UAF). + if (fn_tear_down_isolate(local_thread) == 0) { + *out_torn_down = 1; + } else { + fn_detach_thread(local_thread); + *out_torn_down = 0; + } } -static napi_value napi_cleanup(napi_env env, napi_callback_info info) { +// Spawned only when napi_cleanup finds g_active_ops > 0 on the last release +// (case 5 in the design doc). Blocks until every active streaming/transform +// op has drained, performs isolate teardown exactly like cleanup_thread_fn +// does on the unchanged fast path, then resolves every caller who is waiting +// on this same teardown (there may be more than one -- see g_teardown_waiters). +static void teardown_waiter_thread_fn(void* arg) { + (void)arg; + uv_mutex_lock(&g_mutex); - if (g_initialized) { - g_ref_count--; - if (g_ref_count <= 0) { - // Clean up resolver reference - if (g_resolver_ref != NULL && g_resolver_env != NULL) { - napi_delete_reference(g_resolver_env, g_resolver_ref); + while (g_active_ops > 0 && !g_teardown_cancelled) { + uv_cond_wait(&g_teardown_cond, &g_mutex); + } + bool cancelled = g_teardown_cancelled; + if (!cancelled) { + // Point of no return: from here an adopting initialize() must NOT reuse the + // isolate, so publish TEARING_DOWN under the lock before we drop it to call + // graal_tear_down_isolate(). + g_teardown_state = TEARDOWN_TEARING_DOWN; + } + uv_mutex_unlock(&g_mutex); + + // Perform teardown exactly as the unchanged fast path does: attach a local + // thread to the isolate (g_thread from graal_create_isolate's bootstrap + // thread is invalid here -- see cleanup_thread_fn's comment), then tear + // down. Honor the return code (0 == success); a nonzero teardown leaves the + // isolate live (review #6 #3). Skipped entirely + // when an initialize() call adopted the live isolate instead (see + // napi_initialize's TEARDOWN_PENDING_WAIT branch). + bool torn_down = false; + if (!cancelled && fn_tear_down_isolate && fn_attach_thread && g_isolate) { + void* local_thread = NULL; + if (fn_attach_thread(g_isolate, &local_thread) == 0 && local_thread != NULL) { + // Check the teardown return code (0 == success). On nonzero the isolate is + // still live -- leave torn_down false so the post-teardown block below + // retains the isolate globals and arms the retry (review #6 #3). On that + // failure the isolate was NOT destroyed, so this thread is still attached + // to it -- detach before exiting or the live isolate keeps a phantom + // attached thread that can block/fail a later retry teardown (review #7 + // #1). On success the isolate is gone: do NOT detach (would be a UAF). + if (fn_tear_down_isolate(local_thread) == 0) { + torn_down = true; + } else { + fn_detach_thread(local_thread); + torn_down = false; } - g_resolver_ref = NULL; - g_resolver_env = NULL; - resolver_results_free_all(); - - uv_thread_t tid; - uv_thread_options_t opts; - opts.flags = UV_THREAD_HAS_STACK_SIZE; - opts.stack_size = 2 * 1024 * 1024; - uv_thread_create_ex(&tid, &opts, cleanup_thread_fn, NULL); + } + // else: attach failed -- the isolate is still alive. Do NOT clear g_isolate, + // or it becomes unreachable and can never be torn down. + } else if (!cancelled) { + // Nothing to tear down (no isolate / FFI unavailable) -- safe to clear. + torn_down = true; + } + // if (cancelled): leave torn_down = false -- the isolate stays live for the + // adopter; we tear nothing down. + + uv_mutex_lock(&g_mutex); + if (!cancelled && torn_down) { + g_thread = NULL; + g_isolate = NULL; + g_initialized = 0; + g_ref_count = 0; + } else if (!cancelled && g_isolate != NULL && g_ref_count == 0) { + // Teardown did not happen (attach failed, or graal_tear_down_isolate + // returned nonzero -- review #6 #3) and this async-waiter path IS the last + // release: g_ref_count is already 0 with no owner and no pending waiter. + // Arm the retry signal so a later op-completion drain or a fresh + // initialize() retries teardown -- otherwise the live isolate is stranded + // with nothing to reclaim it (review #6 #4). Mirrors the twin arm in + // isolate_ref_release_n_locked's waiter-spawn-failure path. + g_teardown_needed = true; + } + // If cancelled: g_isolate/g_initialized/g_ref_count are left exactly as the + // adopting initialize() set them (it already did g_ref_count++ on the live + // isolate). + g_teardown_state = TEARDOWN_NONE; + g_teardown_cancelled = false; + // Release any initialize() call blocked waiting for teardown to finish + // (see Task 3). + uv_cond_broadcast(&g_teardown_cond); + teardown_waiter_t* waiters = g_teardown_waiters; + g_teardown_waiters = NULL; + uv_mutex_unlock(&g_mutex); + + // Resolve every waiting caller's promise on its own env/thread via its own + // tsfn -- napi_deferred/napi_env are thread-affine, so this cannot be done + // from this waiter thread directly. napi_call_threadsafe_function only + // ENQUEUES the call for the target thread to run later; it does not wait + // for call_js_teardown_done to execute. So do NOT free/release here -- + // call_js_teardown_done owns and releases each node after it actually runs + // (freeing it here instead would be a use-after-free the moment the + // enqueued callback later dereferences it). + while (waiters != NULL) { + teardown_waiter_t* next = waiters->next; + napi_status enq = napi_call_threadsafe_function(waiters->tsfn, waiters, napi_tsfn_blocking); + if (enq != napi_ok) { + // The waiter's env is tearing down (napi_closing): call_js_teardown_done + // will never run, so it can neither resolve waiter->deferred nor release + // the tsfn nor free the node. Free the node here instead of leaking it + // (one leak per Worker that terminated while this teardown was pending). + // Do NOT napi_release_threadsafe_function(waiters->tsfn, ...): a + // napi_closing return already discharges this tsfn's registration (Node + // may have destroyed the tsfn object), so a release would be a + // double-discharge/UAF -- same reasoning as the sentinel-enqueue-failure + // paths in streaming_thread_fn/transform_thread_fn. The unresolved + // deferred is env-affine and reclaimed when the dead env is destroyed. + free(waiters); + } + waiters = next; + } +} + +// Creates a promise, a threadsafe function bound to call_js_teardown_done for +// THIS call's env, and a teardown_waiter_t node carrying both. The node is +// NOT linked into g_teardown_waiters here -- the caller does that under +// g_mutex, since callers append at two different points in napi_cleanup +// (case 3: joining an existing pending teardown; case 5: starting a new one). +// Returns NULL (and throws) if node allocation fails. +static teardown_waiter_t* teardown_waiter_create(napi_env env, napi_value* out_promise) { + teardown_waiter_t* waiter = (teardown_waiter_t*)calloc(1, sizeof(teardown_waiter_t)); + if (waiter == NULL) { + napi_throw_error(env, NULL, "Failed to allocate teardown waiter"); + return NULL; + } + waiter->env = env; + + if (napi_create_promise(env, &waiter->deferred, out_promise) != napi_ok) { + free(waiter); + napi_throw_error(env, NULL, "Failed to create teardown promise"); + return NULL; + } + + napi_value resource_name; + if (napi_create_string_utf8(env, "dwTeardown", NAPI_AUTO_LENGTH, &resource_name) != napi_ok) { + free(waiter); + napi_throw_error(env, NULL, "Failed to create teardown resource name"); + return NULL; + } + + if (napi_create_threadsafe_function( + env, NULL, NULL, resource_name, 0, 1, NULL, NULL, waiter, call_js_teardown_done, &waiter->tsfn + ) != napi_ok) { + free(waiter); + napi_throw_error(env, NULL, "Failed to create teardown threadsafe function"); + return NULL; + } + + return waiter; +} + +// Creates an already-resolved promise -- used by napi_cleanup's two +// "nothing to wait for" branches (not-the-last-release, and last-release +// with no active ops) so the function's return type is uniformly "a +// promise" regardless of which branch runs. +static napi_value already_resolved_promise(napi_env env) { + napi_deferred deferred; + napi_value promise; + napi_create_promise(env, &deferred, &promise); + napi_value undefined; + napi_get_undefined(env, &undefined); + napi_resolve_deferred(env, deferred, undefined); + return promise; +} + +// Release n (>=0) initialization references at once, then make the teardown +// decision AT MOST ONCE. Caller holds g_mutex and this KEEPS it held. n==0 is a +// no-op. Equivalent to n serial single-releases for the COUNT, but guarantees +// the reached-zero teardown/waiter logic runs exactly once (a serial loop would +// re-enter the decision on an already-zero count). Used by env_init_cleanup +// (round-13 #5) to release all of a dead env's references from one decision +// point. (Previously also used by a single-release wrapper, +// isolate_ref_release_core_locked, retired in round-13 #5 once the per-engine +// finalize path stopped releasing init references directly.) +// Round-14 (#2/#3): retry a teardown that a prior last-release could not carry +// out. Caller holds g_mutex and this KEEPS it held. No-op unless a stranded +// live isolate is waiting (g_teardown_needed) with no owners and no teardown in +// progress and ops drained. Makes the reached-zero teardown decision at most +// once per call (same synchronous cleanup_thread_fn path as Case 4); on repeated +// failure it leaves g_teardown_needed set to retry on the next drain. Spawns+joins +// cleanup_thread_fn while holding g_mutex, exactly as the Case-4 / +// isolate_ref_release_n_locked g_active_ops==0 branch does; cleanup_thread_fn +// takes no lock and makes no napi call, so this is deadlock-free and thread-safe +// from any drain site. +static void retry_stranded_teardown_locked(void) { + if (!g_teardown_needed) return; + if (g_ref_count > 0) { g_teardown_needed = false; return; } // adopted -> keep + if (g_teardown_state != TEARDOWN_NONE) return; // a teardown drives + if (g_active_ops > 0) return; // wait for drain + if (g_isolate == NULL) { g_teardown_needed = false; return; } // nothing to do + uv_thread_t tid; + uv_thread_options_t opts; + opts.flags = UV_THREAD_HAS_STACK_SIZE; + opts.stack_size = 2 * 1024 * 1024; + int torn_down = 0; + int spawn_rc = uv_thread_create_ex(&tid, &opts, cleanup_thread_fn, &torn_down); + if (spawn_rc == 0) uv_thread_join(&tid); + if (torn_down) { + g_thread = NULL; + g_isolate = NULL; + g_initialized = 0; + g_ref_count = 0; + g_teardown_needed = false; + } + // else: spawn/attach failed again -- leave g_teardown_needed set so the next + // drain (or a later initialize() adoption) retries. +} + +static void isolate_ref_release_n_locked(int n) { + if (n <= 0) return; + if (g_ref_count >= n) g_ref_count -= n; else g_ref_count = 0; + if (g_ref_count > 0) return; // other envs still hold references + if (g_teardown_state != TEARDOWN_NONE) return; // a teardown already drives + + if (g_active_ops == 0) { + uv_thread_t tid; + uv_thread_options_t opts; + opts.flags = UV_THREAD_HAS_STACK_SIZE; + opts.stack_size = 2 * 1024 * 1024; + int torn_down = 0; + int spawn_rc = uv_thread_create_ex(&tid, &opts, cleanup_thread_fn, &torn_down); + if (spawn_rc == 0) { uv_thread_join(&tid); + } + if (torn_down) { + g_thread = NULL; + g_isolate = NULL; + g_initialized = 0; + g_ref_count = 0; + } else if (g_isolate != NULL && g_ref_count == 0) { + // Sync teardown failed (spawn or cleanup_thread_fn attach) with the isolate + // still live and no owners: arm the retry signal (round-14 #3). g_active_ops + // is already 0 here, but a later op could still re-pin; the flag is cleared + // on adoption and retried on drain or by the next initialize() (review #6 + // #5). Documented residual: if NO later op or initialize() ever occurs, the + // isolate lingers until process exit, where the OS reclaims it -- benign + // (single process-lifetime isolate, no ref-count violation). + g_teardown_needed = true; + } + return; + } + + // g_active_ops > 0: defer to the waiter thread, no promises attached. + g_teardown_state = TEARDOWN_PENDING_WAIT; + g_teardown_cancelled = false; + g_teardown_waiters = NULL; // no JS caller waiting + uv_thread_t waiter_tid; + uv_thread_options_t waiter_opts; + waiter_opts.flags = UV_THREAD_HAS_STACK_SIZE; + waiter_opts.stack_size = 2 * 1024 * 1024; + int spawn_rc = uv_thread_create_ex(&waiter_tid, &waiter_opts, teardown_waiter_thread_fn, NULL); + if (spawn_rc != 0) { + // Best-effort degradation: the waiter thread never started, so nothing will + // drain the isolate. Restore g_ref_count to the true remaining ownership + // (Σ init_refs, = 0 here) to keep the invariant, and ARM the retry signal so + // the next op-completion drain retries teardown -- otherwise this live + // isolate has zero owners and nothing would ever tear it down (round-14 #3). + g_teardown_state = TEARDOWN_NONE; + g_ref_count = env_init_refs_total_locked(); + if (g_isolate != NULL && g_ref_count == 0) g_teardown_needed = true; + } +} + +// Env-death hook for a per-env init record (round-13 #5). Registered once per +// env by initialize()'s first acquire (env_init_acquire_and_hook). Node runs +// env-cleanup hooks LIFO. In the normal initialize()-then-createEngine() order +// this hook is registered BEFORE any engine's bridge_env_cleanup for the same +// env, so it runs AFTER every engine bridge has finalized on a live isolate. +// The pathological raw-ffi order (createEngine() on this env -- succeeding +// because another env already initialized -- THEN initialize() here) can +// register this hook after an engine hook, so it may run first; that is still +// safe, because bridge_finalize_registry re-checks teardown state under g_mutex +// (registry removal no-ops on a torn-down isolate) and the napi_ref delete runs +// with env_still_alive=true on this env's own live thread. Releases exactly the +// references this env still holds (n), from a single env-scoped decision point: +// because g_ref_count == sum of init_refs, releasing this env's n reaches zero +// ONLY if no other env holds a reference, so an abandoned env can never tear the +// isolate down under a live env. Runs on the dying env's own thread with the +// env alive; does only g_mutex-guarded integer/list work + free (no env-affine +// napi calls). +// Round-14 (#1): the create path now enforces per-env ownership (an env with +// init_refs == 0 is rejected), so the pathological order below -- createEngine() +// on this env BEFORE its own initialize() -- is now rejected at the create call +// rather than relying on the finalize-time teardown-state re-check. +static void env_init_cleanup(void* arg) { + env_init_rec_t* rec = (env_init_rec_t*)arg; + if (rec == NULL) return; + uv_mutex_lock(&g_mutex); + // Unlink from g_env_recs if still present. + env_init_rec_t** pp = &g_env_recs; + while (*pp != NULL) { + if (*pp == rec) { *pp = rec->next; break; } + pp = &(*pp)->next; + } + int n = rec->init_refs; + rec->init_refs = 0; + free(rec); + // Release all n references and make the teardown decision at most once. + isolate_ref_release_n_locked(n); + uv_mutex_unlock(&g_mutex); +} + +// Promise-less core of an isolate-reference release. Caller holds g_mutex and +// this function KEEPS it held (does not unlock). Decrements g_ref_count and, on +// the last release, drives teardown WITHOUT binding any napi promise/waiter: +// - g_active_ops == 0 -> synchronous cleanup_thread_fn (same as Case 4). +// - g_active_ops > 0 -> spawn the waiter thread with an EMPTY waiter list +// (TEARDOWN_PENDING_WAIT); it tears down (or is adopted) +// with no promises to resolve. +// - a teardown already pending (TEARDOWN_NONE != state) -> nothing to do; the +// existing waiter will tear down; this release just +// drops the count. +// Used by env_init_cleanup (round-13 #5), the env-death hook, which has no +// live JS caller to hand a promise to. +// +// Deliberately does NOT call (or get called by) release_isolate_ref_locked +// below: that promise-bearing sibling needs per-caller promise plumbing this +// core omits on purpose (binding a waiter/promise to a tearing-down env is a +// thread-affinity hazard). They share the last-release *policy* only; see +// release_isolate_ref_locked's header comment for the promise-bearing twin. +// +// The isolate reference is now owned per env (env_init_rec), not per engine +// bridge (round-13 #5): initialize()'s acquire sites and env_init_cleanup are +// the only callers that mutate g_ref_count via this function, alongside +// release_isolate_ref_locked below for the explicit cleanup() path. The +// per-engine finalize path (bridge_env_cleanup / bridge_end_op) no longer +// touches g_ref_count at all, so a raw multi-engine-per-initialize() caller's +// abandoned env fires exactly one release for the whole balance it holds, +// regardless of how many engines it created. + +// Releases ONE initialization reference on the shared isolate. Caller MUST +// hold g_mutex; this function UNLOCKS g_mutex before returning (the sync and +// waiter teardown paths both require dropping the lock). Returns the napi +// promise to hand back to the JS caller. This is napi_cleanup's original +// Case 1..5 body. +static napi_value release_isolate_ref_locked(napi_env env) { + // Case 1/2: not the last release (or nothing was ever initialized). Decrement + // only if positive -- a second cleanup() call while g_ref_count is already at + // 0 (e.g. one already dropped it while teardown is pending) must not go + // negative. + // Round-13 (#5): an env may release only a reference IT owns. If this env has + // no outstanding init reference (a cleanup() with no matching initialize() on + // this env, or a double-cleanup()), do NOT touch g_ref_count -- releasing here + // would steal another env's reference and could tear the isolate down under a + // live user. No-op: resolve immediately. (g_ref_count == sum of init_refs, so + // this env's zero balance means it contributes nothing to release.) + env_init_rec_t* self = env_init_rec_find_locked(env); + if (self == NULL || self->init_refs == 0) { + uv_mutex_unlock(&g_mutex); + return already_resolved_promise(env); + } + self->init_refs--; + if (g_ref_count > 0) { + g_ref_count--; + } + if (g_ref_count > 0) { + uv_mutex_unlock(&g_mutex); + return already_resolved_promise(env); + } + // Case 3: a teardown from an earlier cleanup() call is already pending + // (possibly triggered from a different Worker/env). Join its waiter list + // instead of spawning a second waiter thread. + if (g_teardown_state != TEARDOWN_NONE) { + napi_value promise; + teardown_waiter_t* waiter = teardown_waiter_create(env, &promise); + if (waiter == NULL) { + uv_mutex_unlock(&g_mutex); + return NULL; // teardown_waiter_create already threw + } + waiter->next = g_teardown_waiters; + g_teardown_waiters = waiter; + uv_mutex_unlock(&g_mutex); + return promise; + } + + // Case 4: last release, no teardown pending, and nothing active -- the + // original, unchanged synchronous fast path. + if (g_active_ops == 0) { + uv_thread_t tid; + uv_thread_options_t opts; + opts.flags = UV_THREAD_HAS_STACK_SIZE; + opts.stack_size = 2 * 1024 * 1024; + // torn_down is cleanup_thread_fn's out-param (mirrors teardown_waiter_thread_fn's + // `torn_down` local exactly): must be initialized to 0 before the thread runs so + // the attach-failure early-return path (which never touches it) leaves it false. + // uv_thread_join is synchronous, so when spawn_rc == 0 this stack variable safely + // outlives the thread's write to it. + int torn_down = 0; + int spawn_rc = uv_thread_create_ex(&tid, &opts, cleanup_thread_fn, &torn_down); + if (spawn_rc == 0) { + uv_thread_join(&tid); + } + // Only clear global state if the isolate was actually torn down (or there + // was nothing to tear down). If spawn failed, the thread never ran and + // torn_down stays 0 -- leave the globals set rather than orphaning a live + // isolate (unreachable via these globals, could never be torn down), which + // is a strict improvement over unconditionally clearing them here. Same + // reasoning for cleanup_thread_fn's internal attach-failure path: the + // isolate is still alive, g_initialized stays 1, and g_ref_count was + // already decremented to 0 above without being reset here, so a later + // initialize() correctly ref-counts the surviving isolate instead of + // building a second one (identical semantics to teardown_waiter_thread_fn's + // attach-failure path). + if (torn_down) { g_thread = NULL; g_isolate = NULL; g_initialized = 0; g_ref_count = 0; + } else if (g_isolate != NULL && g_ref_count == 0) { + // cleanup_thread_fn spawn/attach failed: the isolate is still live with + // zero owners. Arm the retry signal so a later op-completion drain or the + // next initialize() (review #6 #5) tears it down instead of stranding it — + // mirrors the twin arm in isolate_ref_release_n_locked. Documented residual: + // if no later op or initialize() ever runs, the isolate lingers to process + // exit (OS reclaims it) -- benign, no ref-count violation. + g_teardown_needed = true; } + uv_mutex_unlock(&g_mutex); + return already_resolved_promise(env); + } + + // Case 5: last release, but streaming/transform ops are still active. + // Defer teardown to a dedicated waiter thread instead of blocking this JS + // thread -- this is the deadlock fix. g_initialized/g_isolate/g_thread stay + // set until the waiter thread finishes, matching today's behavior of + // treating "still tearing down" as "still initialized" for concurrent + // initialize() calls (see Task 3). + g_teardown_state = TEARDOWN_PENDING_WAIT; + g_teardown_cancelled = false; + napi_value promise; + teardown_waiter_t* waiter = teardown_waiter_create(env, &promise); + if (waiter == NULL) { + // The last reference was already dropped (g_ref_count == 0) but we cannot + // build the waiter to drain the isolate. Arm the retry signal so the op + // drain retries teardown -- without it this live isolate would have zero + // owners and nothing to tear it down (round-14 #2). + g_teardown_state = TEARDOWN_NONE; + if (g_isolate != NULL && g_ref_count == 0) g_teardown_needed = true; + uv_mutex_unlock(&g_mutex); + return NULL; // teardown_waiter_create already threw + } + waiter->next = NULL; + g_teardown_waiters = waiter; + + uv_thread_t waiter_tid; + uv_thread_options_t waiter_opts; + waiter_opts.flags = UV_THREAD_HAS_STACK_SIZE; + waiter_opts.stack_size = 2 * 1024 * 1024; + int spawn_rc = uv_thread_create_ex(&waiter_tid, &waiter_opts, teardown_waiter_thread_fn, NULL); + // Deliberately not joined -- this thread finishes on its own and resolves + // every waiter's promise itself; joining here would reintroduce exactly + // the blocking-JS-thread problem this fix removes. + + if (spawn_rc != 0) { + // Best-effort degradation: if the waiter thread never starts, nothing + // will ever clear g_teardown_state, which would otherwise permanently + // wedge every future initialize()/cleanup() call. Roll back to "teardown + // did not start" -- the isolate stays up and the caller's promise still + // resolves, mirroring the fast path's ignore-teardown-return-code posture. + g_teardown_state = TEARDOWN_NONE; + g_teardown_waiters = NULL; + + napi_value undefined; + napi_get_undefined(env, &undefined); + napi_resolve_deferred(env, waiter->deferred, undefined); + + napi_release_threadsafe_function(waiter->tsfn, napi_tsfn_release); + free(waiter); + + // Best-effort degradation: the isolate stays live (g_initialized/g_isolate + // untouched) but no waiter will drain it. Restore g_ref_count to the true + // remaining ownership (Σ init_refs) rather than a hardcoded 1: this env just + // decremented its own init_refs above, and reaching Case 5 means g_ref_count + // hit 0, so the sum is 0 (or whatever surviving envs still own). Hardcoding 1 + // here would strand a reference no env owns -- unreleasable by any cleanup() + // or env-death hook -- and would break the invariant g_ref_count == Σ + // init_refs. A later initialize() will re-acquire on the surviving isolate. + g_ref_count = env_init_refs_total_locked(); + // Arm the retry signal: the isolate stays live with no owners and no waiter, + // so the op-completion drain must retry teardown (round-14 #2). + if (g_isolate != NULL && g_ref_count == 0) g_teardown_needed = true; + + uv_mutex_unlock(&g_mutex); + return promise; } + uv_mutex_unlock(&g_mutex); - return NULL; + return promise; +} + +static napi_value napi_cleanup(napi_env env, napi_callback_info info) { + (void)info; + uv_mutex_lock(&g_mutex); + return release_isolate_ref_locked(env); // unlocks g_mutex, returns the promise } // --- Module init --- static void init_g_mutex(void) { uv_mutex_init(&g_mutex); + uv_cond_init(&g_teardown_cond); } static napi_value Init(napi_env env, napi_value exports) { @@ -1042,14 +2909,23 @@ static napi_value Init(napi_env env, napi_value exports) { napi_create_function(env, "runScript", NAPI_AUTO_LENGTH, dw_napi_run_script, NULL, &fn); napi_set_named_property(env, exports, "runScript", fn); - napi_create_function(env, "runScriptStreaming", NAPI_AUTO_LENGTH, napi_run_script_streaming, NULL, &fn); - napi_set_named_property(env, exports, "runScriptStreaming", fn); + napi_create_function(env, "createEngine", NAPI_AUTO_LENGTH, napi_create_engine, NULL, &fn); + napi_set_named_property(env, exports, "createEngine", fn); + + napi_create_function(env, "createEngineWithResolver", NAPI_AUTO_LENGTH, napi_create_engine_with_resolver, NULL, &fn); + napi_set_named_property(env, exports, "createEngineWithResolver", fn); + + napi_create_function(env, "destroyEngine", NAPI_AUTO_LENGTH, napi_destroy_engine, NULL, &fn); + napi_set_named_property(env, exports, "destroyEngine", fn); + + napi_create_function(env, "runScriptEngine", NAPI_AUTO_LENGTH, napi_run_script_engine, NULL, &fn); + napi_set_named_property(env, exports, "runScriptEngine", fn); - napi_create_function(env, "runScriptTransform", NAPI_AUTO_LENGTH, napi_run_script_transform, NULL, &fn); - napi_set_named_property(env, exports, "runScriptTransform", fn); + napi_create_function(env, "runScriptStreamingEngine", NAPI_AUTO_LENGTH, napi_run_script_streaming_engine, NULL, &fn); + napi_set_named_property(env, exports, "runScriptStreamingEngine", fn); - napi_create_function(env, "runWithResolver", NAPI_AUTO_LENGTH, napi_run_with_resolver, NULL, &fn); - napi_set_named_property(env, exports, "runWithResolver", fn); + napi_create_function(env, "runScriptTransformEngine", NAPI_AUTO_LENGTH, napi_run_script_transform_engine, NULL, &fn); + napi_set_named_property(env, exports, "runScriptTransformEngine", fn); napi_create_function(env, "cleanup", NAPI_AUTO_LENGTH, napi_cleanup, NULL, &fn); napi_set_named_property(env, exports, "cleanup", fn); diff --git a/native-lib/node/src/dataweave.ts b/native-lib/node/src/dataweave.ts index dbaa63a3..db23d99d 100644 --- a/native-lib/node/src/dataweave.ts +++ b/native-lib/node/src/dataweave.ts @@ -23,23 +23,10 @@ export interface DataWeaveOptions { * * MUST be synchronous (cannot return Promise). * - * Note: the native layer installs at most one resolver per process - * lifetime, bound on the first resolver-backed {@link DataWeave.run} call - * (not on {@link DataWeave.initialize}, which only loads/ref-counts the - * native library) and to the thread (main thread or `worker_threads` - * Worker) that made that first call. If you construct multiple `DataWeave` - * instances with different `resolveModule` callbacks in the same process, - * whichever instance's `run()` executes first wins; later instances - * silently reuse that resolver instead of their own. If a later instance's - * `run()` executes on a *different* thread, its resolver is not invoked at - * all and custom module paths resolve as "not found" (see - * docs/external-modules.md#multiple-resolvers-in-one-process). - * - * Concurrency warning: calling a resolver-backed `run()` concurrently from - * more than one Worker is not just unsupported — it is memory-unsafe (see - * docs/external-modules.md, Worker threads section). Restrict - * resolver-backed execution to a single thread, or serialize calls across - * Workers. + * Each DataWeave instance owns an independent native engine, so multiple + * instances with different resolvers coexist in one process with no + * cross-talk. Streaming/transform still resolve only built-in modules for a + * resolver-backed engine (custom modules fail closed); see external-modules.md. * * Security: the resolver runs with full process permissions and no * sandboxing (same trust model as the CLI resolving `.dwl` files from @@ -61,7 +48,9 @@ export interface DataWeaveOptions { export class DataWeave { private readonly libPath: string; private readonly resolveModule?: ModuleResolver; - private initialized = false; + private state: "uninitialized" | "ready" | "cleaning-up" = "uninitialized"; + private engineHandle: number | null = null; + private cleanupPromise: Promise | null = null; /** * @param options - Configuration options or a legacy libPath string. @@ -83,25 +72,138 @@ export class DataWeave { * initialized. * * @throws DataWeaveError if the native library fails to load or initialize. + * @throws DataWeaveError if called while a `cleanup()` is still in progress + * — await the cleanup first. */ initialize(): void { - if (this.initialized) return; + if (this.state === "ready") return; + if (this.state === "cleaning-up") { + throw new DataWeaveError( + "Cannot initialize while cleanup is in progress; await cleanup() first." + ); + } + let libRefAcquired = false; try { ffi.initialize(this.libPath); + libRefAcquired = true; + this.engineHandle = this.resolveModule + ? ffi.createEngineWithResolver(this.resolveModule) + : ffi.createEngine(); } catch (e: unknown) { + // If ffi.initialize() already succeeded but engine creation then threw, we + // already hold an increment of the native library's ref-counted handle and + // must release it (ffi.cleanup()), or it leaks for the process lifetime. + // ffi.cleanup() is async, so model the rollback as PENDING state instead of + // firing-and-forgetting it (review #7 #3): (1) an un-awaited rejection must + // not become an unhandledRejection, and (2) a concurrent initialize()/run() + // must not race a fresh graal_create_isolate against the in-flight release. + // Reuse the same cleanupPromise/"cleaning-up" machinery cleanup() uses: + // hold state "cleaning-up" until the release settles (so initialize()'s own + // "cleaning-up" guard rejects a concurrent retry deterministically, and a + // concurrent cleanup() coalesces onto this same promise), then return to + // "uninitialized". The synchronous throw to THIS caller is preserved. + this.engineHandle = null; + if (libRefAcquired) { + this.state = "cleaning-up"; + // ffi.cleanup() can fail synchronously (throw) as well as asynchronously + // (reject a returned promise). Calling it inside a try/catch -- rather + // than eagerly as the argument to Promise.resolve(ffi.cleanup()) -- lets + // a synchronous throw be caught and normalized into a rejected promise + // BEFORE cleanupPromise is assigned, so it still flows through the same + // .finally() state reset instead of escaping here and stranding this + // instance in "cleaning-up" forever (review #8 #2). Existing callers + // still observe ffi.cleanup() invoked synchronously, in the same tick as + // this catch block, exactly as before this fix. + let releaseResult: Promise | void; + try { + releaseResult = ffi.cleanup(); + } catch (cleanupError) { + releaseResult = Promise.reject(cleanupError); + } + this.cleanupPromise = Promise.resolve(releaseResult).finally(() => { + this.state = "uninitialized"; + this.cleanupPromise = null; + }); + // Never let an un-awaited rollback surface as an unhandledRejection. A + // caller that awaits cleanup() (which coalesces onto cleanupPromise) + // still observes the rejection; this handler only covers the un-awaited + // path. + this.cleanupPromise.catch(() => {}); + } throw new DataWeaveError(`Failed to initialize: ${e instanceof Error ? e.message : e}`); } - this.initialized = true; + this.state = "ready"; } /** * Releases the native runtime. Idempotent — a no-op if not initialized. After * cleanup the instance can be re-initialized via {@link DataWeave.initialize}. + * + * Resolution depends on whether this call releases the FINAL shared native + * reference in the process. When it does, it resolves once the underlying + * native isolate has actually finished tearing down; if a streaming/transform + * operation on this or any other instance is still in flight at that point, + * native teardown waits for it to drain before resolving — awaiting this + * rather than firing-and-forgetting avoids racing a subsequent + * {@link initialize} against an isolate that is still tearing down. When other + * initialized instances remain, it resolves as soon as this instance's engine + * is released, leaving the shared isolate live for them. */ - cleanup(): void { - if (!this.initialized) return; - ffi.cleanup(); - this.initialized = false; + async cleanup(): Promise { + // Coalesce first: doCleanup() flips `state` to "cleaning-up" synchronously + // as its first statement, so by the time a second overlapping call runs, + // `state` has already left "ready". If the not-ready guard below ran + // first, that second caller would resolve immediately instead of + // awaiting the first caller's in-flight native teardown -- contradicting + // this method's contract of resolving only once the isolate has actually + // finished tearing down (round-6 review, task-1 fix round 1). Checking + // `cleanupPromise` first ensures every concurrent caller that overlaps + // with an in-flight doCleanup() awaits that SAME promise, so the native + // teardown (ffi.destroyEngine/ffi.cleanup) still happens exactly once. + if (this.cleanupPromise) return this.cleanupPromise; + // Not coalescing with an in-flight cleanup: nothing to do unless we're + // "ready" (covers both never-initialized and already-settled cleanup). + if (this.state !== "ready") return; + this.cleanupPromise = this.doCleanup(); + try { + await this.cleanupPromise; + } finally { + // Clear on both fulfilment and rejection so a later cleanup() (after a + // re-initialize, or a retry of a rejected cleanup) can run again. + this.cleanupPromise = null; + } + } + + private async doCleanup(): Promise { + // Transition BEFORE releasing the engine so run()/initialize() called + // during the async teardown window are rejected deterministically rather + // than seeing a stale "ready" state with a null engineHandle (round-6 #1/#3). + this.state = "cleaning-up"; + let destroyError: unknown; + try { + if (this.engineHandle !== null) { + try { + ffi.destroyEngine(this.engineHandle); + } catch (e) { + // Round-14 (#6): a throwing destroyEngine() (e.g. wrong-thread + // destruction) must NOT skip ffi.cleanup() -- that would strand this + // env's native init reference and block isolate teardown. Capture the + // primary error, clear the handle so a retry does not double-destroy, + // and fall through to release the reference below. + destroyError = e; + } finally { + this.engineHandle = null; + } + } + await ffi.cleanup(); + } finally { + this.state = "uninitialized"; + } + // Surface the primary destruction error after the reference was released. If + // ffi.cleanup() itself rejected, its error already propagated from the await + // (the more actionable reference-release failure wins; the destroy error is + // then suppressed). + if (destroyError !== undefined) throw destroyError; } /** @@ -116,17 +218,10 @@ export class DataWeave { * @throws DataWeaveScriptError if the script fails and `opts.raiseOnError` is set. */ run(script: string, inputs?: Inputs, opts?: { raiseOnError?: boolean }): ExecutionResult { - this.ensureInitialized(); + this.ensureReady(); const inputsJson = buildInputsJson(inputs ?? {}); - let raw: string; - if (this.resolveModule) { - // Use resolver-aware entrypoint - raw = ffi.runWithResolver(script, inputsJson, "application/json", this.resolveModule); - } else { - // Use standard entrypoint (backward compatible) - raw = ffi.runScript(script, inputsJson); - } + const raw = ffi.runScriptEngine(this.engineHandle!, script, inputsJson); const result = parseNativeResponse(raw); @@ -148,9 +243,11 @@ export class DataWeave { * @throws DataWeaveError if the runtime is not initialized. */ async *runStreaming(script: string, inputs?: Inputs): AsyncGenerator { - this.ensureInitialized(); + this.ensureReady(); const inputsJson = buildInputsJson(inputs ?? {}); - return yield* streamFromNative((chunkCb) => ffi.runScriptStreaming(script, inputsJson, chunkCb)); + return yield* streamFromNative((chunkCb) => + ffi.runScriptStreamingEngine(this.engineHandle!, script, inputsJson, chunkCb) + ); } /** @@ -174,7 +271,7 @@ export class DataWeave { input: AsyncIterable | Iterable, opts?: TransformOptions ): AsyncGenerator { - this.ensureInitialized(); + this.ensureReady(); const inputName = opts?.inputName ?? "payload"; const inputMimeType = opts?.mimeType ?? "application/json"; @@ -184,30 +281,132 @@ export class DataWeave { const readCb = await createChunkReader(input); + // The instance may have been cleaned up while an async input pre-buffered + // (createChunkReader can await arbitrarily long). Re-check readiness so a + // caller that raced cleanup() gets a synchronous DataWeaveError rather than + // a resolved "Unknown engine handle" envelope. The C admission pin is the + // authoritative memory-safety guard (round 11 #2/#3); this only improves the + // failure ergonomics for a misused instance. (round 12 #4) + this.ensureReady(); + return yield* streamFromNative((writeCb) => - ffi.runScriptTransform(script, inputsJson, inputName, inputMimeType, inputCharset, readCb, writeCb) + ffi.runScriptTransformEngine( + this.engineHandle!, + script, + inputsJson, + inputName, + inputMimeType, + inputCharset, + readCb, + writeCb + ) ); } - private ensureInitialized(): void { - if (!this.initialized) { - throw new DataWeaveError("DataWeave runtime not initialized. Call initialize() first."); + private ensureReady(): void { + if (this.state === "ready") return; + if (this.state === "cleaning-up") { + throw new DataWeaveError( + "DataWeave runtime is cleaning up; await cleanup() before running again." + ); } + throw new DataWeaveError("DataWeave runtime not initialized. Call initialize() first."); } } // Module-level convenience API with lazy singleton let globalInstance: DataWeave | null = null; +// Guards against beforeExit and exit both driving cleanup for the same +// shutdown. Belt-and-suspenders on top of cleanup()'s own idempotency. +let cleanupStarted = false; +// Coalesces overlapping module-level cleanup() calls, mirroring the +// instance-level DataWeave.cleanupPromise. Without it, the second of two +// overlapping module cleanup() calls sees globalInstance already nulled and +// resolves immediately -- before the first call's native teardown finishes, +// violating cleanup()'s "resolves once native teardown has finished" contract +// for the last reference. (round 12 #5) +let cleanupPromise: Promise | null = null; +// The instance that `cleanupPromise` is currently draining. Needed because +// coalescing must NOT be keyed on the module-global promise alone: if a +// caller revives the singleton (via run()/getGlobalInstance()) while a prior +// drain is still in flight, a subsequent cleanup() must clean the freshly +// revived instance rather than returning the stale promise as if it had +// covered it too -- otherwise the revived instance's native ref is silently +// leaked (final-review round 12 #1, fixing round 12 Task 6's regression). +let cleaningInstance: DataWeave | null = null; +// Process exit hooks are registered exactly once for the lifetime of the +// module, NOT per singleton. Re-creating the singleton after cleanup() must +// not attach a second pair of listeners (that accumulates until Node emits +// MaxListenersExceededWarning). The listeners tolerate a null globalInstance: +// cleanup() no-ops when there is nothing to release, and cleanupStarted +// coalesces beforeExit/exit for a given shutdown. Unlike cleanupStarted, this +// guard is never reset — that is the whole point. +let exitHooksRegistered = false; + +/** + * Registers the process-wide exit-cleanup hooks exactly once for this + * module. Subsequent calls (e.g. from a revived singleton after cleanup()) + * are no-ops: the hooks registered on first use are reused for the rest of + * the process's lifetime, which is safe because they tolerate a null + * `globalInstance` and `cleanupStarted` coalesces beforeExit/exit for a + * given shutdown. + * + * Two hooks are registered, covering complementary cases: + * - `beforeExit` fires when the event loop is about to drain naturally and + * CAN run async work (Node keeps the loop alive until it settles), so it + * drains any in-flight streaming/transform operation gracefully. This is + * the common case. + * - `exit` runs strictly synchronously and is only a best-effort fallback for + * the paths that skip `beforeExit` — `process.exit()`, an uncaught + * exception, and normal process termination. Because it is synchronous it + * can only run the fast cleanup path, so an in-flight async operation may be + * abandoned. Node does NOT emit `exit` (nor `beforeExit`) for termination + * signals such as SIGTERM/SIGINT/SIGKILL, nor for every fatal failure mode, + * so this is not a guarantee: callers that require graceful shutdown must + * register and await their own handlers for the catchable signals (e.g. + * `process.on("SIGTERM", async () => { await cleanup(); process.exit(0); })`); + * SIGKILL cannot be caught, so no in-process cleanup can run for it. + * The `cleanupStarted` guard ensures only one of the two hooks actually + * runs cleanup for a given shutdown. + */ +function registerExitHooksOnce(): void { + if (exitHooksRegistered) return; + exitHooksRegistered = true; + process.on("beforeExit", async () => { + if (cleanupStarted) return; + cleanupStarted = true; + await cleanup(); // beforeExit can await: drains in-flight ops + }); + process.on("exit", () => { + if (cleanupStarted) return; // beforeExit already handled it + cleanup(); // fallback: best-effort sync fast path + }); +} /** * Returns the process-wide {@link DataWeave} singleton, creating and - * initializing it (and registering a process-exit cleanup hook) on first use. + * initializing it on first use (or after a prior {@link cleanup}). + * + * The exit-cleanup hooks are registered exactly once for the process via + * {@link registerExitHooksOnce}, not once per singleton: a singleton revived + * after cleanup() reuses the same pair of listeners rather than adding new + * ones, which would otherwise accumulate a pair per init/cleanup cycle until + * Node emits `MaxListenersExceededWarning`. Reuse is safe because the + * listeners tolerate a null `globalInstance` and `cleanupStarted` coalesces + * beforeExit/exit for a given shutdown. */ function getGlobalInstance(): DataWeave { if (!globalInstance) { - globalInstance = new DataWeave(); - globalInstance.initialize(); - process.on("exit", () => cleanup()); + // Initialize a LOCAL candidate first; publish the singleton only after + // initialize() succeeds. A failed first init (bad DATAWEAVE_NATIVE_LIB + // path / transient native failure) must NOT leave a poisoned, uninitialized + // singleton that makes every later run*() fail "not initialized" even after + // the fault is fixed (review #6 #1). On throw, globalInstance stays null and + // the next call retries cleanly with a fresh instance. + const candidate = new DataWeave(); + candidate.initialize(); + globalInstance = candidate; + registerExitHooksOnce(); } return globalInstance; } @@ -247,9 +446,50 @@ export function runTransform( * Releases the shared {@link DataWeave} singleton, if one was created. A fresh * singleton is created lazily on the next convenience-API call. */ -export function cleanup(): void { - if (globalInstance) { - globalInstance.cleanup(); - globalInstance = null; +export async function cleanup(): Promise { + // Coalesce overlapping calls onto one drain (round 12 #5) -- but ONLY when + // nothing new has been revived since that drain started. If `globalInstance` + // is still the same instance the in-flight promise is draining, or is null + // (nobody has revived since), it's safe to piggyback on the existing + // promise. If a DIFFERENT instance is now the singleton (a caller called + // run() and revived it while the old drain was still in flight), that new + // instance has never been handed to a cleanup() call -- returning the old + // promise here would resolve as if it had been cleaned when it hasn't, + // leaking its native ref for the rest of the process (final-review round 12 + // #1). Fall through and drain the current instance instead. + if (cleanupPromise && (globalInstance === null || globalInstance === cleaningInstance)) { + return cleanupPromise; } -} \ No newline at end of file + if (!globalInstance) return; + const instance = globalInstance; + globalInstance = null; + // Chosen semantics for overlapping different-instance drains: coalescing + // tracks only the MOST RECENT drain. An older drain that is still in flight + // when a newer one starts is not stomped -- it keeps running against its own + // promise, which whoever started it already holds and will await -- but it + // stops being the thing later cleanup() calls coalesce onto. Two distinct + // instances tearing down concurrently is fine: each owns its own engine + // handle and native ref, exactly like two DataWeave instances calling + // .cleanup() independently. This keeps the invariant that matters: no + // cleanup() call ever returns as if it drained an instance it didn't. + cleaningInstance = instance; + cleanupPromise = instance.cleanup(); + try { + await cleanupPromise; + } finally { + // Only clear the shared coalescing state if it's still ours to clear -- + // i.e. nobody has started a newer drain (for a newer revived instance) + // that has since taken over `cleanupPromise`/`cleaningInstance`. Guards + // against this drain's finally clobbering a later drain's in-flight state. + if (cleaningInstance === instance) { + cleanupPromise = null; + cleaningInstance = null; + } + // Reset the exit-hook guard only after THIS drain has fully completed, so + // a revived singleton gets its own live hooks for the next real exit. + // Must stay last: resetting earlier could let a concurrent `exit` firing + // on this same shutdown re-enter cleanup while the async drain above is + // in flight. + cleanupStarted = false; + } +} diff --git a/native-lib/node/src/ffi.ts b/native-lib/node/src/ffi.ts index 924de436..24711ea0 100644 --- a/native-lib/node/src/ffi.ts +++ b/native-lib/node/src/ffi.ts @@ -4,8 +4,18 @@ import type { ModuleResolver } from "./resolver"; interface NativeAddon { initialize(libPath: string): void; runScript(script: string, inputsJson: string): string; - runScriptStreaming(script: string, inputsJson: string, chunkCb: (chunk: Buffer) => void): Promise; - runScriptTransform( + createEngine(): number; + createEngineWithResolver(resolver: ModuleResolver): number; + destroyEngine(handle: number): void; + runScriptEngine(handle: number, script: string, inputsJson: string): string; + runScriptStreamingEngine( + handle: number, + script: string, + inputsJson: string, + chunkCb: (chunk: Buffer) => void + ): Promise; + runScriptTransformEngine( + handle: number, script: string, inputsJson: string, inputName: string, @@ -14,14 +24,7 @@ interface NativeAddon { readCb: (bufSize: number) => Buffer | null, writeCb: (chunk: Buffer) => void ): Promise; - runWithResolver( - script: string, - inputsJson: string, - mimeType: string, - resolverCallback: ModuleResolver, - isolate: null - ): string; - cleanup(): void; + cleanup(): Promise; } let addon: NativeAddon | null = null; @@ -42,15 +45,33 @@ export function runScript(script: string, inputsJson: string): string { return getAddon().runScript(script, inputsJson); } -export function runScriptStreaming( +export function createEngine(): number { + return getAddon().createEngine(); +} + +export function createEngineWithResolver(resolver: ModuleResolver): number { + return getAddon().createEngineWithResolver(resolver); +} + +export function destroyEngine(handle: number): void { + getAddon().destroyEngine(handle); +} + +export function runScriptEngine(handle: number, script: string, inputsJson: string): string { + return getAddon().runScriptEngine(handle, script, inputsJson); +} + +export function runScriptStreamingEngine( + handle: number, script: string, inputsJson: string, chunkCb: (chunk: Buffer) => void ): Promise { - return getAddon().runScriptStreaming(script, inputsJson, chunkCb); + return getAddon().runScriptStreamingEngine(handle, script, inputsJson, chunkCb); } -export function runScriptTransform( +export function runScriptTransformEngine( + handle: number, script: string, inputsJson: string, inputName: string, @@ -59,18 +80,18 @@ export function runScriptTransform( readCb: (bufSize: number) => Buffer | null, writeCb: (chunk: Buffer) => void ): Promise { - return getAddon().runScriptTransform(script, inputsJson, inputName, inputMimeType, inputCharset, readCb, writeCb); -} - -export function runWithResolver( - script: string, - inputsJson: string, - mimeType: string, - resolverCallback: ModuleResolver -): string { - return getAddon().runWithResolver(script, inputsJson, mimeType, resolverCallback, null); + return getAddon().runScriptTransformEngine( + handle, + script, + inputsJson, + inputName, + inputMimeType, + inputCharset, + readCb, + writeCb + ); } -export function cleanup(): void { - getAddon().cleanup(); +export function cleanup(): Promise { + return getAddon().cleanup(); } diff --git a/native-lib/node/src/stream.ts b/native-lib/node/src/stream.ts index 855807f6..032d5a68 100644 --- a/native-lib/node/src/stream.ts +++ b/native-lib/node/src/stream.ts @@ -36,15 +36,27 @@ export async function* streamFromNative( } }; - const metaPromise = start(chunkCb).then((raw) => { - metaRaw = raw; - done = true; - // Wake all waiting consumers + let startError: unknown; + let startRejected = false; + const wakeAll = () => { while (pendingResolves.length > 0) { const resolve = pendingResolves.shift(); if (resolve) resolve(); } - }); + }; + + // Handle BOTH settlement branches. Without the rejection handler, a rejected + // start() leaves `done` false forever: a consumer parked in next() below is + // never woken and the generator hangs, and the rejection is unhandled + // (review #6 #2). On rejection we record the error, flip startRejected, mark + // completion, and wake every waiter; the error is re-thrown (by settlement + // state, not by value -- see below) after draining any chunks that arrived + // before the rejection. Because we handle rejection here, metaPromise itself + // always fulfills -- `await metaPromise` below never throws. + const metaPromise = start(chunkCb).then( + (raw) => { metaRaw = raw; done = true; wakeAll(); }, + (err) => { startError = err; startRejected = true; done = true; wakeAll(); } + ); while (true) { if (chunks.length > 0) { @@ -55,11 +67,15 @@ export async function* streamFromNative( await new Promise((resolve) => { pendingResolves.push(resolve); }); } - // Drain remaining chunks + // Drain remaining chunks buffered before completion/rejection. while (chunks.length > 0) { yield chunks.shift()!; } await metaPromise; + // Track rejection by settlement STATE, not by the rejected value: Promise.reject(undefined) + // is valid JS, so a value sentinel (startError !== undefined) would swallow it as an empty + // result. startRejected is only ever set in the rejection handler above (review #7 #6). + if (startRejected) throw startError; return parseStreamingResult(metaRaw ?? ""); } \ No newline at end of file diff --git a/native-lib/node/tests/integration/admission-during-teardown.test.ts b/native-lib/node/tests/integration/admission-during-teardown.test.ts new file mode 100644 index 00000000..a87b7f18 --- /dev/null +++ b/native-lib/node/tests/integration/admission-during-teardown.test.ts @@ -0,0 +1,136 @@ +import { describe, it, expect } from "vitest"; +import * as ffi from "../../src/ffi"; +import { findLibrary, buildInputsJson } from "../../src/utils"; + +// Round-6 finding #2: napi_run_script_streaming_engine/napi_run_script_transform_engine +// used to read g_initialized outside g_mutex, then reserve g_active_ops in a +// LATER, separate critical section right before spawning the worker thread -- +// with no reference to g_teardown_state at all. The fix folds the lifecycle +// check (including g_teardown_state) and the g_active_ops reservation into one +// atomic critical section, before any work/tsfn/promise/bridge is allocated, +// and rejects admission once a teardown is queued/underway +// (g_teardown_state != TEARDOWN_NONE), not just when the isolate is fully gone. +// +// Why this test drives the addon through the raw `ffi` module instead of the +// module-level `run`/`runStreaming`/`runTransform`/`cleanup` singleton (as the +// original brief sketch does): the module-level `cleanup()` nulls the +// singleton, so a later module-level `runStreaming()`/`runTransform()` call +// re-creates a fresh `DataWeave` instance and calls `initialize()` again. +// `napi_initialize`'s TEARDOWN_PENDING_WAIT branch (round-5's deadlock fix) +// treats that as a legitimate ADOPTION of the still-live isolate: it sets +// g_teardown_cancelled = true and cancels the pending teardown *before* the +// second op's admission check ever runs -- so by the time streaming/transform +// admission is checked, g_teardown_state is already back to TEARDOWN_NONE +// (verified empirically while developing this test: the brief's literal shape +// resolves the second op cleanly on both pre-fix and post-fix code, so it +// cannot distinguish them -- it never reaches the vulnerable window because +// the intervening initialize() call cancels the teardown as a side effect). +// +// To actually observe admission-during-pending-teardown, the second op must +// run against the SAME still-live handle/isolate WITHOUT any intervening +// ffi.initialize() call. Calling `ffi.cleanup()` directly (skipping +// `destroyEngine`) triggers exactly napi_cleanup's Case 5 (last ref release +// with an active op) and sets g_teardown_state = TEARDOWN_PENDING_WAIT +// synchronously, under g_mutex, before napi_cleanup returns its Promise to +// JS -- with no adoption path involved, since nothing calls initialize() +// afterward. +// +// Determinism: `ffi.cleanup()`'s synchronous prefix (native napi_cleanup body) +// runs entirely synchronously up to the point where it returns a Promise; the +// TEARDOWN_PENDING_WAIT transition happens on that same synchronous call, not +// after an await. The immediately-following `ffi.runScriptStreamingEngine` +// call re-enters native code synchronously (it's a plain N-API call), on the +// very same JS callstack, so it deterministically observes +// g_teardown_state == TEARDOWN_PENDING_WAIT with no timing assumptions -- +// mirroring the round-5 teardown-deadlock test's use of a synchronous native +// read-callback to force deterministic ordering instead of timers. +// +// Real addon, no mocking. +describe("admission rejected while teardown pending (round 6 #2)", () => { + it("a streaming op started on the same handle during pending teardown is rejected, not admitted", async () => { + ffi.initialize(findLibrary()); + const handle = ffi.createEngine(); + + let cleanupPromise: Promise | undefined; + let admitErr: unknown; + let admitted = false; + let secondOpSettled: Promise = Promise.resolve(); + + let firstRead = true; + const readCb = (_bufSize: number): Buffer | null => { + if (firstRead) { + firstRead = false; + + // Trigger Case 5 of napi_cleanup: last release of the shared library + // ref-count while this transform's worker is attached and + // g_active_ops > 0. Synchronously sets g_teardown_state = + // TEARDOWN_PENDING_WAIT before returning. Not awaited -- the point is + // to observe the state it leaves behind, not its eventual settlement. + cleanupPromise = ffi.cleanup(); + + // Attempt a second admission on the SAME still-live handle/isolate + // while teardown is pending. Fixed code rejects admission with a + // synchronous napi_throw_error (the atomic admission check sees + // g_teardown_state != TEARDOWN_NONE, before any promise is even + // created). Pre-fix code admits it: the unlocked g_initialized check + // passes (the isolate genuinely hasn't been torn down yet -- + // TEARDOWN_PENDING_WAIT hasn't reached physical teardown) and + // g_active_ops is reserved without ever consulting g_teardown_state, + // so the call returns a promise that goes on to resolve successfully. + // + // On rejection, napi_throw_error fires synchronously from this very + // call (admission fails before any promise is created), so it must + // be caught here rather than only via a rejected-promise `.then` -- + // mirroring the round-5 teardown-deadlock test's care not to let a + // thrown exception escape a native read-callback body (it would be + // reinterpreted as a read error, masking the real outcome). + try { + secondOpSettled = ffi + .runScriptStreamingEngine( + handle, + "%dw 2.0\noutput application/json\n---\n[1,2,3]", + buildInputsJson({}), + () => {} + ) + .then( + () => { admitted = true; }, + (e) => { admitErr = e; } + ); + } catch (e) { + admitErr = e; + } + + return Buffer.from("[1,2,3]"); + } + return null; // EOF after the first chunk + }; + + const chunks: Buffer[] = []; + const writeCb = (chunk: Buffer) => { chunks.push(chunk); }; + + const resultRaw = await ffi.runScriptTransformEngine( + handle, + "output application/json\n---\npayload", + "{}", + "payload", + "application/json", + null, + readCb, + writeCb + ); + const result = JSON.parse(resultRaw); + expect(result.success).toBe(true); + + // Let the second op settle (whichever branch it took) before asserting, + // and drain the pending teardown so the shared native isolate is left in + // a clean, consistent state for sibling test files in this process. + await secondOpSettled; + await cleanupPromise; + + // The second op admitted while teardown was pending must have been + // rejected, not silently admitted against an isolate a concurrent + // teardown could tear down out from under it. + expect(admitErr).toBeTruthy(); + expect(admitted).toBe(false); + }, 20000); +}); diff --git a/native-lib/node/tests/integration/dataweave-resolver.test.ts b/native-lib/node/tests/integration/dataweave-resolver.test.ts index 6578bb6b..a3963d24 100644 --- a/native-lib/node/tests/integration/dataweave-resolver.test.ts +++ b/native-lib/node/tests/integration/dataweave-resolver.test.ts @@ -1,15 +1,16 @@ import { describe, it, expect, afterAll } from "vitest"; import { DataWeave, cleanup } from '../../src/dataweave'; +import { DataWeaveError } from '../../src/errors'; import { modulesFromMap } from '../../src/resolver'; // Every test below constructs its own explicit DataWeave instance (rather // than the module-level singleton) so each can configure its own resolver. // `cleanup()` above only releases the *singleton* (`globalInstance`), which // nothing in this file ever creates -- so without this tracking, every -// explicit instance's native library reference (and the shared addon-level -// ref-count, see addon.c's g_ref_count) would leak for the lifetime of the -// test process. Track every instance created in this file and release them -// all in afterAll. +// explicit instance's native library reference (and its own engine handle, +// see addon.c's create_engine/destroy_engine) would leak for the lifetime of +// the test process. Track every instance created in this file and release +// them all in afterAll. const instances: DataWeave[] = []; function trackedDataWeave(...args: ConstructorParameters): DataWeave { const dw = new DataWeave(...args); @@ -17,31 +18,19 @@ function trackedDataWeave(...args: ConstructorParameters): Dat return dw; } -afterAll(() => { +afterAll(async () => { for (const dw of instances) { - dw.cleanup(); + await dw.cleanup(); } - cleanup(); + await cleanup(); }); -// ScriptRuntime installs at most one resolver for the whole process lifetime -// (see ScriptRuntime.setResolver()): whichever DataWeave instance's resolver -// gets installed first "wins", and every later DataWeave instance in this -// file — regardless of its own resolveModule map — silently reuses it. Since -// vitest runs the `it` blocks in this file sequentially in the same process, -// that's always this first module-map, so it must contain every module path -// any test below needs to resolve for the first time (including the -// cross-thread regression test's two never-before-resolved paths). -const SHARED_RESOLVER_MODULES: Record = { - 'org/test/lib.dwl': '%dw 2.0\nfun greet(n: String) = "Hello " ++ n', - 'org/test/resolverGuardInstall.dwl': '%dw 2.0\nfun greet(n: String) = "Hello " ++ n', - 'org/test/resolverGuardStreamed.dwl': '%dw 2.0\nfun greet(n: String) = "Hello " ++ n', -}; - describe('DataWeave with resolver', () => { it('resolves imported module from map', () => { const dw = trackedDataWeave({ - resolveModule: modulesFromMap(SHARED_RESOLVER_MODULES), + resolveModule: modulesFromMap({ + 'org/test/lib.dwl': '%dw 2.0\nfun greet(n: String) = "Hello " ++ n', + }), }); dw.initialize(); @@ -106,46 +95,27 @@ describe('DataWeave with resolver', () => { expect(JSON.parse(result.getString()!)).toBe("Hello"); }); - // Regression test for the cross-thread resolver hazard: ScriptRuntime's engine - // is a process-wide singleton, so once any .run() call installs a resolver on - // it, that same composite resolver is used by ALL later execution paths -- - // including runStreaming()/runTransform(), whose native call executes on a - // background uv_thread (see addon.c's streaming_thread_fn), not the JS thread - // that registered the resolver. Before the thread-identity guard in addon.c's - // resolve_module_callback, a streamed script importing a non-built-in module - // would trigger a napi call from that background thread -- undefined behavior, - // typically a crash of the whole process. After the guard, the callback fails - // closed (reports "not found" instead of calling back into JS), so the script - // fails cleanly with a compile error and the process survives. - it('runStreaming fails cleanly (does not crash) for a custom module on the shared singleton engine', async () => { - // Once a module name has been resolved anywhere in the process, the - // DataWeave compiler caches it and won't call back into the resolver for - // that same name again — so the install script and the streaming script - // below import two module paths that no earlier test in this file has - // imported yet (both pre-registered in SHARED_RESOLVER_MODULES above, - // since only the first-installed resolver's map is ever consulted). + // Regression test for the cross-thread resolver hazard: each DataWeave + // instance now owns its own native engine (see engine_bridge_t in addon.c), + // but a resolver-backed engine's runStreaming()/runTransform() still + // executes the native call on a background uv_thread (see addon.c's + // streaming_thread_fn/transform_thread_fn), not the JS thread that created + // the engine and its resolver bridge. resolve_module_callback detects that + // thread-identity mismatch and fails closed (reports "not found" instead of + // calling back into JS) rather than making an unsafe cross-thread napi + // call, so the script fails cleanly with a compile error and the process + // survives. + it('runStreaming fails cleanly for a custom module on its own resolver-backed engine', async () => { const dw = trackedDataWeave({ - resolveModule: modulesFromMap(SHARED_RESOLVER_MODULES), + resolveModule: modulesFromMap({ + 'org/test/resolverGuardStreamed.dwl': '%dw 2.0\nfun greet(n: String) = "Hello " ++ n', + }), }); dw.initialize(); - // Install (or confirm already-installed) resolver on the shared singleton - // engine via a synchronous run() call. Per ScriptRuntime.setResolver(), only - // the first resolver registered for the process is ever used, so this is - // safe to call even if an earlier test in this file already installed one. - const installResult = dw.run(` - %dw 2.0 - import org::test::resolverGuardInstall - output application/json - --- - resolverGuardInstall::greet("Installer") - `); - expect(installResult.success).toBe(true); - - // Now stream a script that imports a DIFFERENT non-built-in module, never - // resolved before in this process. The singleton engine's composite - // resolver (ClassLoader + Callback) will miss in the ClassLoader half (not - // a built-in) and fall through to the Callback half, invoking + // Stream a script that imports a non-built-in module. This engine's + // composite resolver (ClassLoader + Callback) misses in the ClassLoader + // half (not a built-in) and falls through to the Callback half, invoking // resolve_module_callback from runStreaming's background thread. const chunks: Buffer[] = []; const gen = dw.runStreaming(` @@ -167,4 +137,343 @@ describe('DataWeave with resolver', () => { expect(metadata.error).toBeTruthy(); expect(chunks.length).toBe(0); }); + + // resolve_module_callback in addon.c catches a JS exception thrown by the + // user-supplied resolver (napi_call_function returning napi_pending_exception), + // clears it via napi_get_and_clear_last_exception, logs a content-free + // diagnostic (see the DATAWEAVE_RESOLVER_DEBUG gating), and reports "not + // found" back to the DataWeave runtime -- rather than letting the pending + // exception leak into a later napi call or crash the process. This is a + // synchronous run() on the JS thread that created the bridge (the "owner" + // thread check in resolve_module_callback passes), so the callback is + // actually invoked, unlike the streaming/transform cross-thread case above. + it('throwing resolver makes run() fail cleanly instead of crashing the process', () => { + const dw = trackedDataWeave({ + resolveModule: () => { + throw new Error('resolver blew up'); + }, + }); + dw.initialize(); + + const result = dw.run(` + %dw 2.0 + import org::test::throwingResolverLib + output application/json + --- + {} + `); + + // The test itself completing (no uncaught exception / segfault) is the + // crash-check; we don't assert on the internal error message wording. + expect(result.success).toBe(false); + }); + + // Regression test for a resolver-backed engine's initialize -> cleanup -> + // initialize cycle. Unlike the resolver-less reinit test in + // edge-cases.test.ts, this exercises createEngineWithResolver's bridge + // (engine_bridge_t) lifecycle: cleanup() destroys the bridge and its engine + // handle, and the following initialize() must build a brand new bridge + // (new napi_ref on the resolver, new owner-thread record) that resolves + // custom modules again, not a stale or dangling one. + it('resolver-backed instance resolves a custom module again after initialize -> cleanup -> initialize', async () => { + const dw = trackedDataWeave({ + resolveModule: modulesFromMap({ + 'org/test/reinitLib.dwl': '%dw 2.0\nfun greet(n: String) = "Hello " ++ n', + }), + }); + dw.initialize(); + await dw.cleanup(); + dw.initialize(); + + const result = dw.run(` + %dw 2.0 + import org::test::reinitLib + output application/json + --- + reinitLib::greet("Reinit") + `); + + expect(result.success).toBe(true); + expect(JSON.parse(result.getString()!)).toBe("Hello Reinit"); + }); + + // Regression test for the F1 use-after-free fix: a resolver-backed engine's + // engine_bridge_t used to be freed by destroy_engine (called from cleanup()) + // even while a background uv_thread (streaming_thread_fn) was still + // mid-flight and could call resolve_module_callback with that bridge as + // ctx -- a use-after-free. The fix adds in-flight accounting under g_mutex: + // destroy_engine now defers the actual free until the background operation + // decrements in_flight back to zero in its completion sentinel. + // + // To race cleanup() against the in-flight operation deterministically, we + // start the generator's *first* `.next()` call but do not await it before + // calling cleanup(). Calling an async generator's .next() runs its body + // synchronously up to the first suspension point (an `await`); by that + // point runStreaming's synchronous prefix -- including the native + // runScriptStreamingEngine call that hands the operation to a libuv + // worker-pool thread -- has already executed. cleanup() is then called + // from the JS thread while that native call may already be running + // concurrently on the worker thread, which is exactly the race the F1 fix + // guards against. Before that fix this was a real crash/UAF risk; after it, + // this must complete cleanly (settle, not crash, not hang) regardless of + // which side of the race wins. + it('cleanup() racing an in-flight resolver-backed runStreaming() does not crash (F1 regression)', async () => { + const dw = trackedDataWeave({ + resolveModule: modulesFromMap({ + 'org/test/cleanupDuringStream.dwl': '%dw 2.0\nfun greet(n: String) = "Hello " ++ n', + }), + }); + dw.initialize(); + + const gen = dw.runStreaming(` + %dw 2.0 + import org::test::cleanupDuringStream + output application/json + --- + cleanupDuringStream::greet("Streaming") + `); + + // Start the native call without awaiting it, then immediately race + // cleanup() against it. + const firstNext = gen.next(); + // Retain the cleanup promise so its rejection cannot escape as an unhandled + // rejection and so native teardown is actually awaited before the test ends + // (review #9 #3). It is awaited in the finally below. + const cleanupPromise = dw.cleanup(); + + try { + // The outcome (a settled chunk, the terminal metadata, or a rejection) + // doesn't matter -- what matters is that it settles instead of crashing + // the process or hanging, and that no unhandled rejection escapes this + // test. We explicitly catch here (rather than asserting a specific + // resolution) and prove settlement, one way or the other. + let settled = false; + try { + await firstNext; + settled = true; + } catch (err) { + settled = true; + expect(err).toBeDefined(); + } + expect(settled).toBe(true); + + // Drain whatever remains so no background callback fires after this test + // (and this file's process) moves on. + try { + let result = await gen.next(); + while (!result.done) { + result = await gen.next(); + } + } catch { + // Draining after a mid-stream cleanup may itself reject; that's fine. + } + } finally { + // Always await the retained cleanup so native teardown finishes before the + // test returns; a cleanup rejection here surfaces rather than dangling, but + // it does not mask a primary assertion failure thrown from the try above. + await cleanupPromise; + } + }); + + // Deadlock regression: unlike the F1 test above (which races cleanup() + // against a stream that fails before emitting data), this test uses a + // script that produces real output with enough volume that the worker + // thread is genuinely attached and mid-delivery -- blocked in + // napi_call_threadsafe_function(..., napi_tsfn_blocking) -- when cleanup() + // drops the last native reference. Before the fix (napi_cleanup's + // synchronous uv_thread_join), this scenario hung the process; after the + // fix, cleanup() defers teardown to a waiter thread until this op drains, + // so both the cleanup() promise and the streaming generator settle. + it('cleanup() during an active, output-producing runStreaming() does not deadlock', async () => { + const dw = trackedDataWeave(); + dw.initialize(); + + const gen = dw.runStreaming( + 'output application/json --- (1 to 5000) map {id: $, name: "item_" ++ $}' + ); + + // Pin the operation without draining it: exactly one .next() call runs + // the generator's synchronous prefix (including the native call that + // hands the op to a background thread) up to its first await. + const firstNext = gen.next(); + + const cleanupPromise = dw.cleanup(); + + await expect( + Promise.race([ + cleanupPromise, + new Promise((_, reject) => setTimeout(() => reject(new Error('cleanup() timed out')), 10000)), + ]) + ).resolves.toBeUndefined(); + + // Drain whatever remains; the stream itself must also settle, not hang. + let result = await firstNext; + while (!result.done) { + result = await gen.next(); + } + expect(result.value).toBeDefined(); + }, 15000); + + // Same deadlock regression as above, for runTransform() -- the design doc + // notes the same problem applies to transform's write_tsfn delivery path. + it('cleanup() during an active, output-producing runTransform() does not deadlock', async () => { + const dw = trackedDataWeave(); + dw.initialize(); + + const parts: Buffer[] = [Buffer.from("[")]; + for (let i = 1; i <= 2000; i++) { + if (i > 1) parts.push(Buffer.from(",")); + parts.push(Buffer.from(`{"id":${i}}`)); + } + parts.push(Buffer.from("]")); + const inputData = [Buffer.concat(parts)]; + + const gen = dw.runTransform( + "output application/json\n---\npayload map $", + inputData, + { mimeType: "application/json" } + ); + + const firstNext = gen.next(); + + // Unlike runStreaming (whose native call is synchronous up to its first + // await), runTransform's generator body awaits createChunkReader(input) + // -- itself a microtask, not real async work for a sync-iterable input -- + // before reaching the native runScriptTransformEngine call. A single + // un-awaited .next() only advances the generator to that intermediate + // await, not past it, so the native op would not yet be dispatched + // (g_active_ops still 0) when cleanup() below fires. One extra microtask + // tick lets that internal await settle so the native call is actually + // in flight, which is what this test needs to race against. + await Promise.resolve(); + + const cleanupPromise = dw.cleanup(); + + await expect( + Promise.race([ + cleanupPromise, + new Promise((_, reject) => setTimeout(() => reject(new Error('cleanup() timed out')), 10000)), + ]) + ).resolves.toBeUndefined(); + + let result = await firstNext; + while (!result.done) { + result = await gen.next(); + } + expect(result.value).toBeDefined(); + }, 15000); + + // Fast-path regression guard: cleanup() called once a stream has already + // fully drained (g_active_ops back to 0 by the time the last reference is + // released) must still resolve via the original, unchanged inline fast + // path -- confirming the new deferred-teardown branch didn't silently + // become the only path through napi_cleanup. + it('cleanup() after a stream has already fully drained resolves via the fast path', async () => { + const dw = trackedDataWeave(); + dw.initialize(); + + const gen = dw.runStreaming('output application/json --- {a: 1}'); + let result = await gen.next(); + while (!result.done) { + result = await gen.next(); + } + expect(result.value.success).toBe(true); + + await expect(dw.cleanup()).resolves.toBeUndefined(); + }); + + // Idempotency / re-entrant cleanup: two cleanup() calls that both arrive + // while a stream is active must both resolve off the same underlying + // teardown -- without spawning a second waiter thread, throwing, or + // decrementing g_ref_count below 0. + it('two concurrent cleanup() calls during an active stream both resolve cleanly', async () => { + const dw = trackedDataWeave(); + dw.initialize(); + + const gen = dw.runStreaming( + 'output application/json --- (1 to 3000) map {id: $}' + ); + const firstNext = gen.next(); + + const [r1, r2] = await Promise.all([ + Promise.race([ + dw.cleanup(), + new Promise((_, reject) => setTimeout(() => reject(new Error('first cleanup() timed out')), 10000)), + ]), + Promise.race([ + dw.cleanup(), + new Promise((_, reject) => setTimeout(() => reject(new Error('second cleanup() timed out')), 10000)), + ]), + ]); + expect(r1).toBeUndefined(); + expect(r2).toBeUndefined(); + + let result = await firstNext; + while (!result.done) { + result = await gen.next(); + } + }, 15000); + + // Re-initialize during pending teardown: starting a stream, calling + // cleanup() without awaiting it, then immediately calling initialize() + // again must block (at the native layer, inside napi_initialize) until the + // pending teardown finishes, rather than racing a second + // graal_create_isolate against an isolate that is still tearing down. The + // instance must be fully usable afterward. + it('initialize() called during a pending teardown waits for it and then works', async () => { + const dw = trackedDataWeave(); + dw.initialize(); + + const gen = dw.runStreaming( + 'output application/json --- (1 to 3000) map {id: $}' + ); + const firstNext = gen.next(); + + // Deliberately not awaited -- this is the pending-teardown state under test. + const cleanupPromise = dw.cleanup(); + + // dw.cleanup() already set dw's own initialized flag false only after its + // internal await resolves; to exercise the *native* pending-teardown path + // independent of this specific instance's TS-level guard, drive a second, + // fresh instance's initialize() concurrently -- it shares the same + // process-global isolate/g_ref_count. + const dw2 = trackedDataWeave(); + const secondInitDone = new Promise((resolve) => { + dw2.initialize(); + resolve(); + }); + + await Promise.race([ + Promise.all([cleanupPromise, secondInitDone]), + new Promise((_, reject) => setTimeout(() => reject(new Error('initialize()-during-teardown timed out')), 10000)), + ]); + + expect(dw2.run("6 * 7").getString()).toBe("42"); + + let result = await firstNext; + while (!result.done) { + result = await gen.next(); + } + }, 15000); + + // Node-layer contract (F4-adjacent): once cleanup() has torn an instance + // down, run() must be rejected by dataweave.ts's own ensureInitialized() + // guard -- a DataWeaveError with a "not initialized" message -- rather than + // reaching the native addon at all with a handle that no longer refers to a + // live engine. This is the TS-level half of the destroyed/unknown-handle + // contract; the native "Unknown engine handle" string is the deeper + // contract the addon enforces if it were ever called with a stale handle, + // which this guard prevents from happening via the public API. + it('run() after cleanup() throws a DataWeaveError via the TS-level ensureInitialized guard', async () => { + const dw = trackedDataWeave({ + resolveModule: modulesFromMap({ + 'org/test/destroyedHandleLib.dwl': '...', + }), + }); + dw.initialize(); + await dw.cleanup(); + + expect(() => dw.run('1 + 1')).toThrow(DataWeaveError); + expect(() => dw.run('1 + 1')).toThrow(/DataWeave runtime not initialized/); + }); }); diff --git a/native-lib/node/tests/integration/dataweave.test.ts b/native-lib/node/tests/integration/dataweave.test.ts index bacf1606..e5af4608 100644 --- a/native-lib/node/tests/integration/dataweave.test.ts +++ b/native-lib/node/tests/integration/dataweave.test.ts @@ -3,8 +3,8 @@ import { readFileSync } from "node:fs"; import { join } from "node:path"; import { DataWeave, run, runStreaming, runTransform, cleanup } from "../../src/index"; -afterAll(() => { - cleanup(); +afterAll(async () => { + await cleanup(); }); describe("DataWeave Node.js API", () => { @@ -21,7 +21,7 @@ describe("DataWeave Node.js API", () => { expect(result.getString()).toBe("42"); }); - it("explicit instance lifecycle", () => { + it("explicit instance lifecycle", async () => { const dw = new DataWeave(); dw.initialize(); try { @@ -30,7 +30,7 @@ describe("DataWeave Node.js API", () => { const r2 = dw.run("sqrt(10000)"); expect(r2.getString()).toBe("100"); } finally { - dw.cleanup(); + await dw.cleanup(); } }); diff --git a/native-lib/node/tests/integration/edge-cases.test.ts b/native-lib/node/tests/integration/edge-cases.test.ts index d1077e67..099659ab 100644 --- a/native-lib/node/tests/integration/edge-cases.test.ts +++ b/native-lib/node/tests/integration/edge-cases.test.ts @@ -6,8 +6,8 @@ import { describe, it, expect, afterAll } from "vitest"; import { DataWeave, run, runStreaming, runTransform, cleanup } from "../../src/index"; import type { StreamingResult } from "../../src/types"; -afterAll(() => { - cleanup(); +afterAll(async () => { + await cleanup(); }); /** Drains a streaming/transform generator, returning its chunks and terminal metadata. */ @@ -61,7 +61,7 @@ describe("runTransform with async-iterable input", () => { }); describe("multi-instance lifecycle", () => { - it("runs two independent instances and cleans them up independently", () => { + it("runs two independent instances and cleans them up independently", async () => { const a = new DataWeave(); const b = new DataWeave(); a.initialize(); @@ -70,8 +70,8 @@ describe("multi-instance lifecycle", () => { expect(a.run("1 + 1").getString()).toBe("2"); expect(b.run("2 + 3").getString()).toBe("5"); } finally { - a.cleanup(); - b.cleanup(); + await a.cleanup(); + await b.cleanup(); } // After cleanup, a fresh instance still works (runtime not permanently torn down). const c = new DataWeave(); @@ -79,22 +79,22 @@ describe("multi-instance lifecycle", () => { try { expect(c.run("6 * 7").getString()).toBe("42"); } finally { - c.cleanup(); + await c.cleanup(); } }); - it("initialize is idempotent and re-initialization after cleanup works", () => { + it("initialize is idempotent and re-initialization after cleanup works", async () => { const dw = new DataWeave(); dw.initialize(); dw.initialize(); // no-op, must not throw expect(dw.run("1").getString()).toBe("1"); - dw.cleanup(); - dw.cleanup(); // double cleanup, must not throw + await dw.cleanup(); + await dw.cleanup(); // double cleanup, must not throw dw.initialize(); // re-init try { expect(dw.run("2").getString()).toBe("2"); } finally { - dw.cleanup(); + await dw.cleanup(); } }); diff --git a/native-lib/node/tests/integration/engine-handle-contract.test.ts b/native-lib/node/tests/integration/engine-handle-contract.test.ts new file mode 100644 index 00000000..04a192c2 --- /dev/null +++ b/native-lib/node/tests/integration/engine-handle-contract.test.ts @@ -0,0 +1,279 @@ +import { describe, it, expect, beforeAll, afterAll } from "vitest"; +import * as ffi from "../../src/ffi"; +import { findLibrary, buildInputsJson } from "../../src/utils"; + +// W-23692110 round 11 finding #6. +// +// The Java `ScriptRuntimeTest` only asserts on the UNKNOWN_ENGINE_HANDLE_JSON +// constant -- the @CEntryPoint methods it wraps cannot run in a hosted JVM, so +// nothing has ever driven the real `*_engine` entrypoints through the +// compiled addon against an unknown or destroyed handle. This file closes +// that gap: it loads the REAL addon (no `vi.mock` of ffi) and drives +// `runScriptEngine` / `runScriptStreamingEngine` / `runScriptTransformEngine` +// directly through the raw `ffi` module -- the addon boundary the finding is +// about -- against handles that were never registered and against handles +// that were registered and then destroyed. +// +// Confirmed empirically (see task-6-report.md) against the real addon: +// - sync `runScriptEngine` RETURNS the JSON string +// `{"success":false,"error":"Unknown engine handle"}` -- it does not throw. +// - `runScriptStreamingEngine` / `runScriptTransformEngine` RESOLVE (never +// reject) their promise with that same JSON string as the terminal +// metadata; no chunk callback fires for an unknown/destroyed handle. +// This is the same envelope produced by NativeLib.UNKNOWN_ENGINE_HANDLE_JSON +// on the Java side (native-lib/src/main/java/org/mule/weave/lib/NativeLib.java), +// threaded back through addon.c's engine entrypoints and unmodified by the TS +// parsing layer (parseNativeResponse / parseStreamingResult in src/result.ts). +// +// The native addon globals (g_ref_count, g_initialized, g_bridges, etc.) are +// process-wide C statics -- vitest's per-file module isolation does NOT reset +// them, and napi_initialize/napi_cleanup are plain integer ref-counts (one +// increment per initialize(), one decrement per cleanup(), teardown only on +// the transition to zero). So this file calls ffi.initialize() exactly ONCE +// for the whole suite (beforeAll), balanced by exactly one ffi.cleanup() that +// brings the ref count to zero (in the last real test, "final cleanup..." +// below) -- mirroring independent-engines.test.ts's single +// initialize()/cleanup() pair rather than handle-validation.test.ts's +// per-test balancing (that file calls initialize()/cleanup() once per test, +// which does not fit here since several tests below deliberately build on a +// still-live engine/isolate from a prior test). The trailing afterAll is a +// pure safety net (idempotent no-op on the happy path) in case an earlier +// assertion throws before the drainage test runs, so this file never strands +// a ref-count bump for sibling integration test files sharing the same +// vitest worker process. +describe("*_engine unknown/destroyed-handle contract (round 11 #6)", () => { + beforeAll(() => { + ffi.initialize(findLibrary()); + }); + + afterAll(async () => { + // Idempotent: a no-op if the ref count already reached zero (the normal + // case -- the drainage test below already did that). A genuine safety + // net only if an earlier test threw before reaching that point. + await ffi.cleanup(); + }); + + // A handle value that was never handed out by createEngine()/ + // createEngineWithResolver() (those only ever return small positive + // handles from the Java-side registry) and can never collide with one. + const UNKNOWN_HANDLE = Number.MAX_SAFE_INTEGER; + const UNKNOWN_ENVELOPE = { success: false, error: "Unknown engine handle" }; + + it("runScriptEngine on a never-registered handle returns the terminal envelope, does not throw", () => { + let raw: string | undefined; + expect(() => { + raw = ffi.runScriptEngine( + UNKNOWN_HANDLE, + "%dw 2.0\noutput application/json\n---\n1 + 1", + buildInputsJson({}) + ); + }).not.toThrow(); + + expect(JSON.parse(raw!)).toEqual(UNKNOWN_ENVELOPE); + }); + + it("runScriptStreamingEngine on a never-registered handle resolves (never rejects) with the terminal envelope", async () => { + const chunks: Buffer[] = []; + const raw = await ffi.runScriptStreamingEngine( + UNKNOWN_HANDLE, + "%dw 2.0\noutput application/json\n---\n[1, 2, 3]", + buildInputsJson({}), + (chunk) => chunks.push(chunk) + ); + + expect(JSON.parse(raw)).toEqual(UNKNOWN_ENVELOPE); + // No output was ever produced for an engine that doesn't exist. + expect(chunks).toHaveLength(0); + }); + + it("runScriptTransformEngine on a never-registered handle resolves (never rejects) with the terminal envelope", async () => { + let readCalls = 0; + let firstRead = true; + const readCb = (_bufSize: number): Buffer | null => { + readCalls++; + if (firstRead) { + firstRead = false; + return Buffer.from("1"); + } + return null; + }; + const chunks: Buffer[] = []; + const writeCb = (chunk: Buffer) => chunks.push(chunk); + + const raw = await ffi.runScriptTransformEngine( + UNKNOWN_HANDLE, + "output application/json\n---\npayload", + "{}", + "payload", + "application/json", + null, + readCb, + writeCb + ); + + expect(JSON.parse(raw)).toEqual(UNKNOWN_ENVELOPE); + // The unknown-handle rejection happens before a worker is ever spawned, + // so the read/write callbacks are never invoked. + expect(readCalls).toBe(0); + expect(chunks).toHaveLength(0); + }); + + it("all three entrypoints on a destroyed handle return/resolve the same terminal envelope, after proving the handle worked", async () => { + const handle = ffi.createEngine(); + + // Prove the handle is genuinely live before destroying it. + const preDestroy = JSON.parse( + ffi.runScriptEngine(handle, "%dw 2.0\noutput application/json\n---\n1 + 1", buildInputsJson({})) + ); + expect(preDestroy.success).toBe(true); + + ffi.destroyEngine(handle); + + // Sync entrypoint: returns the envelope, does not throw. + let syncRaw: string | undefined; + expect(() => { + syncRaw = ffi.runScriptEngine(handle, "%dw 2.0\noutput application/json\n---\n1", buildInputsJson({})); + }).not.toThrow(); + expect(JSON.parse(syncRaw!)).toEqual(UNKNOWN_ENVELOPE); + + // Streaming entrypoint: resolves with the envelope. + const streamChunks: Buffer[] = []; + const streamRaw = await ffi.runScriptStreamingEngine( + handle, + "%dw 2.0\noutput application/json\n---\n[1, 2, 3]", + buildInputsJson({}), + (chunk) => streamChunks.push(chunk) + ); + expect(JSON.parse(streamRaw)).toEqual(UNKNOWN_ENVELOPE); + expect(streamChunks).toHaveLength(0); + + // Transform entrypoint: resolves with the envelope. + let transformReadCalls = 0; + let transformFirstRead = true; + const transformReadCb = (_bufSize: number): Buffer | null => { + transformReadCalls++; + if (transformFirstRead) { + transformFirstRead = false; + return Buffer.from("1"); + } + return null; + }; + const transformChunks: Buffer[] = []; + const transformRaw = await ffi.runScriptTransformEngine( + handle, + "output application/json\n---\npayload", + "{}", + "payload", + "application/json", + null, + transformReadCb, + (chunk) => transformChunks.push(chunk) + ); + expect(JSON.parse(transformRaw)).toEqual(UNKNOWN_ENVELOPE); + expect(transformReadCalls).toBe(0); + expect(transformChunks).toHaveLength(0); + }); + + // Same-thread post-admission ordering (deterministic, not best-effort): + // destroyEngine() is fired synchronously immediately after admission of the + // op (right after starting runScriptStreamingEngine, before awaiting it). + // The round-11 #2/#3 pin is taken atomically at admission, under g_mutex, in + // bridge_begin_op_locked -- so this same-thread ordering deterministically + // lands AFTER the pin is already held. That means the op MUST complete + // successfully with complete chunks; there is no closed set of "success or + // Unknown-engine-handle envelope" to tolerate here, because the envelope can + // only arise if the pin were NOT held at admission. Requiring success (and + // no longer accepting the envelope) makes this test fail if a future + // regression drops the admission-time pin, instead of silently passing by + // returning the accepted terminal envelope. + // + // Genuinely concurrent cross-thread interleavings (a real Worker racing + // destroyEngine() against admission on a different thread) are a distinct, + // non-deterministic window that this same-thread ordering does not exercise + // and cannot stand in for. That case remains covered best-effort by the + // forthcoming Worker-based suite (Task 8), matching the documented posture + // of rounds 5-10's cross-Worker races (see run-admission.test.ts / + // admission-during-teardown.test.ts) -- it is not tolerated away in this + // test. + it( + "destroyEngine() fired right after admission of an in-flight streaming op deterministically succeeds (pin held at admission)", + async () => { + const ITERATIONS = 50; + for (let i = 0; i < ITERATIONS; i++) { + const handle = ffi.createEngine(); + const chunks: Buffer[] = []; + const resultPromise = ffi.runScriptStreamingEngine( + handle, + "%dw 2.0\noutput application/json\n---\n[1, 2, 3]", + buildInputsJson({}), + (chunk) => chunks.push(chunk) + ); + // Fire destroy immediately after admission, before awaiting. The round-11 + // pin is taken atomically at admission (under g_mutex, in + // bridge_begin_op_locked), so this ordering lands AFTER the pin and the + // op MUST complete successfully. Requiring success (not tolerating the + // Unknown-engine-handle envelope) makes this test fail if a regression + // drops the admission-time pin. + expect(() => ffi.destroyEngine(handle)).not.toThrow(); + + const raw = await resultPromise; + const parsed = JSON.parse(raw); + expect(parsed.success).toBe(true); + expect(JSON.parse(Buffer.concat(chunks).toString("utf-8"))).toEqual([1, 2, 3]); + } + }, + 60000 + ); + + it("deferred registry removal after an in-flight op finalizes without wedging the isolate (round 12 #3)", async () => { + // Uses the shared beforeAll isolate. Create an engine, start a streaming + // op, destroy the engine while the op is admitted, drain the op. The + // deferred finalize (bridge_end_op -> bridge_finalize_registry) must + // complete and a subsequent run on a fresh engine must still work + // (isolate not torn down / not wedged by the transient reservation). + const handle = ffi.createEngine(); + const chunks: Buffer[] = []; + const resultPromise = ffi.runScriptStreamingEngine( + handle, + "%dw 2.0\noutput application/json\n---\n[1, 2, 3]", + buildInputsJson({}), + (chunk) => chunks.push(chunk) + ); + expect(() => ffi.destroyEngine(handle)).not.toThrow(); + const raw = await resultPromise; + const parsed = JSON.parse(raw); + // Pin held at admission (round 11) -> success expected; either way no crash. + if (parsed.success) { + expect(JSON.parse(Buffer.concat(chunks).toString("utf-8"))).toEqual([1, 2, 3]); + } + // Isolate still healthy after the deferred finalize ran: + const h2 = ffi.createEngine(); + const envelope = JSON.parse( + ffi.runScriptEngine(h2, "%dw 2.0\noutput application/json\n---\n2 + 2", buildInputsJson({})) + ); + expect(envelope.success).toBe(true); + expect(JSON.parse(Buffer.from(envelope.result, "base64").toString("utf-8"))).toBe(4); + ffi.destroyEngine(h2); + }); + + it("final cleanup drains the shared isolate (idempotent)", async () => { + // Exactly one ffi.initialize() ran for this whole file (beforeAll), so + // this is the ONE balancing ffi.cleanup() that brings the native + // g_ref_count to zero and genuinely tears the isolate down (napi_cleanup + // Case 4, since no op is in flight) -- not a no-op decrement of a + // still-positive count left over from other tests. Prove that teardown + // actually happened, not just that the call resolved: a subsequent + // engine-level call must now observe "not initialized" rather than + // silently succeeding against a still-live isolate. + await ffi.cleanup(); + + expect(() => + ffi.runScriptEngine(UNKNOWN_HANDLE, "%dw 2.0\noutput application/json\n---\n1", buildInputsJson({})) + ).toThrow(/not initialized/i); + + // A second cleanup() call after the ref count already reached zero must + // remain a safe no-op, mirroring independent-engines.test.ts's final + // teardown discipline. + await expect(ffi.cleanup()).resolves.toBeUndefined(); + }); +}); diff --git a/native-lib/node/tests/integration/env-init-ownership.test.ts b/native-lib/node/tests/integration/env-init-ownership.test.ts new file mode 100644 index 00000000..fe30aa90 --- /dev/null +++ b/native-lib/node/tests/integration/env-init-ownership.test.ts @@ -0,0 +1,94 @@ +import { describe, it, expect } from "vitest"; +import * as ffi from "../../src/ffi"; +import { findLibrary, buildInputsJson } from "../../src/utils"; + +// W-23692110 round 13 #5: the init reference is owned per napi_env, not per +// engine. These raw-ffi tests (no vi.mock) drive the addon boundary directly -- +// the exact surface the finding is about -- and use the ref-count proxy from +// instance-lifecycle.test.ts: after balancing to zero refs a raw engine call +// throws /not initialized/; while the isolate is live a run succeeds. +// +// IMPORTANT -- these are single-env SMOKE tests, NOT true #5 regression teeth. +// #5 is a CROSS-ENV bug: an abandoned/dying env with N engines under one +// initialize() firing N per-engine releases against the one reference it owns, +// or one env's cleanup()/env-death releasing a reference another env owns. Both +// require either a real dying env or two distinct napi_envs with asymmetric +// init/cleanup. Vitest runs these on the single main-thread env, so they cannot +// distinguish the fixed isolate from the pre-fix (buggy) one -- it was verified +// empirically that both cases below pass unchanged when rebuilt against the +// pre-round-13 addon (destroyEngine() never released the init ref in any +// revision, and the second cleanup() was already a no-op via the long-standing +// `if (g_ref_count > 0)` floor). They guard that the sanctioned single-env path +// still behaves (liveness + no double-decrement corruption); they do NOT prove +// #5 is fixed. The cross-env behavior that #5 is actually about -- an abandoned env with N +// engines under one initialize() -- is now pinned by the dedicated cross-env +// regression test in worker-lifecycle.test.ts ("a Worker that inits once + +// creates N engines + exits without cleanup() does NOT tear down the isolate +// under a live main engine"), which fails RED on the round-12 implementation +// and passes at round 13+. These single-env smoke tests remain as a fast guard +// on the sanctioned single-env liveness path. + +const LIB = findLibrary(); + +function runOn(handle: number, expr: string): unknown { + const envelope = JSON.parse( + ffi.runScriptEngine(handle, `%dw 2.0\noutput application/json\n---\n${expr}`, buildInputsJson({})) + ); + expect(envelope.success).toBe(true); + return JSON.parse(Buffer.from(envelope.result, "base64").toString("utf-8")); +} + +describe("per-env init-reference ownership -- single-env smoke tests (round 13 #5)", () => { + // Smoke test (NOT a #5 regression test -- see file header): destroyEngine() + // never released the init reference in any revision, so this held pre-fix too. + it("smoke: one initialize() + multiple engines stays live when a single engine is destroyed", () => { + ffi.initialize(LIB); // ONE init reference for this env + const h1 = ffi.createEngine(); + const h2 = ffi.createEngine(); + expect(runOn(h2, "6 * 7")).toBe(42); + + // Destroy one engine. The isolate reference belongs to initialize(), not to + // an engine, so the isolate must stay alive and h2 must still run. + ffi.destroyEngine(h1); + expect(runOn(h2, "1 + 1")).toBe(2); + + // Balance: destroy the other engine and release the single init reference. + ffi.destroyEngine(h2); + // eslint-disable-next-line @typescript-eslint/no-floating-promises + return ffi.cleanup().then(() => { + expect(() => + ffi.runScriptEngine(Number.MAX_SAFE_INTEGER, "%dw 2.0\noutput application/json\n---\n1", buildInputsJson({})) + ).toThrow(/not initialized/i); + }); + }); + + // Smoke test (NOT a #5 regression test -- see file header): the second + // cleanup() was already a no-op pre-fix via the `if (g_ref_count > 0)` floor, + // so with one env this passes on the buggy addon too. #5's gate protects the + // CROSS-env case (one env stealing another's reference), not observable here. + it("smoke: a second cleanup() on an env that owns no reference does not corrupt the count", async () => { + ffi.initialize(LIB); // init_refs = 1 + const h = ffi.createEngine(); + expect(runOn(h, "2 + 2")).toBe(4); + ffi.destroyEngine(h); + + // First cleanup releases this env's one reference -> isolate torn down. + await ffi.cleanup(); + // Second cleanup: this env's init_refs is already 0. Must be a no-op -- + // it must NOT drive g_ref_count negative or perturb a later isolate. + await ffi.cleanup(); + + // Prove the count was not corrupted: a fresh, fully-balanced init/run/cleanup + // cycle still nets to zero (a corrupted negative count would leave the next + // isolate un-torn-down and this final probe would NOT report not-initialized). + ffi.initialize(LIB); + const h2 = ffi.createEngine(); + expect(runOn(h2, "3 + 4")).toBe(7); + ffi.destroyEngine(h2); + await ffi.cleanup(); + + expect(() => + ffi.runScriptEngine(Number.MAX_SAFE_INTEGER, "%dw 2.0\noutput application/json\n---\n1", buildInputsJson({})) + ).toThrow(/not initialized/i); + }); +}); diff --git a/native-lib/node/tests/integration/first-resolver-wins.test.ts b/native-lib/node/tests/integration/first-resolver-wins.test.ts deleted file mode 100644 index 75e7da59..00000000 --- a/native-lib/node/tests/integration/first-resolver-wins.test.ts +++ /dev/null @@ -1,34 +0,0 @@ -// Verifies the process-wide "first resolver wins" behavior documented in -// docs/external-modules.md#multiple-resolvers-in-one-process and -// ScriptRuntime.setResolver(): once a DataWeave instance's resolver is -// installed on the native engine singleton, a second instance constructed -// with a *different* resolver in the same process never has its resolver -// installed. That's only observable when the second instance's resolver is -// the second one ever installed for the whole process, so — like -// init-bad-path.test.ts — this runs in a dedicated child process rather than -// in-lane, making it order- and pool-configuration-independent. -import { describe, it, expect } from "vitest"; -import { execFileSync } from "node:child_process"; -import { join } from "node:path"; -import { existsSync } from "node:fs"; - -const FIXTURE = join(__dirname, "fixtures", "first-resolver-wins.cjs"); -const DIST_ENTRY = join(__dirname, "..", "..", "dist", "index.js"); - -describe("first-resolver-wins (isolated process)", () => { - it("a second DataWeave instance's resolver is silently ignored in favor of the first", () => { - expect(existsSync(DIST_ENTRY), `built entry missing at ${DIST_ENTRY} — run \`npm run build:ts\``).toBe(true); - - // execFileSync throws on a non-zero exit, so a "wrong resolver won" / - // native-crash outcome in the child fails this test. A timeout is also - // required: execFileSync blocks synchronously with no way for Vitest to - // interrupt it, so a native deadlock in the child would otherwise hang - // the whole suite instead of failing this one test. - const stdout = execFileSync(process.execPath, [FIXTURE], { - encoding: "utf-8", - timeout: 30_000, - }); - - expect(stdout).toContain("OK:first-resolver-wins"); - }); -}); diff --git a/native-lib/node/tests/integration/fixtures/first-resolver-wins.cjs b/native-lib/node/tests/integration/fixtures/first-resolver-wins.cjs deleted file mode 100644 index 3dc2fd42..00000000 --- a/native-lib/node/tests/integration/fixtures/first-resolver-wins.cjs +++ /dev/null @@ -1,103 +0,0 @@ -// Child-process fixture for the first-resolver-wins regression test. -// -// Runs in a FRESH process (spawned by first-resolver-wins.test.ts) so the -// process-wide ScriptRuntime singleton in the native layer starts with no -// resolver installed (see ScriptRuntime.setResolver(): once any DataWeave -// instance's resolver is installed, every later instance's resolver is -// silently ignored — a warning is logged and the first resolver keeps being -// used). That behavior is only observable on the FIRST resolver installation -// of a process, so this fixture -- not an in-lane vitest test -- is the only -// reliable way to exercise it. -// -// Contract with the parent: -// - Requires the built CommonJS entry at ../../../dist/index.js. -// - Constructs dw1 with a resolver for 'first.dwl' and dw2 with a -// *different* resolver for 'second.dwl', then initializes both. -// - Runs a script through dw1 that imports 'first.dwl' to force-install -// dw1's resolver on the singleton engine (must succeed). -// - Runs a script through dw2 that imports 'second.dwl'. Per the singleton -// semantics, dw2's resolver is never installed, so this import must fail. -// - Runs a THIRD script, through dw2, that imports 'first.dwl' again and -// asserts it still returns "Hello World". This is the check that actually -// distinguishes "the first resolver remains active" from "custom -// resolution broke entirely after the first call" — the second script -// alone would fail identically under either explanation. -// - Always calls cleanup() on both instances via try/finally, so teardown -// is exercised even on failure, then exits naturally (no process.exit()). -// - Prints "OK:first-resolver-wins" when all three expectations hold, or -// "FAIL:" (with a non-zero exitCode) otherwise. A native crash -// surfaces as a non-zero signal exit, which the parent also treats as -// failure. -const path = require("node:path"); - -const { DataWeave, modulesFromMap } = require(path.join(__dirname, "..", "..", "..", "dist", "index.js")); - -const dw1 = new DataWeave({ - resolveModule: modulesFromMap({ - "first.dwl": '%dw 2.0\nfun greet(n: String) = "Hello " ++ n', - }), -}); - -const dw2 = new DataWeave({ - resolveModule: modulesFromMap({ - "second.dwl": '%dw 2.0\nfun shout(n: String) = n ++ "!"', - }), -}); - -let failure = null; - -try { - dw1.initialize(); - dw2.initialize(); - - const firstResult = dw1.run(` - %dw 2.0 - import first - output application/json - --- - first::greet("World") - `); - - if (!firstResult.success) { - failure = "first-resolver-did-not-resolve:" + firstResult.error; - } else { - const secondResult = dw2.run(` - %dw 2.0 - import second - output application/json - --- - second::shout("hi") - `); - - if (secondResult.success) { - failure = "second-resolver-unexpectedly-won"; - } else { - // Prove the first resolver is still ACTIVE on dw2 (not merely that - // dw2's own resolver lost). A resolver that died entirely after the - // first call would also make second.dwl fail above -- this second - // check on dw2 is what actually distinguishes "first resolver wins" - // from "custom resolution stopped working after the first run". - const stillFirstResult = dw2.run(` - %dw 2.0 - import first - output application/json - --- - first::greet("World") - `); - - if (!stillFirstResult.success || JSON.parse(stillFirstResult.getString()) !== "Hello World") { - failure = "first-resolver-no-longer-active-on-dw2:" + (stillFirstResult.error || stillFirstResult.getString()); - } - } - } -} finally { - dw1.cleanup(); - dw2.cleanup(); -} - -if (failure) { - console.log("FAIL:" + failure); - process.exitCode = 1; -} else { - console.log("OK:first-resolver-wins"); -} diff --git a/native-lib/node/tests/integration/handle-validation.test.ts b/native-lib/node/tests/integration/handle-validation.test.ts new file mode 100644 index 00000000..2feddfb8 --- /dev/null +++ b/native-lib/node/tests/integration/handle-validation.test.ts @@ -0,0 +1,73 @@ +import { describe, it, expect, afterEach } from "vitest"; +import * as ffi from "../../src/ffi"; +import { findLibrary } from "../../src/utils"; + +// Round-6 finding #1 (defense-in-depth): the native handle-read sites +// (napi_get_value_int64 in napi_run_script_engine, +// napi_run_script_streaming_engine, napi_run_script_transform_engine) must +// reject a non-integer handle argument instead of silently using +// uninitialized/garbage stack data as the engine handle. +// +// This is driven through `ffi` (the raw addon boundary), not through the +// `DataWeave` class, because Task 1's JS-layer state guard only ever passes +// `this.engineHandle` (always a number once initialized) down to the native +// call -- so a bad handle can never reach these C sites through the public +// TS API. Each `ffi.xxx` export is a pure pass-through to the native addon +// (see src/ffi.ts: no validation of its own), so calling them directly with +// a non-numeric "handle" exercises the raw C boundary while reusing the same +// initialize()/findLibrary() bootstrap the other integration tests use. +// +// One test covers all three sites (rather than three separate tests) to keep +// the suite's test count increasing by exactly one for this task. +// +// Real addon, no mocking. +// +// The native addon globals (g_ref_count, g_initialized, etc.) are +// process-wide C statics -- vitest's per-file module isolation does NOT +// reset them. Every ffi.initialize() here must be balanced by a matching +// ffi.cleanup() so this file doesn't leak a ref-count bump into sibling +// integration test files sharing the same vitest worker process (mirrors +// admission-during-teardown.test.ts's care to drain/settle before the file +// ends, and instance-lifecycle.test.ts's afterEach cleanup pattern). +describe("native handle validation (round 6 #1)", () => { + afterEach(async () => { + await ffi.cleanup(); + }); + + it("runScriptEngine/runScriptStreamingEngine/runScriptTransformEngine all throw on a non-integer handle rather than using garbage", () => { + ffi.initialize(findLibrary()); + + // napi_get_value_int64 must fail (and be checked) for a non-numeric + // handle argument; each site must throw cleanly instead of proceeding + // with whatever `handle64` happened to contain on the stack. + expect(() => + ffi.runScriptEngine( + {} as unknown as number, + "%dw 2.0\noutput application/json\n---\n1", + "{}" + ) + ).toThrow(); + + expect(() => + ffi.runScriptStreamingEngine( + {} as unknown as number, + "%dw 2.0\noutput application/json\n---\n1", + "{}", + () => {} + ) + ).toThrow(); + + expect(() => + ffi.runScriptTransformEngine( + {} as unknown as number, + "output application/json\n---\npayload", + "{}", + "payload", + "application/json", + null, + () => null, + () => {} + ) + ).toThrow(); + }); +}); diff --git a/native-lib/node/tests/integration/independent-engines.test.ts b/native-lib/node/tests/integration/independent-engines.test.ts new file mode 100644 index 00000000..6dfdc7f9 --- /dev/null +++ b/native-lib/node/tests/integration/independent-engines.test.ts @@ -0,0 +1,108 @@ +import { describe, it, expect, afterAll } from "vitest"; +import { DataWeave, cleanup } from "../../src/dataweave"; +import { modulesFromMap } from "../../src/resolver"; + +const instances: DataWeave[] = []; +function tracked(...args: ConstructorParameters): DataWeave { + const dw = new DataWeave(...args); + instances.push(dw); + return dw; +} +afterAll(async () => { + for (const dw of instances) await dw.cleanup(); + await cleanup(); +}); + +const scriptImporting = (mod: string) => + `%dw 2.0\nimport org::test::${mod}\noutput application/json\n---\n${mod}::greet("X")`; + +describe("independent engines (W-23692110)", () => { + it("two instances resolve only their OWN module, with no cross-talk", () => { + const dwA = tracked({ resolveModule: modulesFromMap({ + "org/test/a.dwl": '%dw 2.0\nfun greet(n: String) = "A:" ++ n' }) }); + const dwB = tracked({ resolveModule: modulesFromMap({ + "org/test/b.dwl": '%dw 2.0\nfun greet(n: String) = "B:" ++ n' }) }); + dwA.initialize(); + dwB.initialize(); + + expect(JSON.parse(dwA.run(scriptImporting("a")).getString()!)).toBe("A:X"); + expect(JSON.parse(dwB.run(scriptImporting("b")).getString()!)).toBe("B:X"); + + // Each engine misses the other's module. + expect(dwA.run(scriptImporting("b")).success).toBe(false); + expect(dwB.run(scriptImporting("a")).success).toBe(false); + }); + + it("built-in modules resolve in a resolver-backed engine", () => { + const dw = tracked({ resolveModule: modulesFromMap({ "x.dwl": "..." }) }); + dw.initialize(); + const r = dw.run('%dw 2.0\nimport dw::core::Strings\noutput application/json\n---\nStrings::capitalize("hello")'); + expect(r.success).toBe(true); + expect(JSON.parse(r.getString()!)).toBe("Hello"); + }); + + // Carried forward from Task 3's review: runScriptEngine now returns "" (not + // a thrown error) for a NULL native result, pushing error interpretation + // entirely to parseNativeResponse() in this TS layer. A genuine script + // error (as opposed to a NULL/empty native response) must still surface as + // an ordinary unsuccessful ExecutionResult through the new handle-based + // path -- not an unhandled parse exception or process crash. + it("a genuine script error on a resolver-backed engine surfaces as success:false, not a throw", () => { + const dw = tracked({ resolveModule: modulesFromMap({ "x.dwl": "..." }) }); + dw.initialize(); + + let result: ReturnType | undefined; + expect(() => { result = dw.run("invalid_var_xyz"); }).not.toThrow(); + expect(result!.success).toBe(false); + expect(result!.error).toBeTruthy(); + }); + + // Confirms addon.c's argument-shifted runScriptStreamingEngine wiring (handle + // as first argument, per Task 3) actually threads the handle through to a + // real per-engine streaming run, not just the non-streaming run() path + // exercised above. Uses a built-in import (not a custom resolver module): + // runStreaming's native call executes on a background uv_thread whose + // identity differs from the engine's owner thread, so a resolver-backed + // engine fails closed for *custom* modules over streaming by design (see + // dataweave-resolver.test.ts) -- that's not what this test is checking. + it("runStreaming produces output on its own resolver-backed engine", async () => { + const dw = tracked({ resolveModule: modulesFromMap({ "x.dwl": "..." }) }); + dw.initialize(); + + const chunks: Buffer[] = []; + const gen = dw.runStreaming( + '%dw 2.0\nimport dw::core::Strings\noutput application/json\n---\nStrings::capitalize("stream")' + ); + let result = await gen.next(); + while (!result.done) { + chunks.push(result.value); + result = await gen.next(); + } + const metadata = result.value; + + expect(metadata.success).toBe(true); + expect(JSON.parse(Buffer.concat(chunks).toString("utf-8"))).toBe("Stream"); + }); + + // Confirms addon.c's argument-shifted runScriptTransformEngine wiring + // likewise threads the handle through to a real per-engine transform run. + it("runTransform produces output on its own resolver-backed engine", async () => { + const dw = tracked({ resolveModule: modulesFromMap({ "x.dwl": "..." }) }); + dw.initialize(); + + const inputData = [Buffer.from("[1, 2, 3]")]; + const script = "output application/json\n---\npayload map ($ * 10)"; + + const chunks: Buffer[] = []; + const gen = dw.runTransform(script, inputData, { mimeType: "application/json" }); + let result = await gen.next(); + while (!result.done) { + chunks.push(result.value); + result = await gen.next(); + } + const metadata = result.value; + + expect(metadata.success).toBe(true); + expect(JSON.parse(Buffer.concat(chunks).toString("utf-8"))).toEqual([10, 20, 30]); + }); +}); diff --git a/native-lib/node/tests/integration/instance-lifecycle.test.ts b/native-lib/node/tests/integration/instance-lifecycle.test.ts new file mode 100644 index 00000000..e644662e --- /dev/null +++ b/native-lib/node/tests/integration/instance-lifecycle.test.ts @@ -0,0 +1,239 @@ +import { describe, it, expect, afterEach } from "vitest"; +import { DataWeave, run, cleanup } from "../../src/dataweave"; +import { DataWeaveError } from "../../src/errors"; +import * as ffi from "../../src/ffi"; +import { findLibrary, buildInputsJson } from "../../src/utils"; + +// Same-instance lifecycle regression tests (round 6, W-23692110). Round 5's +// coverage used a second instance; the same-instance cleanup window is exactly +// what findings #1 and #3 exploit. Real addon, no mocking. +describe("instance lifecycle during cleanup (round 6)", () => { + let dw: DataWeave | undefined; + afterEach(async (ctx) => { + // Whatever state each test leaves it in, drain and release so the shared + // process-wide isolate is clean for sibling tests. + if (dw) { + const inst = dw; + dw = undefined; + let cleanupErr: unknown; + try { + await inst.cleanup(); + } catch (e) { + cleanupErr = e; + } + // A cleanup() failure is itself a real lifecycle regression: surface it + // when the test body PASSED. Suppress it only when the body already FAILED, + // so the original, more actionable assertion failure keeps propagating + // (review #9 #4; mirrors the worker-lifecycle balancing pattern). + if (cleanupErr !== undefined && ctx.task.result?.state !== "fail") { + throw cleanupErr; + } + } + }); + + // Finding #3: initialize() during the same instance's pending cleanup must + // reject deterministically, not be a silent no-op that leaves the instance + // uninitialized after cleanup settles. + it("initialize() during pending cleanup throws, and re-init works after cleanup settles", async () => { + dw = new DataWeave(); + dw.initialize(); + const closing = dw.cleanup(); // not awaited: instance is now "cleaning-up" + expect(() => dw!.initialize()).toThrow(DataWeaveError); + expect(() => dw!.initialize()).toThrow(/cleanup is in progress/i); + await closing; // now "uninitialized" + // Explicit re-init now succeeds and the instance is usable again. + dw.initialize(); + const r = dw.run("%dw 2.0\noutput application/json\n---\n1 + 1"); + expect(r.success).toBe(true); + expect(JSON.parse(r.getString()!)).toBe(2); + }); + + // Finding #1: run() during the cleanup window must throw a clean DataWeaveError + // (never send a null handle to C), because doCleanup() nulls engineHandle + // synchronously before awaiting native cleanup. + it("run() during pending cleanup throws DataWeaveError, not a native/null-handle error", async () => { + dw = new DataWeave(); + dw.initialize(); + const closing = dw.cleanup(); + expect(() => dw!.run("%dw 2.0\noutput application/json\n---\n1")).toThrow(DataWeaveError); + expect(() => dw!.run("%dw 2.0\noutput application/json\n---\n1")).toThrow(/cleaning up/i); + await closing; + }); + + // Finding #1, streaming/transform variants: the async generators must reject + // on first pull when started during the cleanup window. + it("runStreaming()/runTransform() during pending cleanup reject on first pull", async () => { + dw = new DataWeave(); + dw.initialize(); + const closing = dw.cleanup(); + + const sgen = dw.runStreaming("%dw 2.0\noutput application/json\n---\n[1,2,3]"); + await expect(sgen.next()).rejects.toThrow(DataWeaveError); + + const tgen = dw.runTransform( + "output application/json\n---\npayload", + [Buffer.from("[1,2,3]")], + { mimeType: "application/json" } + ); + await expect(tgen.next()).rejects.toThrow(DataWeaveError); + + await closing; + }); + + // Idempotency preserved: cleanup() before initialize() is a no-op; double + // cleanup() coalesces (round-4 F1 must survive this refactor). + it("cleanup() is a no-op when uninitialized and coalesces when called twice", async () => { + dw = new DataWeave(); + await expect(dw.cleanup()).resolves.toBeUndefined(); // uninitialized no-op + dw.initialize(); + const a = dw.cleanup(); + const b = dw.cleanup(); // must return the same in-flight settlement, one native teardown + await Promise.all([a, b]); + }); +}); + +// Round 12, Task 1: napi_cleanup's Case 1..5 decrement-and-teardown body was +// lifted verbatim into release_isolate_ref_locked() so a later task (round-12 +// #2) can reuse it from the abandoned-env path. This is a behavior-preserving +// refactor; this test pins the observable contract it must not disturb: the +// balancing cleanup() call that drops the ref count to zero must actually +// tear the isolate down synchronously, not leave it silently live. +// +// Driven through the raw `ffi` boundary (like handle-validation.test.ts and +// engine-handle-contract.test.ts), with a balanced initialize()/cleanup() +// pair, so this file doesn't leak a ref-count bump into sibling integration +// test files sharing the same vitest worker process. +describe("napi_cleanup refactor preserves last-release teardown (round 12 Task 1)", () => { + it("the balancing cleanup() actually tears the isolate down (subsequent engine call sees not-initialized)", async () => { + ffi.initialize(findLibrary()); + const h = ffi.createEngine(); + const envelope = JSON.parse( + ffi.runScriptEngine(h, "%dw 2.0\noutput application/json\n---\n1 + 1", buildInputsJson({})) + ); + expect(envelope.success).toBe(true); + expect(JSON.parse(Buffer.from(envelope.result, "base64").toString("utf-8"))).toBe(2); + ffi.destroyEngine(h); + await ffi.cleanup(); + // Ref count reached 0 and the isolate was torn down: a fresh engine call + // must observe "not initialized", not silently run on a live isolate. + expect(() => + ffi.runScriptEngine(Number.MAX_SAFE_INTEGER, "%dw 2.0\noutput application/json\n---\n1", buildInputsJson({})) + ).toThrow(/not initialized/i); + }); +}); + +// Round 12, Task 4: createChunkReader pre-buffers async inputs by awaiting +// the entire iterable up front (see reader.ts), because the native read +// callback is invoked synchronously and cannot await. That await can span +// arbitrarily long, so if the caller cleans up the instance while it's in +// flight, runTransform must re-check readiness on resume rather than +// dispatching to a nulled/destroyed engine handle. +describe("runTransform re-checks readiness after async input pre-buffering (round 12 Task 4)", () => { + it("throws a synchronous DataWeaveError if cleanup() runs during createChunkReader's await, instead of resolving an error envelope", async () => { + const dw = new DataWeave(); + dw.initialize(); + + // An async input whose iterator blocks until released, so cleanup() can + // run while createChunkReader is still pre-buffering it. + let release!: () => void; + const gate = new Promise((r) => { release = r; }); + async function* slowInput(): AsyncGenerator { + await gate; + yield Buffer.from("[1,2,3]"); + } + + const gen = dw.runTransform("%dw 2.0\noutput application/json\n---\npayload", slowInput(), { + mimeType: "application/json", + }); + + // Start driving the generator; it suspends awaiting createChunkReader -> + // slowInput's gate. + const firstNext = gen.next(); + // Clean up while the input is still pre-buffering. + await dw.cleanup(); + // Release the gate so createChunkReader's await resolves; the readiness + // re-check must now throw synchronously rather than proceeding to a + // nulled engine handle. + release(); + + await expect(firstNext).rejects.toBeInstanceOf(DataWeaveError); + }); +}); + +// Round 12, Task 6: the exported module-level cleanup() nulls globalInstance +// synchronously, then awaits instance.cleanup(). A second overlapping +// module-level cleanup() call must coalesce onto the SAME in-flight drain +// rather than seeing globalInstance already nulled and resolving immediately +// -- before the first call's native teardown actually finishes. +describe("module-level cleanup() coalescing (round 12 Task 6)", () => { + it("module-level cleanup() coalesces overlapping calls (round 12 #5)", async () => { + // Create the singleton. + expect(run("%dw 2.0\noutput application/json\n---\n1 + 1").success).toBe(true); + + let firstSettled = false; + const p1 = cleanup().then(() => { firstSettled = true; }); + // Second call overlaps the first's in-flight drain. + const p2 = cleanup(); + // The coalesced second call must not resolve before the first's drain does. + await p2; + expect(firstSettled).toBe(true); + await p1; + + // A subsequent run lazily revives the singleton (no wedged state). + expect(run("%dw 2.0\noutput application/json\n---\n2 + 2").success).toBe(true); + await cleanup(); + }); +}); + +// Final review round 12 #1: Task 6's coalescing guard (`if (cleanupPromise) +// return cleanupPromise;`) is unconditional, so a caller that revives the +// singleton (via run()) while an OLDER drain is still in flight gets the OLD +// drain's promise handed back by the newer cleanup() call -- the freshly +// revived instance is never hooked up to any doCleanup()/ffi.cleanup() call +// and its native ref leaks for the rest of the process. Pinned via the same +// ref-count proxy as the "napi_cleanup refactor" test above: after both +// cleanup() calls settle, the isolate's ref count must have actually returned +// to zero (not be left at 1 by a leaked, unrevived-then-abandoned instance). +describe("module-level cleanup() does not orphan a revived singleton (final review round 12 #1)", () => { + it("cleanup() started during an in-flight drain cleans the CURRENT (revived) singleton, not the stale one", async () => { + // (a) Create the singleton (instance A). + expect(run("%dw 2.0\noutput application/json\n---\n1 + 1").success).toBe(true); + + // (b) Start draining A WITHOUT awaiting. + const p1 = cleanup(); + + // (c) Revive a FRESH singleton (instance B) while A's drain is in flight. + expect(run("%dw 2.0\noutput application/json\n---\n2 + 2").success).toBe(true); + + // (d) Call cleanup() again. Under the bug this returns p1 verbatim, + // leaving B's native ref uncleaned once both promises settle. + const p2 = cleanup(); + await Promise.all([p1, p2]); + + // (e) Prove B was actually torn down via the isolate's ref count, the same + // technique as "napi_cleanup refactor preserves last-release teardown" + // above: do one extra balanced initialize()/cleanup() pair. If the ref + // count was already back to zero (both A and B cleaned), this nets back + // to zero and a subsequent raw engine call observes "not initialized". If + // B's ref instead leaked, the ref count is already >=1 going into this + // balanced pair, so it nets to >=1 afterward and the isolate stays alive + // -- the subsequent call would NOT report "not initialized". + ffi.initialize(findLibrary()); + const h = ffi.createEngine(); + const envelope = JSON.parse( + ffi.runScriptEngine(h, "%dw 2.0\noutput application/json\n---\n5 + 5", buildInputsJson({})) + ); + expect(envelope.success).toBe(true); + expect(JSON.parse(Buffer.from(envelope.result, "base64").toString("utf-8"))).toBe(10); + ffi.destroyEngine(h); + await ffi.cleanup(); // Balances the initialize() just above, ONLY. + + expect(() => + ffi.runScriptEngine(Number.MAX_SAFE_INTEGER, "%dw 2.0\noutput application/json\n---\n1", buildInputsJson({})) + ).toThrow(/not initialized/i); + + // The singleton revives cleanly again afterward -- no wedged module state. + expect(run("%dw 2.0\noutput application/json\n---\n3 + 3").success).toBe(true); + await cleanup(); + }); +}); diff --git a/native-lib/node/tests/integration/malformed-inputs.test.ts b/native-lib/node/tests/integration/malformed-inputs.test.ts new file mode 100644 index 00000000..2a6194b0 --- /dev/null +++ b/native-lib/node/tests/integration/malformed-inputs.test.ts @@ -0,0 +1,110 @@ +import { describe, it, expect, afterEach } from "vitest"; +import * as ffi from "../../src/ffi"; +import { findLibrary, buildInputsJson } from "../../src/utils"; + +// Round-7 finding #2 (whole-class sweep): every FFI-facing entrypoint must +// check the status of each napi_get_value_* conversion and throw before using +// the converted value. Pre-fix, non-string script/inputs left *_len +// uninitialized before malloc(len+1) and the buffer write, and destroyEngine +// used an indeterminate handle64 from an ignored napi_get_value_int64. +// +// Driven through the raw `ffi` boundary (the DataWeave TS class always passes +// well-typed values), so these calls exercise the C conversion checks directly. +// The addon globals are process-wide C statics -- balance every initialize() +// with a cleanup() so this file does not leak a ref-count into siblings. +// +// Real addon, no mocking. +describe("malformed raw-ffi inputs throw (round 7 #2)", () => { + afterEach(async () => { + await ffi.cleanup(); + }); + + it("destroyEngine throws on a non-integer handle", () => { + ffi.initialize(findLibrary()); + expect(() => ffi.destroyEngine({} as unknown as number)).toThrow(); + }); + + it("runScriptEngine throws on non-string script/inputs", () => { + ffi.initialize(findLibrary()); + const handle = ffi.createEngine(); + expect(() => + ffi.runScriptEngine(handle, {} as unknown as string, buildInputsJson({})) + ).toThrow(); + expect(() => + ffi.runScriptEngine(handle, "%dw 2.0\n---\n1", {} as unknown as string) + ).toThrow(); + ffi.destroyEngine(handle); + }); + + it("runScriptStreamingEngine throws on non-string script/inputs", () => { + ffi.initialize(findLibrary()); + const handle = ffi.createEngine(); + expect(() => + ffi.runScriptStreamingEngine( + handle, + {} as unknown as string, + buildInputsJson({}), + () => {} + ) + ).toThrow(); + ffi.destroyEngine(handle); + }); + + it("runScriptStreamingEngine throws on non-string inputsJson", () => { + ffi.initialize(findLibrary()); + const handle = ffi.createEngine(); + expect(() => + ffi.runScriptStreamingEngine( + handle, + "%dw 2.0\noutput application/json\n---\n[1,2,3]", + {} as unknown as string, + () => {} + ) + ).toThrow(); + ffi.destroyEngine(handle); + }); + + it("runScriptTransformEngine throws on non-string script", () => { + ffi.initialize(findLibrary()); + const handle = ffi.createEngine(); + expect(() => + ffi.runScriptTransformEngine( + handle, + {} as unknown as string, + "{}", + "payload", + "application/json", + null, + () => null, + () => {} + ) + ).toThrow(); + ffi.destroyEngine(handle); + }); + + // The transform entrypoint converts four string args (script already covered + // above): inputsJson, inputName, inputMimeType, and a non-null inputCharset. + // A dropped napi_get_value_string check on any of them must throw (review #9 #6). + it.each([ + { name: "inputsJson", script: "%dw 2.0\noutput application/json\n---\npayload", inputsJson: {} as unknown as string, inputName: "payload", mimeType: "application/json", charset: null as string | null }, + { name: "inputName", script: "%dw 2.0\noutput application/json\n---\npayload", inputsJson: "{}", inputName: {} as unknown as string, mimeType: "application/json", charset: null as string | null }, + { name: "inputMimeType", script: "%dw 2.0\noutput application/json\n---\npayload", inputsJson: "{}", inputName: "payload", mimeType: {} as unknown as string, charset: null as string | null }, + { name: "non-null inputCharset", script: "%dw 2.0\noutput application/json\n---\npayload", inputsJson: "{}", inputName: "payload", mimeType: "application/json", charset: {} as unknown as string }, + ])("runScriptTransformEngine throws on non-string $name", ({ script, inputsJson, inputName, mimeType, charset }) => { + ffi.initialize(findLibrary()); + const handle = ffi.createEngine(); + expect(() => + ffi.runScriptTransformEngine( + handle, + script, + inputsJson, + inputName, + mimeType, + charset, + () => null, + () => {} + ) + ).toThrow(); + ffi.destroyEngine(handle); + }); +}); diff --git a/native-lib/node/tests/integration/run-admission.test.ts b/native-lib/node/tests/integration/run-admission.test.ts new file mode 100644 index 00000000..88dea6a2 --- /dev/null +++ b/native-lib/node/tests/integration/run-admission.test.ts @@ -0,0 +1,93 @@ +import { describe, it, expect } from "vitest"; +import * as ffi from "../../src/ffi"; +import { findLibrary, buildInputsJson } from "../../src/utils"; + +// Round-7 finding #1: the synchronous napi_run_script_engine touched the +// isolate (fn_attach_thread -> fn_run_script_engine -> fn_detach_thread) with +// only a top-of-function !g_initialized fast-path and NO g_active_ops +// reservation under g_mutex. A second Worker's last cleanup() (napi_cleanup +// Case 4) could observe g_active_ops == 0 and tear down g_isolate while this +// op was attaching/executing -- a use-after-free. +// +// The genuine cross-Worker TOCTOU is not reliably forceable from single-thread +// JS (same limitation the round-6 #2 admission-during-teardown test documents: +// re-init would trigger the adoption path and cancel the pending teardown +// before the admission check runs). What we CAN assert deterministically is +// the admission-rejection path the fix introduces: once a teardown is pending +// (g_teardown_state != TEARDOWN_NONE), a freshly started run() is rejected with +// a synchronous throw rather than attaching to an isolate a concurrent teardown +// could pull out from under it. The C-level reasoning -- check-and-reserve is +// now one atomic critical section on the run() path -- is what covers the race +// itself. +// +// We drive the addon through the raw `ffi` module (not the module-level +// singleton) so the second op runs against the SAME still-live handle/isolate +// with no intervening ffi.initialize() call to trigger adoption. Calling +// ffi.cleanup() directly triggers napi_cleanup Case 5 and sets +// g_teardown_state = TEARDOWN_PENDING_WAIT synchronously, before its Promise is +// returned; the immediately-following ffi.runScriptEngine re-enters native code +// synchronously on the same callstack and deterministically observes it. +// +// Real addon, no mocking. +describe("run() admission rejected while teardown pending (round 7 #1)", () => { + it("a synchronous run() started during pending teardown throws, not attach to a dead isolate", async () => { + ffi.initialize(findLibrary()); + const handle = ffi.createEngine(); + + // Keep one op in flight so the ref release becomes Case 5 (pending + // teardown) rather than Case 4 (immediate teardown): use a transform whose + // read callback triggers cleanup() and then attempts a run() on the same + // handle, all on the same synchronous callstack. + let cleanupPromise: Promise | undefined; + let runErr: unknown; + let ran = false; + + let firstRead = true; + const readCb = (_bufSize: number): Buffer | null => { + if (firstRead) { + firstRead = false; + // Case 5: last ref release with g_active_ops > 0 -> TEARDOWN_PENDING_WAIT, + // set synchronously before this returns. Not awaited. + cleanupPromise = ffi.cleanup(); + // Synchronous run() on the same still-live handle while teardown is + // pending. Fixed code rejects admission with a synchronous throw + // (g_teardown_state != TEARDOWN_NONE). Must be caught here -- it is a + // synchronous throw, not a rejected promise. Do not let it escape the + // native read-callback body. + try { + ffi.runScriptEngine( + handle, + "%dw 2.0\noutput application/json\n---\n1 + 1", + buildInputsJson({}) + ); + ran = true; + } catch (e) { + runErr = e; + } + return Buffer.from("[1,2,3]"); + } + return null; + }; + + const writeCb = (_chunk: Buffer) => {}; + + const resultRaw = await ffi.runScriptTransformEngine( + handle, + "output application/json\n---\npayload", + "{}", + "payload", + "application/json", + null, + readCb, + writeCb + ); + const result = JSON.parse(resultRaw); + expect(result.success).toBe(true); + + await cleanupPromise; + + // run() started while teardown was pending must have been rejected. + expect(runErr).toBeTruthy(); + expect(ran).toBe(false); + }, 20000); +}); diff --git a/native-lib/node/tests/integration/teardown-deadlock.test.ts b/native-lib/node/tests/integration/teardown-deadlock.test.ts new file mode 100644 index 00000000..efa44cec --- /dev/null +++ b/native-lib/node/tests/integration/teardown-deadlock.test.ts @@ -0,0 +1,120 @@ +import { describe, it, expect } from "vitest"; +import { run, runTransform, cleanup } from "../../src/dataweave"; + +// Regression test for W-23692110 round 5 (Task 1 fix in native-lib/node/src/addon.c). +// +// Bug: napi_initialize used to block the JS thread forever whenever it ran +// while a teardown was pending on the shared native isolate and a +// streaming/transform op was still active elsewhere -- because draining that +// active op can need the very same JS thread napi_initialize was blocking. +// The fix makes napi_initialize adopt the still-live isolate instead of +// waiting, in the window before the teardown waiter thread commits to +// physical teardown. +// +// This loads the REAL native addon (no `vi.mock` of ffi) -- the deadlock is +// entirely in C and cannot be reproduced at the mocked-ffi layer. +// +// Why runTransform (not runStreaming) drives this repro: runStreaming's +// output-chunk delivery uses an unbounded napi_threadsafe_function queue, and +// g_active_ops is decremented on the background worker thread right after it +// detaches from the isolate -- independent of whether the JS event loop ever +// turns. So a blocked JS thread does NOT stop a runStreaming() op from +// draining; there is no genuine circular wait on that path (verified +// empirically: the brief's originally-suggested runStreaming shape resolves +// promptly even against pre-Task-1 addon.c, because an earlier round already +// moved that decrement off the JS thread -- see commit ac8d520). +// +// runTransform's INPUT side is different: transform_read_cb (addon.c) calls +// napi_call_threadsafe_function(w->read_tsfn, &req, napi_tsfn_blocking) and +// then genuinely blocks the background worker thread on a condition variable +// until call_js_read runs on the JS thread and signals it. That JS-thread +// callback synchronously invokes our JS read callback (a plain +// Iterable consumed by a sync generator) via napi_call_function -- +// so firing cleanup() and a concurrent run() from *inside* that generator +// deterministically executes them while the background worker is attached +// and blocked waiting for this exact call to return. No timing assumptions +// (no setTimeout/microtask races) are needed: the call graph itself +// guarantees the ordering "worker attached and mid-read" -> "cleanup() +// fired" -> "run() fired", all on the JS thread, before the generator call +// returns and the worker can proceed. +describe("re-init during pending teardown (W-23692110, round 5 P1)", () => { + // On the UNFIXED addon.c this deadlocks for real: the JS thread never + // returns from run()'s napi_initialize (blocked waiting for g_active_ops to + // drain), so the background transform worker -- itself blocked waiting for + // the JS thread to service its read callback -- can never proceed either. + // Vitest kills the test at the timeout below, a bounded/deterministic red. + // On the fixed code, napi_initialize adopts the still-live isolate and + // run() returns promptly, letting everything drain normally. + it( + "module-level cleanup() during an active transform read does not deadlock a concurrent run()", + async () => { + let fired = false; + let cleanupPromise: Promise | undefined; + let runResult: ReturnType | undefined; + let runError: unknown; + + // Large enough that, at the moment of the very first read pull, the + // vast majority of reads (and thus the transform op) are still + // genuinely ahead -- not a timing-sensitive assumption, since the + // trigger below fires unconditionally on the first pull regardless of + // how many total reads there are. + const totalReads = 200000; + + function* input(): Generator { + for (let i = 0; i < totalReads; i++) { + if (!fired) { + fired = true; + // We are executing synchronously inside the native read + // callback (call_js_read in addon.c), on the JS thread, while + // the background transform worker thread is blocked inside + // transform_read_cb waiting for this exact call to return. + // Deliberately do NOT await cleanup() here, and do NOT let an + // assertion throw from inside this generator -- a thrown + // exception here would be caught by the native read-callback + // wrapper and reinterpreted as a read error, silently masking a + // real assertion failure instead of surfacing it as a test + // failure. Capture results and assert on them after the + // generator (and the transform) have fully drained. + cleanupPromise = cleanup(); + try { + runResult = run('%dw 2.0\noutput application/json\n---\n1 + 1'); + } catch (e) { + runError = e; + } + } + yield Buffer.from("x"); + } + } + + const gen = runTransform( + "output application/octet-stream\n---\npayload", + input(), + { mimeType: "application/octet-stream" } + ); + + // Drain the whole transform. On unfixed code, execution never reaches + // here: the trigger inside input() already froze the JS thread + // forever before the first read even returns. + let result = await gen.next(); + while (!result.done) { + result = await gen.next(); + } + + expect(fired).toBe(true); + expect(runError).toBeUndefined(); + expect(runResult?.success).toBe(true); + expect(JSON.parse(runResult!.getString()!)).toBe(2); + expect(result.value.success).toBe(true); + + // Let both the deferred teardown/cleanup and this test settle cleanly. + // This is essential: the process shares one native isolate across all + // integration test files, so leaving an unresolved cleanup here would + // perturb sibling test files. + await cleanupPromise; + // Idempotent final cleanup: a no-op if the singleton is already fully + // released, leaving the module in a clean state for subsequent tests. + await cleanup(); + }, + 20000 + ); +}); diff --git a/native-lib/node/tests/integration/worker-lifecycle.test.ts b/native-lib/node/tests/integration/worker-lifecycle.test.ts new file mode 100644 index 00000000..75772ca0 --- /dev/null +++ b/native-lib/node/tests/integration/worker-lifecycle.test.ts @@ -0,0 +1,337 @@ +import { describe, it, expect, afterAll } from "vitest"; +import { Worker } from "node:worker_threads"; +import { join } from "node:path"; +import * as ffi from "../../src/ffi"; +import { findLibrary, buildInputsJson } from "../../src/utils"; + +// W-23692110 round 12 #9: real worker_threads coverage for the documented +// per-Worker engine model (README "Custom module resolvers and Worker threads"). +// +// Workers cannot execute the TS sources (npm test runs vitest with no build for +// worker code, and a Worker spawns a fresh Node runtime), so each worker body is +// an inline JS string (eval:true) that require()s the BUILT addon directly -- +// the same raw-addon boundary engine-handle-contract.test.ts drives. addonPath +// and the dwlib path are resolved on the main thread and passed via workerData. +// +// Determinism posture: exact cross-thread teardown interleavings are NOT +// deterministically forceable (best-effort, matching rounds 5-11). The +// deterministic assertions here are: resolver-backed/less engines produce +// correct output inside a Worker, and after N Worker create/exit-without- +// cleanup() cycles the main thread still initializes/runs and the final +// teardown is clean (the round-12 #2 behavioral proof). + +const ADDON_PATH = join(__dirname, "..", "..", "build", "Release", "dwlib_addon.node"); +const LIB_PATH = findLibrary(); + +// Runs one Worker to completion and returns its posted message. `mode` selects +// resolver-backed vs resolver-less and whether the Worker cleans up or abandons. +function runWorker(opts: { + mode: "resolver" | "plain"; + cleanup: boolean; + script: string; +}): Promise<{ ok: boolean; output?: string; error?: string }> { + const body = ` + const { parentPort, workerData } = require('node:worker_threads'); + (async () => { + const addon = require(workerData.addonPath); + addon.initialize(workerData.libPath); + let handle; + if (workerData.mode === 'resolver') { + const resolver = (modulePath) => + modulePath === 'org/test/w.dwl' + ? '%dw 2.0\\nfun greet(n) = "W:" ++ n' + : null; + handle = addon.createEngineWithResolver(resolver); + } else { + handle = addon.createEngine(); + } + let msg; + try { + const raw = addon.runScriptEngine(handle, workerData.script, '{}'); + const parsed = JSON.parse(raw); + if (parsed.success === false) { + msg = { ok: false, error: parsed.error }; + } else { + // Non-streaming engine result carries base64 'result'; decode it. + const out = parsed.result ? Buffer.from(parsed.result, 'base64').toString('utf-8') : ''; + msg = { ok: true, output: out }; + } + } catch (e) { + msg = { ok: false, error: String(e) }; + } + if (workerData.cleanup) { + let destroyErr; + try { + addon.destroyEngine(handle); + } catch (e) { + destroyErr = e; // preserve; do NOT let cleanup() mask a broken destroy + } finally { + await addon.cleanup(); + } + if (destroyErr) msg = { ok: false, error: 'destroyEngine failed: ' + String(destroyErr) }; + } + parentPort.postMessage(msg); + // For the abandon variant we deliberately return WITHOUT cleanup so the + // env cleanup hook fires as the Worker env tears down. + })().catch((e) => { parentPort.postMessage({ ok: false, error: String(e) }); }); + `; + return new Promise((resolve, reject) => { + const w = new Worker(body, { + eval: true, + workerData: { addonPath: ADDON_PATH, libPath: LIB_PATH, mode: opts.mode, cleanup: opts.cleanup, script: opts.script }, + }); + let msg: { ok: boolean; output?: string; error?: string } | undefined; + w.once("message", (m) => { msg = m; }); + w.once("error", reject); + // Resolve only on a CLEAN exit that posted a result. A Worker can post a + // success message and THEN exit nonzero (e.g. an env-cleanup-hook failure + // during teardown) -- resolving on the message alone would hide that. So + // wait for exit: reject every nonzero code, and treat a zero exit with no + // posted message as its own diagnosable failure (round-14 #5). + w.once("exit", (code) => { + if (code !== 0) { + reject(new Error("Worker exited with code " + code + (msg ? "" : " and posted no message"))); + } else if (msg === undefined) { + reject(new Error("Worker exited 0 without posting a result")); + } else { + resolve(msg); + } + }); + }); +} + +describe("worker_threads engine lifecycle (round 12 #9)", () => { + afterAll(async () => { + // Final main-thread balancing cleanup so this file does not perturb sibling + // integration files sharing the vitest worker process. + await ffi.cleanup(); + }); + + it("a resolver-backed engine in a Worker resolves the Worker's own module", async () => { + const script = "%dw 2.0\nimport org::test::w\noutput application/json\n---\nw::greet(\"X\")"; + const msg = await runWorker({ mode: "resolver", cleanup: true, script }); + expect(msg.ok).toBe(true); + expect(JSON.parse(msg.output!)).toBe("W:X"); + }); + + it("a resolver-less engine in a Worker runs a plain script", async () => { + const script = "%dw 2.0\noutput application/json\n---\n6 * 7"; + const msg = await runWorker({ mode: "plain", cleanup: true, script }); + expect(msg.ok).toBe(true); + expect(JSON.parse(msg.output!)).toBe(42); + }); + + it("built-in modules resolve in a resolver-backed engine inside a Worker", async () => { + const script = + "%dw 2.0\nimport dw::core::Strings\noutput application/json\n---\nStrings::capitalize(\"hello\")"; + const msg = await runWorker({ mode: "resolver", cleanup: true, script }); + expect(msg.ok).toBe(true); + expect(JSON.parse(msg.output!)).toBe("Hello"); + }); + + it("N Workers that exit WITHOUT cleanup() do not wedge the isolate; main thread stays healthy (round 12 #2)", async () => { + const CYCLES = 5; + for (let i = 0; i < CYCLES; i++) { + const msg = await runWorker({ + mode: "resolver", + cleanup: false, // exit without cleanup -> env cleanup hook fires + script: "%dw 2.0\noutput application/json\n---\n" + i, + }); + expect(msg.ok).toBe(true); + } + // After all those abandoned Workers, the main thread must still initialize + // and run. Pre-fix, each abandoned Worker leaked its init reference and the + // isolate never returned to zero; the assertion here is behavioral (the + // process is not wedged and cleanup still tears down cleanly at afterAll). + ffi.initialize(LIB_PATH); + const h = ffi.createEngine(); + const envelope = JSON.parse( + ffi.runScriptEngine(h, "%dw 2.0\noutput application/json\n---\n1 + 1", buildInputsJson({})) + ); + expect(envelope.success).toBe(true); + expect(JSON.parse(Buffer.from(envelope.result, "base64").toString("utf-8"))).toBe(2); + ffi.destroyEngine(h); + await ffi.cleanup(); + // Prove this test INDEPENDENTLY that no abandoned Worker leaked its init + // reference: after the main thread balances its own reference to zero, a raw + // op must observe "not initialized". A leaked Worker reference would keep + // g_ref_count >= 1 here, so the isolate would still be live and this would + // NOT throw (review #9 #2). + expect(() => + ffi.runScriptEngine(Number.MAX_SAFE_INTEGER, "%dw 2.0\noutput application/json\n---\n1", buildInputsJson({})) + ).toThrow(/not initialized/i); + }); + + it("Worker.terminate() mid-life leaves the main thread able to initialize and run", async () => { + const body = ` + const { parentPort, workerData } = require('node:worker_threads'); + const addon = require(workerData.addonPath); + addon.initialize(workerData.libPath); + addon.createEngineWithResolver((p) => null); + // Signal readiness only once the engine is actually live, so the parent + // terminates a worker that genuinely has a live engine rather than + // racing a fixed sleep against initialize()/createEngineWithResolver on + // a possibly-loaded box (final review round 12 #3). + parentPort.postMessage('ready'); + // Spin so the parent can terminate() us mid-life (no message posted). + setInterval(() => {}, 10); + `; + const w = new Worker(body, { + eval: true, + workerData: { addonPath: ADDON_PATH, libPath: LIB_PATH }, + }); + // A throw in the worker body (e.g. a bad addon path) must fail this test + // cleanly rather than crash the vitest process -- the ad hoc Worker here, + // unlike runWorker() above, previously had no error listener wired up + // (final review round 12 #2). + const workerError = new Promise((_, reject) => w.once("error", reject)); + // Avoid an unhandled-rejection warning if "error" fires (or would fire) + // after the race below has already settled via the "ready" path. + workerError.catch(() => {}); + // Wait for the worker to report the engine is live, racing against a + // generous timeout so a slow box doesn't false-fail this test, then + // terminate abruptly. + const ready = new Promise((resolve) => w.once("message", (m) => { if (m === "ready") resolve(); })); + await Promise.race([ + ready, + workerError, + new Promise((_, reject) => setTimeout(() => reject(new Error("worker did not signal ready in time")), 10000)), + ]); + await w.terminate(); + + ffi.initialize(LIB_PATH); + const h = ffi.createEngine(); + const envelope = JSON.parse( + ffi.runScriptEngine(h, "%dw 2.0\noutput application/json\n---\n3 + 4", buildInputsJson({})) + ); + expect(envelope.success).toBe(true); + expect(JSON.parse(Buffer.from(envelope.result, "base64").toString("utf-8"))).toBe(7); + ffi.destroyEngine(h); + await ffi.cleanup(); + // Prove INDEPENDENTLY that the terminated Worker's engine reference was + // released: after the main thread balances its own reference, a raw op must + // observe "not initialized" (g_ref_count == 0). A leaked reference from the + // terminated Worker would leave the isolate live and this would NOT throw + // (review #9 #2). + expect(() => + ffi.runScriptEngine(Number.MAX_SAFE_INTEGER, "%dw 2.0\noutput application/json\n---\n1", buildInputsJson({})) + ).toThrow(/not initialized/i); + }); + + it("a Worker that inits once + creates N engines + exits without cleanup() does NOT tear down the isolate under a live main engine (round 13 #5)", async () => { + // This is the cross-env regression the round-13 smoke tests could not pin + // (env-init-ownership.test.ts is single-env). It fails RED on the round-12 + // implementation: the Worker's env death fired N per-engine init-reference + // releases against the ONE reference the Worker owned, driving g_ref_count to + // zero and tearing the shared isolate down under the live main engine -> the + // main engine's run below would fail (isolate gone) or the process wedges. On + // round-13+ each abandoned env releases exactly one reference regardless of + // engine count, so the main engine survives. + const N = 3; + + let hMain: number | null = null; + let bodySucceeded = false; + try { + // 1. Main thread: initialize and keep a live engine. + ffi.initialize(LIB_PATH); + hMain = ffi.createEngine(); + const first = JSON.parse( + ffi.runScriptEngine(hMain, "%dw 2.0\noutput application/json\n---\n6 * 7", buildInputsJson({})) + ); + expect(first.success).toBe(true); + expect(JSON.parse(Buffer.from(first.result, "base64").toString("utf-8"))).toBe(42); + + // 2. Worker: initialize ONCE, create N engines, run one, exit WITHOUT cleanup. + const workerBody = ` + const { parentPort, workerData } = require('node:worker_threads'); + (async () => { + const addon = require(workerData.addonPath); + addon.initialize(workerData.libPath); // ONE init reference for this env + const handles = []; + for (let i = 0; i < workerData.n; i++) handles.push(addon.createEngine()); + const raw = addon.runScriptEngine(handles[0], workerData.script, '{}'); + const parsed = JSON.parse(raw); + parentPort.postMessage({ ok: parsed.success !== false, count: handles.length }); + // Return WITHOUT destroyEngine/cleanup: the env dies with N engines under + // one init reference -> env_init_cleanup releases exactly ONE reference. + })().catch((e) => { parentPort.postMessage({ ok: false, error: String(e) }); }); + `; + const workerMsg = await new Promise<{ ok: boolean; count?: number; error?: string }>((resolve, reject) => { + const w = new Worker(workerBody, { + eval: true, + workerData: { + addonPath: ADDON_PATH, + libPath: LIB_PATH, + n: N, + script: "%dw 2.0\noutput application/json\n---\n1 + 1", + }, + }); + let msg: { ok: boolean; count?: number; error?: string } | undefined; + w.once("message", (m) => { msg = m; }); + w.once("error", reject); + // Wait for EXIT (not just message) so the Worker env's death hooks + // (env_init_cleanup) have run before we assert the main engine survived. + w.once("exit", (code) => { + if (code !== 0) reject(new Error("Worker exited with code " + code + (msg ? "" : " and posted no message"))); + else if (msg === undefined) reject(new Error("Worker exited 0 without posting a result")); + else resolve(msg); + }); + }); + expect(workerMsg.ok).toBe(true); + expect(workerMsg.count).toBe(N); + + // 3. The Worker abandoned N engines under one init reference and its env + // died. The main engine's reference must be intact and the isolate live. + const second = JSON.parse( + ffi.runScriptEngine(hMain, "%dw 2.0\noutput application/json\n---\n1 + 1", buildInputsJson({})) + ); + expect(second.success).toBe(true); + expect(JSON.parse(Buffer.from(second.result, "base64").toString("utf-8"))).toBe(2); + + // 4. Balance the main reference and prove the count reached exactly zero + // (no leak, no over-release): a raw op now throws "not initialized". + ffi.destroyEngine(hMain); + hMain = null; // destroyed; finally must not double-destroy + await ffi.cleanup(); + expect(() => + ffi.runScriptEngine(Number.MAX_SAFE_INTEGER, "%dw 2.0\noutput application/json\n---\n1", buildInputsJson({})) + ).toThrow(/not initialized/i); + bodySucceeded = true; + } finally { + // Balance global native state even if a Worker/assertion above threw, so + // this test cannot strand a live isolate + held reference for sibling + // integration tests (review #6 #7). Suppress a balancing-cleanup error + // ONLY when the body already failed (so the original, more actionable + // failure keeps propagating). When the body SUCCEEDED, a cleanup failure + // is itself a real lifecycle regression and must fail the test rather than + // be silently discarded (review #7 #7). + let destroyErr: unknown; + try { + if (hMain !== null) ffi.destroyEngine(hMain); + } catch (e) { + // Capture but do not early-exit: the global init reference must still be + // released below, or it contaminates sibling integration tests (review #8 + // #3), matching the production cleanup path that releases even when + // destroyEngine() throws. + destroyErr = e; + } + let cleanupErr: unknown; + try { + await ffi.cleanup(); + } catch (e) { + // Always attempt cleanup (never skipped by a destroyEngine throw), but + // capture its failure rather than letting it propagate unconditionally -- + // an already-failing body must keep its original, more actionable error. + cleanupErr = e; + } + // Surface a balancing failure (destroy or cleanup) ONLY when the body + // succeeded; when the body already failed, both are suppressed so the + // original failure keeps propagating (review #7 #7, extended to the + // destroyEngine() throw + cleanup() throw double-fault case). + if (bodySucceeded) { + if (destroyErr !== undefined) throw destroyErr; + if (cleanupErr !== undefined) throw cleanupErr; + } + } + }, 20000); +}); diff --git a/native-lib/node/tests/unit/dataweave-initialize.test.ts b/native-lib/node/tests/unit/dataweave-initialize.test.ts new file mode 100644 index 00000000..913ffcec --- /dev/null +++ b/native-lib/node/tests/unit/dataweave-initialize.test.ts @@ -0,0 +1,383 @@ +import { describe, it, expect, vi, beforeEach } from "vitest"; + +// Pure-logic test of DataWeave.initialize()'s lifecycle/error handling, with +// the native addon mocked out entirely -- no dwlib required (see the "unit" +// project in vitest.config.ts). This covers a ref-count leak that is only +// observable in the sequencing of calls into ffi.ts, not in any externally +// visible native state, so a real end-to-end native failure isn't a +// practical way to assert on it (see task-4-report.md's fix report for why). +vi.mock("../../src/ffi", () => ({ + initialize: vi.fn(), + createEngine: vi.fn(), + createEngineWithResolver: vi.fn(), + destroyEngine: vi.fn(), + runScriptEngine: vi.fn(), + runScriptStreamingEngine: vi.fn(), + runScriptTransformEngine: vi.fn(), + cleanup: vi.fn(), +})); + +import * as ffi from "../../src/ffi"; +import { DataWeave, run, cleanup } from "../../src/dataweave"; +import { DataWeaveError } from "../../src/errors"; + +describe("DataWeave.initialize() native ref-count safety", () => { + beforeEach(() => { + vi.mocked(ffi.initialize).mockReset(); + vi.mocked(ffi.createEngine).mockReset(); + vi.mocked(ffi.createEngineWithResolver).mockReset(); + vi.mocked(ffi.destroyEngine).mockReset(); + vi.mocked(ffi.cleanup).mockReset(); + }); + + it("releases the native library ref-count if engine creation fails after ffi.initialize() succeeded", () => { + vi.mocked(ffi.initialize).mockImplementation(() => {}); + vi.mocked(ffi.createEngineWithResolver).mockImplementation(() => { + throw new Error("native engine creation boom"); + }); + + const dw = new DataWeave({ libPath: "mock-lib-path", resolveModule: () => null }); + + expect(() => dw.initialize()).toThrow(DataWeaveError); + + // ffi.initialize() already succeeded, incrementing the native library's + // ref count. Since `initialized` never became true, cleanup()'s + // early-return guard means nothing else would ever call ffi.cleanup() -- + // initialize()'s own catch block must have released it. + expect(ffi.cleanup).toHaveBeenCalledTimes(1); + }); + + it("does not call ffi.cleanup() when ffi.initialize() itself is what fails", () => { + vi.mocked(ffi.initialize).mockImplementation(() => { + throw new Error("library not found"); + }); + + const dw = new DataWeave({ libPath: "mock-lib-path" }); + + expect(() => dw.initialize()).toThrow(DataWeaveError); + + // No ref count was ever acquired, so there is nothing to release. + expect(ffi.cleanup).not.toHaveBeenCalled(); + }); + + it("leaves engineHandle unset and the instance cleanly re-initializable after the failed attempt's rollback settles", async () => { + vi.mocked(ffi.initialize).mockImplementation(() => {}); + vi.mocked(ffi.createEngine) + .mockImplementationOnce(() => { + throw new Error("transient native failure"); + }) + .mockImplementationOnce(() => 42); + + const dw = new DataWeave({ libPath: "mock-lib-path" }); + + expect(() => dw.initialize()).toThrow(DataWeaveError); + expect(ffi.cleanup).toHaveBeenCalledTimes(1); + + // The rollback release is modeled as pending state (review #7 #3): until it + // settles the instance is "cleaning-up" and re-init is deliberately rejected + // rather than racing the in-flight release. Let the rollback settle first. + await new Promise((r) => setImmediate(r)); + + // A later initialize() call (e.g. once the transient failure clears) + // must succeed cleanly -- the failed attempt must not have left the + // instance permanently "half-initialized" (this.initialized stuck true + // without an engine handle, or vice versa). + vi.mocked(ffi.cleanup).mockClear(); + dw.initialize(); + expect(ffi.createEngine).toHaveBeenCalledTimes(2); + + await dw.cleanup(); + expect(ffi.destroyEngine).toHaveBeenCalledWith(42); + expect(ffi.cleanup).toHaveBeenCalledTimes(1); + }); + + it("does not call ffi.cleanup() from initialize() on the successful path", () => { + vi.mocked(ffi.initialize).mockImplementation(() => {}); + vi.mocked(ffi.createEngine).mockImplementation(() => 7); + + const dw = new DataWeave({ libPath: "mock-lib-path" }); + dw.initialize(); + + expect(ffi.cleanup).not.toHaveBeenCalled(); + + dw.cleanup(); + expect(ffi.destroyEngine).toHaveBeenCalledWith(7); + expect(ffi.cleanup).toHaveBeenCalledTimes(1); + }); + + it("still clears `initialized` when ffi.cleanup() rejects, so the instance is re-initializable", async () => { + vi.mocked(ffi.initialize).mockImplementation(() => {}); + vi.mocked(ffi.createEngine).mockImplementation(() => 7); + vi.mocked(ffi.cleanup).mockRejectedValueOnce(new Error("native cleanup boom")); + + const dw = new DataWeave({ libPath: "mock-lib-path" }); + dw.initialize(); + + await expect(dw.cleanup()).rejects.toThrow("native cleanup boom"); + + // Even though ffi.cleanup() rejected, the engine handle was already + // destroyed and nulled -- `initialized` must not stay stuck `true`, or a + // later initialize() call becomes a permanent no-op (the early-return + // guard `if (this.initialized) return;`) and the instance is stranded + // with a null engineHandle. + vi.mocked(ffi.initialize).mockClear(); + vi.mocked(ffi.createEngine).mockClear(); + vi.mocked(ffi.createEngine).mockImplementation(() => 9); + + dw.initialize(); + + expect(ffi.initialize).toHaveBeenCalledTimes(1); + expect(ffi.createEngine).toHaveBeenCalledTimes(1); + }); + + it("coalesces concurrent cleanup() calls into a single native teardown", async () => { + vi.mocked(ffi.createEngine).mockReturnValue(1); + let resolveNative!: () => void; + vi.mocked(ffi.cleanup).mockReturnValue( + new Promise((resolve) => { + resolveNative = resolve; + }) + ); + + const dw = new DataWeave("/fake/lib"); + dw.initialize(); + + // Two overlapping cleanup() calls while ffi.cleanup() is still pending. + const p1 = dw.cleanup(); + const p2 = dw.cleanup(); + resolveNative(); + await Promise.all([p1, p2]); + + // The native ref-count decrement (ffi.cleanup) and destroyEngine each run + // exactly once, not once per caller -- this is the double-decrement fix. + expect(ffi.cleanup).toHaveBeenCalledTimes(1); + expect(ffi.destroyEngine).toHaveBeenCalledTimes(1); + }); + + it("second overlapping cleanup() call awaits the SAME in-flight native teardown, not an early resolution", async () => { + // Regression test for task-1 fix round 1: doCleanup() flips `state` to + // "cleaning-up" synchronously as its first statement (an async function + // body runs synchronously up to its first await). If cleanup()'s + // not-ready guard (`if (this.state !== "ready") return;`) ran BEFORE the + // `cleanupPromise` coalescing check, a second overlapping call would see + // state already left "ready" and resolve immediately -- never actually + // awaiting the first call's in-flight native teardown. That would + // contradict cleanup()'s documented contract ("resolves once the + // underlying native isolate has actually finished tearing down") and + // silently regress round-4's coalescing timing. This test asserts the + // second call's promise has NOT settled while ffi.cleanup() is still + // pending, by racing it against a marker that only resolves after + // ffi.cleanup() is allowed to settle. + vi.mocked(ffi.createEngine).mockReturnValue(1); + let resolveNative!: () => void; + vi.mocked(ffi.cleanup).mockReturnValue( + new Promise((resolve) => { + resolveNative = resolve; + }) + ); + + const dw = new DataWeave("/fake/lib"); + dw.initialize(); + + const p1 = dw.cleanup(); + const p2 = dw.cleanup(); // overlaps while doCleanup() is in flight + + const SETTLED = Symbol("settled"); + const PENDING = Symbol("pending"); + // A same-tick race: if p2 resolved early (the regression), it wins; + // Promise.resolve() flushes on the same microtask queue, so this + // reliably distinguishes "already settled" from "still pending" without + // relying on real timers. + const raceResult = await Promise.race([ + p2.then(() => SETTLED), + Promise.resolve().then(() => PENDING), + ]); + expect(raceResult).toBe(PENDING); + + resolveNative(); + await Promise.all([p1, p2]); + + expect(ffi.cleanup).toHaveBeenCalledTimes(1); + expect(ffi.destroyEngine).toHaveBeenCalledTimes(1); + }); + + it("does not accumulate process exit listeners across init/cleanup cycles", async () => { + // The module-level `run`/`cleanup` convenience API drives the lazily + // created singleton through `getGlobalInstance()`, which is what + // registers the process-wide beforeExit/exit hooks (registerExitHooksOnce + // in src/dataweave.ts). Unlike the other tests in this file, this doesn't + // construct DataWeave directly, so it hits DataWeave's default + // `findLibrary()` lookup. Point DATAWEAVE_NATIVE_LIB at this test file + // (guaranteed to exist) so that lookup succeeds without depending on a + // real built dwlib -- ffi.initialize() is mocked, so the path's contents + // are never touched. + const prevEnvLib = process.env.DATAWEAVE_NATIVE_LIB; + process.env.DATAWEAVE_NATIVE_LIB = __filename; + try { + const before = process.listenerCount("exit") + process.listenerCount("beforeExit"); + // Drive several singleton create -> cleanup cycles via the module API. + for (let i = 0; i < 5; i++) { + run("%dw 2.0\noutput application/json\n---\n1 + 1"); // creates the singleton (+ hooks on first) + await cleanup(); // releases the singleton + } + const after = process.listenerCount("exit") + process.listenerCount("beforeExit"); + // Register-once: at most the single pair added on the very first create, + // never one pair per cycle. + expect(after - before).toBeLessThanOrEqual(2); + } finally { + if (prevEnvLib === undefined) delete process.env.DATAWEAVE_NATIVE_LIB; + else process.env.DATAWEAVE_NATIVE_LIB = prevEnvLib; + } + }); + + it("still calls ffi.cleanup() (releasing the native init reference) when destroyEngine() throws", async () => { + // Real path: wrong-thread destroyEngine() throws synchronously. If cleanup() + // skipped ffi.cleanup() on that throw, the native init reference for this env + // would leak and block isolate teardown. cleanup() must release it anyway and + // still surface the primary destruction error. + vi.mocked(ffi.initialize).mockImplementation(() => {}); + vi.mocked(ffi.createEngine).mockReturnValue(7); + vi.mocked(ffi.destroyEngine).mockImplementation(() => { + throw new Error("wrong-thread destroy boom"); + }); + vi.mocked(ffi.cleanup).mockResolvedValue(undefined); + + const dw = new DataWeave("/fake/lib"); + dw.initialize(); + + await expect(dw.cleanup()).rejects.toThrow("wrong-thread destroy boom"); + + // The native init reference was still released despite the destroy throw. + expect(ffi.cleanup).toHaveBeenCalledTimes(1); + + // The instance is not stranded "ready": a later initialize() works. + vi.mocked(ffi.destroyEngine).mockReset(); + vi.mocked(ffi.createEngine).mockReturnValue(9); + vi.mocked(ffi.createEngine).mockClear(); // ignore the first init's call + dw.initialize(); + // Prove the re-init genuinely created a fresh engine (not a no-op that + // false-passes toHaveBeenLastCalledWith because the FIRST init already + // called createEngine() with the same args -- review #6 #8). + expect(ffi.createEngine).toHaveBeenCalledTimes(1); + }); + + it("does not publish a poisoned singleton when the first module-level init fails", async () => { + // Isolate module state: a fresh import gives a null globalInstance so this + // test controls the very first getGlobalInstance() call. + vi.resetModules(); + const ffiMod = await import("../../src/ffi"); + const dwMod = await import("../../src/dataweave"); + + // First module-level run(): ffi.initialize() throws (e.g. bad lib path). + vi.mocked(ffiMod.initialize).mockImplementationOnce(() => { + throw new Error("library not found"); + }); + expect(() => dwMod.run("%dw 2.0\noutput application/json\n---\n1")).toThrow(); + + // The fault is corrected; the NEXT module-level run() must build a fresh, + // working singleton -- not reuse a poisoned, uninitialized one that fails + // "not initialized" forever (review #6 #1). + vi.mocked(ffiMod.initialize).mockImplementation(() => {}); + vi.mocked(ffiMod.createEngine).mockReturnValue(1); + vi.mocked(ffiMod.runScriptEngine).mockReturnValue( + JSON.stringify({ + success: true, + result: Buffer.from("1").toString("base64"), + mimeType: "application/json", + charset: "utf-8", + binary: false, + }) + ); + const result = dwMod.run("%dw 2.0\noutput application/json\n---\n1"); + expect(result.success).toBe(true); + }); + + it("gates re-initialization on the in-flight rollback when engine creation fails", async () => { + // Engine creation fails after ffi.initialize() succeeded. The rollback + // ffi.cleanup() is async; until it settles the instance must be in the + // "cleaning-up" state so a concurrent initialize() is rejected deterministically + // rather than racing a fresh isolate against the in-flight release (review #7 #3). + vi.mocked(ffi.initialize).mockImplementation(() => {}); + vi.mocked(ffi.createEngine).mockImplementation(() => { + throw new Error("native engine creation boom"); + }); + let resolveRollback!: () => void; + vi.mocked(ffi.cleanup).mockReturnValue( + new Promise((resolve) => { resolveRollback = resolve; }) + ); + + const dw = new DataWeave("/fake/lib"); + expect(() => dw.initialize()).toThrow(DataWeaveError); + + // Rollback is still in flight: a concurrent initialize() must be rejected, + // not allowed to race a fresh isolate against the pending native release. + expect(() => dw.initialize()).toThrow(/cleanup is in progress/i); + + // Once the rollback settles, the instance is cleanly re-initializable. + resolveRollback(); + await new Promise((r) => setImmediate(r)); // let the .finally run + vi.mocked(ffi.createEngine).mockReturnValue(5); + dw.initialize(); + dw.cleanup(); + expect(ffi.destroyEngine).toHaveBeenCalledWith(5); + }); + + it("does not emit an unhandled rejection when the rollback ffi.cleanup() rejects", async () => { + // The rollback release can itself reject; initialize() must observe it (via + // the stored cleanupPromise) so it never becomes an unhandledRejection, while + // still surfacing the ORIGINAL engine-creation error synchronously (review #7 #3). + vi.mocked(ffi.initialize).mockImplementation(() => {}); + vi.mocked(ffi.createEngine).mockImplementation(() => { + throw new Error("native engine creation boom"); + }); + vi.mocked(ffi.cleanup).mockRejectedValueOnce(new Error("rollback release boom")); + + const dw = new DataWeave("/fake/lib"); + expect(() => dw.initialize()).toThrow(/native engine creation boom/); + + // Give the rejected rollback promise a tick to settle; the .catch() attached + // in initialize() must have consumed it (no unhandledRejection), and the + // instance must be re-initializable afterward. + await new Promise((r) => setImmediate(r)); + // Clear so the assertion below proves the RETRY re-invoked createEngine, + // not the earlier failed attempt's stale no-arg call (review #9 #1). + vi.mocked(ffi.createEngine).mockClear(); + vi.mocked(ffi.createEngine).mockReturnValue(6); + dw.initialize(); + expect(ffi.createEngine).toHaveBeenCalledTimes(1); + expect(ffi.createEngine).toHaveBeenLastCalledWith(); + }); + + it("does not strand the instance in cleaning-up when the rollback ffi.cleanup() throws synchronously", async () => { + // ffi.cleanup() can fail SYNCHRONOUSLY (throw) rather than returning a + // rejected promise. The rollback must still settle its pending state and the + // instance must stay re-initializable; the ORIGINAL engine-creation error is + // what surfaces synchronously to this caller (review #8 #2). + vi.mocked(ffi.initialize).mockImplementation(() => {}); + vi.mocked(ffi.createEngine).mockImplementationOnce(() => { + throw new Error("native engine creation boom"); + }); + vi.mocked(ffi.cleanup).mockImplementationOnce(() => { + throw new Error("synchronous cleanup boom"); + }); + + const dw = new DataWeave("/fake/lib"); + // The synchronous throw to THIS caller is the ORIGINAL engine-creation error, + // not the cleanup throw. + expect(() => dw.initialize()).toThrow(/native engine creation boom/); + + // Let the deferred rollback settle; state must return to "uninitialized" so a + // later initialize() is not permanently rejected with "cleanup is in progress". + await new Promise((r) => setImmediate(r)); + // Clear so the assertion proves the retry actually re-invoked createEngine + // rather than passing on the failed attempt's stale call (review #9 #1). + vi.mocked(ffi.createEngine).mockClear(); + vi.mocked(ffi.createEngine).mockReturnValue(11); + dw.initialize(); + expect(ffi.createEngine).toHaveBeenCalledTimes(1); + expect(ffi.createEngine).toHaveBeenLastCalledWith(); + + await dw.cleanup(); + expect(ffi.destroyEngine).toHaveBeenCalledWith(11); + }); +}); diff --git a/native-lib/node/tests/unit/stream.test.ts b/native-lib/node/tests/unit/stream.test.ts index 46ca0e4a..ab6e3580 100644 --- a/native-lib/node/tests/unit/stream.test.ts +++ b/native-lib/node/tests/unit/stream.test.ts @@ -20,8 +20,9 @@ async function collect( function deferred() { let resolve!: (v: T) => void; - const promise = new Promise((res) => { resolve = res; }); - return { promise, resolve }; + let reject!: (e: unknown) => void; + const promise = new Promise((res, rej) => { resolve = res; reject = rej; }); + return { promise, resolve, reject }; } describe("streamFromNative", () => { @@ -95,4 +96,48 @@ describe("streamFromNative", () => { expect(result.success).toBe(false); expect(result.error).toBe("Empty response"); }); + + it("rejects a parked consumer when native start() rejects (no hang)", async () => { + const startGate = deferred(); + const gen = streamFromNative(() => startGate.promise); + + // Park a consumer in next() BEFORE the start promise settles: no chunk is + // ready and done is false, so next() awaits on pendingResolves. + const pending = gen.next(); + + // Now reject the native start. The parked consumer must be woken and see a + // rejection -- on the pre-fix code done never flips and this hangs forever. + startGate.reject(new Error("native start boom")); + + await expect(pending).rejects.toThrow("native start boom"); + }); + + it("drains buffered chunks, then throws, when start() rejects after pushing chunks", async () => { + const gen = streamFromNative((cb) => { + cb(Buffer.from("x")); + cb(Buffer.from("y")); + return Promise.reject(new Error("late boom")); + }); + + // Buffered chunks yield first... + const a = await gen.next(); + const b = await gen.next(); + expect([a.value?.toString(), b.value?.toString()]).toEqual(["x", "y"]); + + // ...then the drained generator surfaces the start error. + await expect(gen.next()).rejects.toThrow("late boom"); + }); + + it("propagates a native start() rejection of undefined instead of returning empty metadata", async () => { + // Promise.reject(undefined) is valid JS. The old value-sentinel + // (startError !== undefined) treated it as 'never rejected' and returned the + // normal empty-metadata result; a settlement-state flag must propagate it (review #7 #6). + const gen = streamFromNative(() => Promise.reject(undefined)); + await expect( + (async () => { + // Drain fully: iterate to completion so the post-drain re-throw runs. + for await (const _ of gen) { /* no chunks */ } + })() + ).rejects.toBeUndefined(); + }); }); \ No newline at end of file diff --git a/native-lib/src/main/java/org/mule/weave/lib/CallbackWeaveResourceResolver.java b/native-lib/src/main/java/org/mule/weave/lib/CallbackWeaveResourceResolver.java index d6b80912..c2596084 100644 --- a/native-lib/src/main/java/org/mule/weave/lib/CallbackWeaveResourceResolver.java +++ b/native-lib/src/main/java/org/mule/weave/lib/CallbackWeaveResourceResolver.java @@ -3,6 +3,7 @@ import org.graalvm.nativeimage.CurrentIsolate; import org.graalvm.nativeimage.c.type.CCharPointer; import org.graalvm.nativeimage.c.type.CTypeConversion; +import org.graalvm.word.PointerBase; import org.mule.weave.v2.parser.ast.variables.NameIdentifier; import org.mule.weave.v2.sdk.NameIdentifierHelper; import org.mule.weave.v2.sdk.WeaveResource; @@ -20,12 +21,14 @@ */ public class CallbackWeaveResourceResolver implements WeaveResourceResolver { private final NativeCallbacks.ResolveModuleCallback callback; + private final PointerBase ctx; - public CallbackWeaveResourceResolver(NativeCallbacks.ResolveModuleCallback callback) { + public CallbackWeaveResourceResolver(NativeCallbacks.ResolveModuleCallback callback, PointerBase ctx) { if (callback.isNull()) { throw new IllegalArgumentException("Resolver callback cannot be null"); } this.callback = callback; + this.ctx = ctx; } @Override @@ -42,6 +45,7 @@ public Option resolve(NameIdentifier nameIdentifier) { // Invoke callback (blocks if threadsafe function is in use) CCharPointer resultPtr = callback.invoke( CurrentIsolate.getCurrentThread(), + ctx, pathPtr ); @@ -59,8 +63,21 @@ public Option resolve(NameIdentifier nameIdentifier) { ); } } catch (Exception e) { - // Log and return empty on any error - System.err.println("Error resolving module " + path + ": " + e.getMessage()); + // Log and return empty on any error. Mirrors the C-side resolver bridge's + // policy (see resolve_module_callback in addon.c): both the exception + // message AND the module path are resolver-controlled/dynamic content + // (module source, file paths, credentials can leak through either), so + // the default log line is fully static/content-free, with no path and no + // message. Only include them when the caller has opted in via + // DATAWEAVE_RESOLVER_DEBUG=1. + if ("1".equals(System.getenv("DATAWEAVE_RESOLVER_DEBUG"))) { + System.err.println("Error resolving module " + path + ": " + e.getMessage()); + } else { + System.err.println( + "Error resolving module (details suppressed; set " + + "DATAWEAVE_RESOLVER_DEBUG=1 to log path/message — may expose " + + "resolver-controlled data)."); + } return Option.empty(); } } diff --git a/native-lib/src/main/java/org/mule/weave/lib/NativeCallbacks.java b/native-lib/src/main/java/org/mule/weave/lib/NativeCallbacks.java index 3e993c7e..2deaddd3 100644 --- a/native-lib/src/main/java/org/mule/weave/lib/NativeCallbacks.java +++ b/native-lib/src/main/java/org/mule/weave/lib/NativeCallbacks.java @@ -55,6 +55,6 @@ public interface ReadCallback extends CFunctionPointer { */ public interface ResolveModuleCallback extends CFunctionPointer { @InvokeCFunctionPointer - CCharPointer invoke(IsolateThread thread, CCharPointer modulePath); + CCharPointer invoke(IsolateThread thread, PointerBase ctx, CCharPointer modulePath); } } diff --git a/native-lib/src/main/java/org/mule/weave/lib/NativeLib.java b/native-lib/src/main/java/org/mule/weave/lib/NativeLib.java index 549ea3ac..f635ccf0 100644 --- a/native-lib/src/main/java/org/mule/weave/lib/NativeLib.java +++ b/native-lib/src/main/java/org/mule/weave/lib/NativeLib.java @@ -19,6 +19,17 @@ */ public class NativeLib { + /** + * The exact JSON error payload returned by the per-engine entrypoints + * ({@link #runScriptEngine}, {@link #runScriptCallbackEngine}, + * {@link #runScriptInputOutputCallbackEngine}) when {@code handle} does not identify a + * live engine. Package-visible (rather than embedded as a string literal at each call + * site) so the exact contract can be asserted directly from a JVM unit test, since the + * {@code @CEntryPoint} methods themselves rely on GraalVM word types that only resolve + * inside a compiled native image. + */ + static final String UNKNOWN_ENGINE_HANDLE_JSON = "{\"success\":false,\"error\":\"Unknown engine handle\"}"; + /** * Native method that executes a DataWeave script with inputs and returns the result. * Can be called from Python via FFI. @@ -89,6 +100,17 @@ public static CCharPointer runScriptCallback( String inputs = inputsJson.isNull() ? null : CTypeConversion.toJavaString(inputsJson); ScriptRuntime runtime = ScriptRuntime.getInstance(); + return streamToWriteCallback(runtime, dwScript, inputs, writeCallback, ctx); + } + + /** + * Runs the streaming write-callback loop shared by the legacy singleton entrypoint + * ({@link #runScriptCallback}) and the per-engine entrypoint + * ({@link #runScriptCallbackEngine}). + */ + private static CCharPointer streamToWriteCallback( + ScriptRuntime runtime, String dwScript, String inputs, + NativeCallbacks.WriteCallback writeCallback, PointerBase ctx) { StreamSession session = runtime.runStreaming(dwScript, inputs); if (session.isError()) { @@ -170,6 +192,22 @@ public static CCharPointer runScriptInputOutputCallback( String inMime = CTypeConversion.toJavaString(inputMimeType); String inCharset = inputCharset.isNull() ? null : CTypeConversion.toJavaString(inputCharset); + ScriptRuntime runtime = ScriptRuntime.getInstance(); + return transformViaCallbacks(runtime, dwScript, inputs, inName, inMime, inCharset, + readCallback, writeCallback, ctx); + } + + /** + * Runs the input-feeder + output-streaming loop shared by the legacy singleton entrypoint + * ({@link #runScriptInputOutputCallback}) and the per-engine entrypoint + * ({@link #runScriptInputOutputCallbackEngine}). + */ + private static CCharPointer transformViaCallbacks( + ScriptRuntime runtime, String dwScript, String inputs, + String inName, String inMime, String inCharset, + NativeCallbacks.ReadCallback readCallback, NativeCallbacks.WriteCallback writeCallback, + PointerBase ctx) { + // Create a piped input stream session for the callback-supplied input InputStreamSession inputSession = new InputStreamSession(inMime, inCharset); long inputHandle = inputSession.register(); @@ -191,7 +229,6 @@ public static CCharPointer runScriptInputOutputCallback( feeder.start(); // Execute the script and stream output via the writeCallback - ScriptRuntime runtime = ScriptRuntime.getInstance(); StreamSession session = runtime.runStreaming(dwScript, mergedInputs); if (session.isError()) { @@ -330,239 +367,139 @@ private static CCharPointer toUnmanagedCString(String value) { return ptr; } - // ── Resolver-aware FFI Entrypoints ─────────────────────────────────── + // ── Multi-Engine FFI Entrypoints (W-23692110) ──────────────────────── /** - * Runs a DataWeave script with module resolver callback. + * Creates a new isolated engine (ClassLoader-only resolver) and returns its handle. * - *

This variant accepts a {@link NativeCallbacks.ResolveModuleCallback} to resolve - * external modules during script execution. The resolver is installed before script - * execution and remains active for the lifetime of the process.

+ * @param thread the isolate thread + * @return a non-zero handle identifying the new engine + */ + @CEntryPoint(name = "create_engine") + public static long createEngine(IsolateThread thread) { + return ScriptRuntime.register(new ScriptRuntime(null)); + } + + /** + * Creates a new isolated engine backed by a caller-supplied module resolver callback, + * and returns its handle. * - * @param thread GraalVM isolate thread - * @param script DataWeave script source (C string) - * @param inputsJson JSON string of inputs (C string) - * @param resolverCallback Callback for resolving external modules - * @return JSON result or error message (unmanaged C string, must be freed) + * @param thread the isolate thread + * @param resolverCallback callback used to resolve external modules for this engine only + * @param ctx opaque context pointer forwarded to every resolver invocation + * @return a non-zero handle identifying the new engine */ - @CEntryPoint(name = "run_script_with_resolver") - public static CCharPointer runScriptWithResolver( + @CEntryPoint(name = "create_engine_with_resolver") + public static long createEngineWithResolver( IsolateThread thread, - CCharPointer script, - CCharPointer inputsJson, - NativeCallbacks.ResolveModuleCallback resolverCallback) { - - try { - // Install resolver (idempotent if already set) - ScriptRuntime.setResolver(resolverCallback); - - // Delegate to existing run logic - String dwScript = CTypeConversion.toJavaString(script); - String inputs = CTypeConversion.toJavaString(inputsJson); + NativeCallbacks.ResolveModuleCallback resolverCallback, + PointerBase ctx) { + CallbackWeaveResourceResolver resolver = + new CallbackWeaveResourceResolver(resolverCallback, ctx); + return ScriptRuntime.register(new ScriptRuntime(resolver)); + } - ScriptRuntime runtime = ScriptRuntime.getInstance(); - String result = runtime.run(dwScript, inputs); - return toUnmanagedCString(result); - } catch (Exception e) { - return toUnmanagedCString("{\"success\":false,\"error\":\"" - + escapeJsonString(e.getMessage()) + "\"}"); - } + /** + * Destroys an engine created by {@link #createEngine} / {@link #createEngineWithResolver}. + * A no-op if the handle is unknown or already destroyed. + * + * @param thread the isolate thread + * @param handle the engine handle to remove + */ + @CEntryPoint(name = "destroy_engine") + public static void destroyEngine(IsolateThread thread, long handle) { + ScriptRuntime.destroy(handle); } /** - * Runs a DataWeave script with streaming output and module resolver. + * Executes a DataWeave script against a specific engine. * - *

This variant combines streaming output via a write callback with external module - * resolution. The resolver is installed before script execution.

+ *

If {@code handle} does not identify a live engine, returns + * {@code {"success":false,"error":"Unknown engine handle"}} rather than throwing.

* - * @param thread GraalVM isolate thread - * @param script DataWeave script source (C string) + * @param thread the isolate thread + * @param handle the target engine's handle + * @param script the DataWeave script (C string) * @param inputsJson JSON-encoded inputs map (C string), may be null - * @param writeCallback function pointer invoked with each output chunk - * @param ctx opaque context pointer forwarded to callback - * @param resolverCallback Callback for resolving external modules - * @return an unmanaged C string with JSON metadata/error (must be freed) - * - *

NOTE: compiled/linked but intentionally NOT invoked from the Node binding's - * TypeScript layer. runStreaming() deliberately uses the resolver-less streaming entrypoint - * instead: streaming runs its native call on a background thread, and wiring a resolver - * callback there would call back into JS from a non-owning OS thread (undefined behavior / - * crash). Do not wire this up without first solving that cross-thread hazard.

+ * @return the script execution result (unmanaged C string, must be freed) */ - @CEntryPoint(name = "run_script_callback_with_resolver") - public static CCharPointer runScriptCallbackWithResolver( - IsolateThread thread, - CCharPointer script, - CCharPointer inputsJson, - NativeCallbacks.WriteCallback writeCallback, - PointerBase ctx, - NativeCallbacks.ResolveModuleCallback resolverCallback) { - - try { - // Install resolver - ScriptRuntime.setResolver(resolverCallback); - - // Delegate to existing streaming logic - String dwScript = CTypeConversion.toJavaString(script); - String inputs = inputsJson.isNull() ? null : CTypeConversion.toJavaString(inputsJson); - - ScriptRuntime runtime = ScriptRuntime.getInstance(); - StreamSession session = runtime.runStreaming(dwScript, inputs); - - if (session.isError()) { - return toUnmanagedCString("{\"success\":false,\"error\":\"" - + escapeJsonString(session.getError()) + "\"}"); - } - - try { - byte[] buf = new byte[CALLBACK_BUFFER_SIZE]; - CCharPointer nativeBuf = UnmanagedMemory.malloc(CALLBACK_BUFFER_SIZE); - try { - int n; - while ((n = session.read(buf, buf.length)) > 0) { - for (int i = 0; i < n; i++) { - nativeBuf.write(i, buf[i]); - } - int rc = writeCallback.invoke(ctx, nativeBuf, n); - if (rc != 0) { - return toUnmanagedCString("{\"success\":false,\"error\":\"" - + "Write callback returned error: " + rc + "\"}"); - } - } - } finally { - UnmanagedMemory.free(nativeBuf); - } - } catch (IOException e) { - return toUnmanagedCString("{\"success\":false,\"error\":\"" - + escapeJsonString(e.getMessage()) + "\"}"); - } finally { - session.closeStream(); - } + @CEntryPoint(name = "run_script_engine") + public static CCharPointer runScriptEngine( + IsolateThread thread, long handle, CCharPointer script, CCharPointer inputsJson) { + ScriptRuntime runtime = ScriptRuntime.get(handle); + if (runtime == null) { + return toUnmanagedCString(UNKNOWN_ENGINE_HANDLE_JSON); + } + String dwScript = CTypeConversion.toJavaString(script); + String inputs = inputsJson.isNull() ? null : CTypeConversion.toJavaString(inputsJson); + return toUnmanagedCString(runtime.run(dwScript, inputs)); + } - return toUnmanagedCString("{\"success\":true" - + ",\"mimeType\":\"" + session.getMimeType() + "\"" - + ",\"charset\":\"" + session.getCharset() + "\"" - + ",\"binary\":" + session.isBinary() - + "}"); - } catch (Exception e) { - return toUnmanagedCString("{\"success\":false,\"error\":\"" - + escapeJsonString(e.getMessage()) + "\"}"); + /** + * Executes a DataWeave script against a specific engine, streaming the result to a + * caller-supplied write callback. See {@link #runScriptCallback} for the callback contract. + * + *

If {@code handle} does not identify a live engine, returns + * {@code {"success":false,"error":"Unknown engine handle"}} rather than throwing.

+ * + * @param thread the isolate thread + * @param handle the target engine's handle + * @param script the DataWeave script (C string) + * @param inputsJson JSON-encoded inputs map (C string), may be null + * @param writeCallback function pointer invoked with each output chunk; must return 0 on success + * @param ctx opaque context pointer forwarded to every callback invocation + * @return an unmanaged C string with JSON metadata/error + */ + @CEntryPoint(name = "run_script_callback_engine") + public static CCharPointer runScriptCallbackEngine( + IsolateThread thread, long handle, CCharPointer script, CCharPointer inputsJson, + NativeCallbacks.WriteCallback writeCallback, PointerBase ctx) { + ScriptRuntime runtime = ScriptRuntime.get(handle); + if (runtime == null) { + return toUnmanagedCString(UNKNOWN_ENGINE_HANDLE_JSON); } + String dwScript = CTypeConversion.toJavaString(script); + String inputs = inputsJson.isNull() ? null : CTypeConversion.toJavaString(inputsJson); + return streamToWriteCallback(runtime, dwScript, inputs, writeCallback, ctx); } /** - * Runs a DataWeave script with streaming input/output and module resolver. + * Executes a DataWeave script against a specific engine, with a callback-supplied input + * and callback-streamed output. See {@link #runScriptInputOutputCallback} for the callback + * contract. * - *

This variant combines streaming input via read callback, streaming output via write - * callback, and external module resolution. The resolver is installed before script execution.

+ *

If {@code handle} does not identify a live engine, returns + * {@code {"success":false,"error":"Unknown engine handle"}} rather than throwing.

* - * @param thread GraalVM isolate thread - * @param script DataWeave script source (C string) - * @param inputsJson JSON-encoded inputs map (C string), may be null - * @param inputName the binding name for the callback-supplied input (C string) + * @param thread the isolate thread + * @param handle the target engine's handle + * @param script the DataWeave script (C string) + * @param inputsJson JSON-encoded inputs map (C string), may be null + * @param inputName the binding name for the callback-supplied input (C string) * @param inputMimeType the MIME type of the callback-supplied input (C string) - * @param inputCharset the charset of the callback-supplied input (C string), may be null - * @param readCallback function pointer invoked to read input chunks - * @param writeCallback function pointer invoked with output chunks - * @param ctx opaque context pointer forwarded to callbacks - * @param resolverCallback Callback for resolving external modules - * @return an unmanaged C string with JSON metadata/error (must be freed) - * - *

NOTE: compiled/linked but intentionally NOT invoked from the Node binding's - * TypeScript layer. runTransform() deliberately uses the resolver-less transform entrypoint - * instead: transform runs its native call on a background thread, and wiring a resolver - * callback there would call back into JS from a non-owning OS thread (undefined behavior / - * crash). Do not wire this up without first solving that cross-thread hazard.

+ * @param inputCharset the charset of the callback-supplied input (C string), may be null for UTF-8 + * @param readCallback function pointer invoked to read the next chunk + * @param writeCallback function pointer invoked with each output chunk; must return 0 on success + * @param ctx opaque context pointer forwarded to every callback invocation + * @return an unmanaged C string with JSON metadata/error */ - @CEntryPoint(name = "run_script_input_output_callback_with_resolver") - public static CCharPointer runScriptInputOutputCallbackWithResolver( - IsolateThread thread, - CCharPointer script, - CCharPointer inputsJson, - CCharPointer inputName, - CCharPointer inputMimeType, - CCharPointer inputCharset, - NativeCallbacks.ReadCallback readCallback, - NativeCallbacks.WriteCallback writeCallback, - PointerBase ctx, - NativeCallbacks.ResolveModuleCallback resolverCallback) { - - try { - // Install resolver - ScriptRuntime.setResolver(resolverCallback); - - // Delegate to existing streaming I/O logic - String dwScript = CTypeConversion.toJavaString(script); - String inputs = inputsJson.isNull() ? null : CTypeConversion.toJavaString(inputsJson); - String inName = CTypeConversion.toJavaString(inputName); - String inMime = CTypeConversion.toJavaString(inputMimeType); - String inCharset = inputCharset.isNull() ? null : CTypeConversion.toJavaString(inputCharset); - - // Create a piped input stream session for the callback-supplied input - InputStreamSession inputSession = new InputStreamSession(inMime, inCharset); - long inputHandle = inputSession.register(); - - // Merge the stream handle into the inputs JSON - String streamEntry = "{\"streamHandle\":\"" + inputHandle + "\",\"mimeType\":\"" + inMime + "\"" - + (inCharset != null ? ",\"charset\":\"" + inCharset + "\"" : "") + "}"; - String mergedInputs = mergeInputEntry(inputs, inName, streamEntry); - - // Start background thread for reading input - final long readCallbackAddr = readCallback.rawValue(); - final long ctxAddr = ctx.rawValue(); - Thread feeder = new Thread(new InputCallbackFeeder( - readCallbackAddr, ctxAddr, inputSession), "dw-input-callback-feeder"); - feeder.setDaemon(true); - feeder.start(); - - // Execute the script and stream output via the writeCallback - ScriptRuntime runtime = ScriptRuntime.getInstance(); - StreamSession session = runtime.runStreaming(dwScript, mergedInputs); - - if (session.isError()) { - cleanupFeeder(feeder, inputHandle); - return toUnmanagedCString("{\"success\":false,\"error\":\"" - + escapeJsonString(session.getError()) + "\"}"); - } - - try { - byte[] buf = new byte[CALLBACK_BUFFER_SIZE]; - CCharPointer writeBuf = UnmanagedMemory.malloc(CALLBACK_BUFFER_SIZE); - try { - int n; - while ((n = session.read(buf, buf.length)) > 0) { - for (int i = 0; i < n; i++) { - writeBuf.write(i, buf[i]); - } - int rc = writeCallback.invoke(ctx, writeBuf, n); - if (rc != 0) { - cleanupFeeder(feeder, inputHandle); - return toUnmanagedCString("{\"success\":false,\"error\":\"" - + "Write callback returned error: " + rc + "\"}"); - } - } - } finally { - UnmanagedMemory.free(writeBuf); - } - } catch (IOException e) { - cleanupFeeder(feeder, inputHandle); - return toUnmanagedCString("{\"success\":false,\"error\":\"" - + escapeJsonString(e.getMessage()) + "\"}"); - } finally { - session.closeStream(); - } - - cleanupFeeder(feeder, inputHandle); - - return toUnmanagedCString("{\"success\":true" - + ",\"mimeType\":\"" + session.getMimeType() + "\"" - + ",\"charset\":\"" + session.getCharset() + "\"" - + ",\"binary\":" + session.isBinary() - + "}"); - } catch (Exception e) { - return toUnmanagedCString("{\"success\":false,\"error\":\"" - + escapeJsonString(e.getMessage()) + "\"}"); + @CEntryPoint(name = "run_script_input_output_callback_engine") + public static CCharPointer runScriptInputOutputCallbackEngine( + IsolateThread thread, long handle, CCharPointer script, CCharPointer inputsJson, + CCharPointer inputName, CCharPointer inputMimeType, CCharPointer inputCharset, + NativeCallbacks.ReadCallback readCallback, NativeCallbacks.WriteCallback writeCallback, + PointerBase ctx) { + ScriptRuntime runtime = ScriptRuntime.get(handle); + if (runtime == null) { + return toUnmanagedCString(UNKNOWN_ENGINE_HANDLE_JSON); } + String dwScript = CTypeConversion.toJavaString(script); + String inputs = inputsJson.isNull() ? null : CTypeConversion.toJavaString(inputsJson); + String inName = CTypeConversion.toJavaString(inputName); + String inMime = CTypeConversion.toJavaString(inputMimeType); + String inCharset = inputCharset.isNull() ? null : CTypeConversion.toJavaString(inputCharset); + return transformViaCallbacks(runtime, dwScript, inputs, inName, inMime, inCharset, + readCallback, writeCallback, ctx); } } diff --git a/native-lib/src/main/java/org/mule/weave/lib/ScriptRuntime.java b/native-lib/src/main/java/org/mule/weave/lib/ScriptRuntime.java index 3371127a..d8db13ba 100644 --- a/native-lib/src/main/java/org/mule/weave/lib/ScriptRuntime.java +++ b/native-lib/src/main/java/org/mule/weave/lib/ScriptRuntime.java @@ -20,9 +20,16 @@ import java.io.InputStream; import java.nio.charset.Charset; import java.util.Base64; +import java.util.concurrent.ConcurrentHashMap; +import java.util.concurrent.atomic.AtomicLong; /** - * Singleton wrapper around a {@link DWScriptingEngine} used to compile and execute DataWeave scripts. + * Wrapper around a {@link DWScriptingEngine} used to compile and execute DataWeave scripts. + * + *

Each {@link ScriptRuntime} instance owns its own engine (and therefore its own module + * resolver and script cache), so multiple isolated engines can coexist within one process. + * Instances are tracked in a handle-keyed registry so native callers can address a specific + * engine by an opaque {@code long} handle.

* *

Execution results are returned as a JSON string containing a base64-encoded payload plus metadata * (mime type, charset, and whether the result is binary). Errors are returned as a JSON string with @@ -30,84 +37,82 @@ */ public class ScriptRuntime { - private static final ScriptRuntime INSTANCE = new ScriptRuntime(); + // ── Handle registry ────────────────────────────────────────────────── + private static final ConcurrentHashMap REGISTRY = new ConcurrentHashMap<>(); + private static final AtomicLong NEXT_HANDLE = new AtomicLong(1); + + /** Registers a runtime and returns its non-zero handle. */ + public static long register(ScriptRuntime runtime) { + long handle = NEXT_HANDLE.getAndIncrement(); + REGISTRY.put(handle, runtime); + return handle; + } + + /** Returns the runtime for a handle, or {@code null} if unknown/destroyed. */ + public static ScriptRuntime get(long handle) { + return REGISTRY.get(handle); + } + + /** Removes a runtime; returns {@code true} if one was present. */ + public static boolean destroy(long handle) { + return REGISTRY.remove(handle) != null; + } - // Static field for callback resolver, volatile for thread-safe double-checked locking - private static volatile CallbackWeaveResourceResolver resolver = null; + // ── Legacy singleton (ClassLoader-only) for Python entrypoints ──────── + private static volatile ScriptRuntime defaultInstance = null; /** - * Returns the singleton instance. + * Returns the process-wide legacy singleton instance (ClassLoader-only resolver). * * @return the shared {@link ScriptRuntime} */ public static ScriptRuntime getInstance() { - return INSTANCE; + ScriptRuntime local = defaultInstance; + if (local == null) { + synchronized (ScriptRuntime.class) { + local = defaultInstance; + if (local == null) { + local = new ScriptRuntime(null); + defaultInstance = local; + } + } + } + return local; } + // ── Per-instance engine ─────────────────────────────────────────────── + private final DWScriptingEngine engine; + /** - * Sets the module resolver callback and rebuilds the engine. - * Can only be called once per process (engine is a singleton). - * Thread-safe but should be called early in application lifecycle before script execution. - * - *

IMPORTANT: The callback function must be thread-safe if using - * GraalVM's threadsafe function pointers, as it may be invoked from multiple threads - * during concurrent module resolution.

+ * Builds an engine whose resolver is Composite(ClassLoader-built-ins + {@code customResolver}); + * a null {@code customResolver} yields ClassLoader-only. * - * @param callback Thread-safe function pointer for resolving modules + * @param customResolver additional resolver for user-supplied modules, or {@code null} */ - public static synchronized void setResolver(NativeCallbacks.ResolveModuleCallback callback) { - if (resolver != null) { - System.err.println("WARNING: Module resolver already set for this process. " + - "Only one resolver configuration is supported. Ignoring new resolver."); - return; - } - - if (callback.isNull()) { - System.err.println("WARNING: Attempted to set null resolver, ignoring."); - return; - } - - resolver = new CallbackWeaveResourceResolver(callback); - - // Rebuild engine with composite resolver (built-ins + callback) - synchronized (INSTANCE) { - INSTANCE.engine = DWScriptingEngine.builder() - .withDWModuleComponentsFactory(createModuleComponentsFactory()) - .build(); - } + public ScriptRuntime(WeaveResourceResolver customResolver) { + this.engine = DWScriptingEngine.builder() + .withDWModuleComponentsFactory(createModuleComponentsFactory(customResolver)) + .build(); } /** - * Creates composite resolver: ClassLoader (built-ins) + Callback (user modules). - * If no callback resolver is set, returns ClassLoader only. + * Creates composite resolver: ClassLoader (built-ins) + custom (user modules). + * If no custom resolver is provided, returns ClassLoader only. */ - private static WeaveResourceResolver compositeResolver() { + private static WeaveResourceResolver compositeResolver(WeaveResourceResolver customResolver) { WeaveResourceResolver classLoaderResolver = ClassLoaderWeaveResourceResolver.apply(); - - CallbackWeaveResourceResolver currentResolver = resolver; - if (currentResolver == null) { + if (customResolver == null) { return classLoaderResolver; } - return CompositeWeaveResourceResolver.apply( classLoaderResolver, // Try built-ins first - currentResolver // Then callback for user modules + customResolver // Then callback for user modules ); } - private static DWModuleComponentsFactory createModuleComponentsFactory() { + private static DWModuleComponentsFactory createModuleComponentsFactory(WeaveResourceResolver customResolver) { return DWModuleComponentsFactory.createSimpleDWModuleComponentsFactoryBuilder() - .withWeaveResourceResolver(compositeResolver()) - .build(); - } - - // Instance field for the scripting engine, access synchronized in setResolver - private volatile DWScriptingEngine engine; - - private ScriptRuntime() { - // Initialize with ClassLoader-only resolver (no callback yet) - engine = DWScriptingEngine.builder() - .withDWModuleComponentsFactory(createModuleComponentsFactory()) + .withWeaveResourceResolver(compositeResolver(customResolver)) .build(); } diff --git a/native-lib/src/test/java/org/mule/weave/lib/ScriptRuntimeTest.java b/native-lib/src/test/java/org/mule/weave/lib/ScriptRuntimeTest.java index 70f8044b..bf35264a 100644 --- a/native-lib/src/test/java/org/mule/weave/lib/ScriptRuntimeTest.java +++ b/native-lib/src/test/java/org/mule/weave/lib/ScriptRuntimeTest.java @@ -580,6 +580,108 @@ void callbackOutputStreamingError() { System.out.println("=".repeat(50)); } + // --- Multi-engine registry (W-23692110) --- + + /** In-memory WeaveResourceResolver fake — the JVM-constructable seam standing + * in for CallbackWeaveResourceResolver (a CFunctionPointer, which cannot be + * built in test mode). */ + static final class MapResolver + implements org.mule.weave.v2.sdk.WeaveResourceResolver { + private final java.util.Map modules; + MapResolver(java.util.Map modules) { this.modules = modules; } + + @Override + public scala.Option resolve( + org.mule.weave.v2.parser.ast.variables.NameIdentifier id) { + String path = org.mule.weave.v2.sdk.NameIdentifierHelper.toWeaveFilePath(id, "/"); + String key = path.startsWith("/") ? path.substring(1) : path; + String src = modules.get(key); + if (src == null) return scala.Option.empty(); + return scala.Option.apply(org.mule.weave.v2.sdk.WeaveResource.apply(path, src)); + } + + @Override + public scala.collection.immutable.Seq resolveAll( + org.mule.weave.v2.parser.ast.variables.NameIdentifier id) { + scala.Option r = resolve(id); + if (r.isDefined()) { + return scala.collection.JavaConverters + .asScalaBuffer(java.util.Collections.singletonList(r.get())).toList(); + } + return (scala.collection.immutable.Seq) + scala.collection.immutable.Seq$.MODULE$.empty(); + } + } + + private static final String IMPORT_A = + "%dw 2.0\nimport org::test::a\noutput application/json\n---\na::greet(\"X\")"; + private static final String IMPORT_B = + "%dw 2.0\nimport org::test::b\noutput application/json\n---\nb::greet(\"X\")"; + + @Test + void twoEnginesResolveOnlyTheirOwnModule() { + ScriptRuntime engineA = new ScriptRuntime(new MapResolver(java.util.Map.of( + "org/test/a.dwl", "%dw 2.0\nfun greet(n: String) = \"A:\" ++ n"))); + ScriptRuntime engineB = new ScriptRuntime(new MapResolver(java.util.Map.of( + "org/test/b.dwl", "%dw 2.0\nfun greet(n: String) = \"B:\" ++ n"))); + + long hA = ScriptRuntime.register(engineA); + long hB = ScriptRuntime.register(engineB); + assertNotNull(ScriptRuntime.get(hA)); + assertNotNull(ScriptRuntime.get(hB)); + + // Each engine resolves its own module... + assertEquals("\"A:X\"", Result.parse(ScriptRuntime.get(hA).run(IMPORT_A)).result); + assertEquals("\"B:X\"", Result.parse(ScriptRuntime.get(hB).run(IMPORT_B)).result); + + // ...and NOT the other's (no cross-talk). + assertNotNull(Result.parse(ScriptRuntime.get(hA).run(IMPORT_B)).error); + assertNotNull(Result.parse(ScriptRuntime.get(hB).run(IMPORT_A)).error); + + // destroy removes it; a fresh handle is distinct. + assertTrue(ScriptRuntime.destroy(hA)); + assertNull(ScriptRuntime.get(hA)); + assertFalse(ScriptRuntime.destroy(hA)); // already gone + assertNotNull(ScriptRuntime.get(hB)); + + ScriptRuntime.destroy(hB); + } + + @Test + void engineWithoutResolverStillRunsBuiltins() { + ScriptRuntime engine = new ScriptRuntime(null); // ClassLoader-only + long h = ScriptRuntime.register(engine); + String r = ScriptRuntime.get(h).run( + "%dw 2.0\nimport dw::core::Strings\noutput application/json\n---\nStrings::capitalize(\"hello\")"); + assertEquals("\"Hello\"", Result.parse(r).result); + ScriptRuntime.destroy(h); + } + + /** + * Locks in the hard contract for the per-engine FFI entrypoints + * ({@code run_script_engine}, {@code run_script_callback_engine}, + * {@code run_script_input_output_callback_engine} in {@link NativeLib}): running a + * script against an unknown or already-destroyed engine handle must return exactly + * {@code {"success":false,"error":"Unknown engine handle"}} rather than throwing. + * + *

The {@code @CEntryPoint} methods themselves cannot be invoked from a plain JVM + * unit test — they take GraalVM word types ({@code IsolateThread}, {@code CCharPointer}) + * whose boxing infrastructure is only initialized inside a compiled native image (calling + * e.g. {@code WordFactory.nullPointer()} from a hosted JVM test throws + * {@code NullPointerException} from {@code WordBoxFactory}). All three entrypoints funnel + * the unknown-handle case through the same {@code UNKNOWN_ENGINE_HANDLE_JSON} constant, so + * asserting on that constant — combined with {@link #twoEnginesResolveOnlyTheirOwnModule} + * proving {@link ScriptRuntime#get} returns {@code null} for an unregistered/destroyed + * handle — verifies the full contract without needing the native runtime.

+ */ + @Test + void unknownEngineHandleProducesExactErrorJson() { + long unregisteredHandle = Long.MAX_VALUE; + assertNull(ScriptRuntime.get(unregisteredHandle)); + assertEquals("{\"success\":false,\"error\":\"Unknown engine handle\"}", + NativeLib.UNKNOWN_ENGINE_HANDLE_JSON); + } + static class Result { boolean success; String result;