Replace Duktape 1.0.2 with QuickJS 2026-06-04 (ES2025) - #221
Closed
evgeny-boger wants to merge 18 commits into
Closed
Replace Duktape 1.0.2 with QuickJS 2026-06-04 (ES2025)#221evgeny-boger wants to merge 18 commits into
evgeny-boger wants to merge 18 commits into
Conversation
QuickJS is pinned as a git submodule (third_party/quickjs, bellard/quickjs @ 3d5e064 = the 2026-06-04 release, verified byte-identical to the official tarball) and compiled by cgo through one-line wrapper files in internal/quickjsduk — no extra build system, cross-compilation rides the existing CGO cross-CC flow. internal/quickjsduk reimplements the go-duktape API surface wbrules uses (wired via a go.mod replace, so wbrules code is untouched): value-stack semantics, per-file realms via JS_NewContext, heap stash, Go-func trampolines with Duktape's rc-error strings, Duktape 1.x CommonJS (per-realm cache, relative ids, cycle-safe), calling-realm dispatch for native callbacks, a pending-job pump for promise reactions, and JS_UpdateStackTop per API entry (Go migrates goroutines across OS threads). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
escontext.go: parse QuickJS stack-line format; take the error message from the error value (Duktape embedded it in .stack); substitute the script path into syntax-error tracebacks. Engine-behavior test-data updates: rule locations attribute multi-line defineRule calls to their first line; StorableObject bookkeeping fields are now non-enumerable (spec-correct for-in walks the proxy prototype chain); the email-emoji log shows proper UTF-8 instead of Duktape's CESU-8 leak. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Makefile: prefer bfd for arm/arm64 external linking (Go forces gold, gone from binutils >= 2.44). debian: build with system golang-go. Samples: sample-es2025.js (ES2024/25 feature showcase incl. arrow-function rules) and sample-bench.js (ES5 benchmark rules used for the published numbers). PORT-QUICKJS.md documents the port, test status (36/36 suites), and the WB8 hardware validation: memory parity, latency parity, ~4x compute. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Not up to standards ⛔🔴 Issues
|
| Category | Results |
|---|---|
| UnusedCode | 39 medium |
| BestPractice | 1 medium 11 high |
| ErrorProne | 23 high |
| Security | 2 minor 5 high 3 critical 13 medium |
| CodeStyle | 3 minor |
🟢 Metrics 609 complexity · 17 duplication
Metric Results Complexity 609 Duplication 17
NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.
Scripts harvested from wiki.wirenboard.com, support.wirenboard.com and
GitHub were loaded through the engine (corpus_test.go, WB_RULES_CORPUS
env, shardable). The corpus is not distributed; every bug it found is
pinned by a minimal synthetic regression test instead.
- Shutdown races: never close syncQueue (a send on a closed channel
panics even inside select); Stop handshakes via syncDone, late
timer/cron/shell thunks are dropped deterministically. TestStopUnderLoad.
- trackMqtt() as a script's first action lazily starts the engine MQTT
client, like Publish does. TestTrackMqttAsFirstAction.
- JS execution watchdog via JS_SetInterruptHandler: runaway rules
(while(true)) error out instead of freezing the engine loop forever.
SetExecutionTimeLimit + JsExecutionLimit option. TestRunawayScriptInterrupted.
- Strict-mode writes: dev/PersistentStorage/StorableObject proxy set
traps return true ('use strict' scripts threw TypeError). TestStrictModeProxyWrites.
- exports/module.exports in scope for plain rule files. TestExportsInRuleFile.
- require() of a missing module throws 'cannot find module id', not a
bare rc -100. TestMissingModuleErrorMessage.
- Spurious 'stack overflow': QuickJS's stack anchor and the deep
operation must share one cgo call (goroutines migrate OS threads
between calls) - qjd_eval/qjd_call/... wrappers.
- syncQueueActive is atomic (race found by -race); opaque-id pointer
fabrication moved into C for checkptr; tests resolve ../modules from
the source-file path, immune to cwd changes by chdir-ing fixtures.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AoTGMFABCCENSPaHThxAru
.ts rule files transpile on load through a persistent 'tsgo --api --async' child (~1ms warm, LSP-framed JSON-RPC) and run immediately; type checking runs afterwards in the background (tsgo --noEmit --lib esnext with types/wb-rules.d.ts) and never delays execution. Findings are logged as '[rule warning] TS check: file:line:col' and published as retained JSON on /wbrules/ts-check/<virtual path> for editor UIs; a clean re-check clears the topic. Tracebacks map through the V3 source map back to .ts lines. Declaration files (.d.ts) in watched dirs are skipped. The tsgo child dies with the daemon (PDEATHSIG) and is shipped in the deb (/usr/lib/wb-rules/tsgo) together with the declarations (/usr/share/wb-rules/types/wb-rules.d.ts). New flags: -tsgo, -ts-types, -js-timeout. Samples: demo-typescript.ts, demo-ts-typecheck.ts. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AoTGMFABCCENSPaHThxAru
Verified against the pinned QuickJS build: all probed ES2025 features pass (iterator helpers, Set methods, Promise.try, RegExp.escape, duplicate named groups, Float16Array); of the ES2026 candidates only explicit resource management (using/DisposableStack) is missing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AoTGMFABCCENSPaHThxAru
All confirmed by independent verification against the code: - PcallProp (every rule/timer/device callback) and JsonEncode/JsonDecode still called JS_Invoke/JS_JSONStringify/JS_ParseJSON as bare cgo calls, re-opening the stack-anchor/thread-migration window on the hottest dispatch path; now fused via qjd_invoke/qjd_json_* wrappers. - The JS execution watchdog never fired inside promise jobs (execStart was zero during the pump): a spinning .then reaction hung the engine forever despite -js-timeout. Each pending job is now armed as an outermost entry. Regression: TestExecutionTimeLimitInPromiseJob. - .ts files loaded as raw JavaScript when tsgo was unavailable; they now fail with a clear 'TypeScript rules need tsgo' error. - handleStop closes syncDone before flipping the active flag - the old order left a window where MaybeCallSync ran thunks inline on foreign goroutines. - ESEngine.Stop stops the engine loop before killing tsgo (a draining .ts load could respawn an orphan child); transpile no longer kills the warm tsgo child on syntax errors (only on transport errors); background checks are capped at 2 concurrent tsgo processes; a per-file generation guard stops a slow check for an old file revision from overwriting the newer verdict. - DestroyHeap deletes the rtReg entry (runtimeState leak). - Test robustness: corpus drainer survives post-test Verify timeouts, module-missing classification matches the new error text, watchdog tests fail cleanly instead of hanging the binary, TsTypesPath resolved from source location, corpus fixture closes its bolt DB. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AoTGMFABCCENSPaHThxAru
Downloaded the es2026 tag of tc39/ecma262 and probed the pinned QuickJS build against the actual 7-proposal feature list: 6 of 7 supported (getOrInsert, Iterator.concat, JSON.rawJSON, Uint8Array base64/hex, Math.sumPrecise, Error.isError); only Array.fromAsync is missing. using/DisposableStack, Atomics.pause and Temporal are ES2027 track, not ES2026 (Atomics.pause is supported anyway). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AoTGMFABCCENSPaHThxAru
The shared Jenkins deb pipeline has no prebuilt tsgo binaries, so the install step now packages without TypeScript support when the binary is absent (the engine rejects .ts files with a clear error in that case) instead of failing the build. The Actions workflow runs vet + the shim and engine test suites on every PR, building wbgo.so from wbgo-private with the same toolchain in the same job (plugin/binary pairing). Requires a WBGO_PRIVATE_TOKEN repo secret with read access to wirenboard/wbgo-private. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AoTGMFABCCENSPaHThxAru
… ts-check topic The retained /wbrules/ts-check/<file> topic had structural lifecycle problems (stale diags surviving daemon restarts, nothing cleared on file delete/rename/disable) and no consumer besides the web editor. Replaced by the established mqtt-rpc surface, both documented in asyncapi.mqtt-rpc.yml: - Editor.Check(path): synchronous type check of one rule file with the engine's own tsgo and installed declarations - the authoritative verdict, pulled by editors on open/save instead of pushed retained. - Editor.GetTypes: returns the installed wb-rules.d.ts so editors can validate against the API of the engine actually running, eliminating vendored-declaration version skew. The load-time background check and its 'TS check:' log warnings stay unchanged - they are the only type feedback for rule files that never pass through the editor (scp/vim/ansible deployments, the dominant wild-corpus workflow). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AoTGMFABCCENSPaHThxAru
Fixes the suite failure the previous commit shipped with (the rpc fixture verifies the exact announcement list). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AoTGMFABCCENSPaHThxAru
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AoTGMFABCCENSPaHThxAru
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AoTGMFABCCENSPaHThxAru
…dings Findings from the dedicated adversarial review of the RPC restructure: - Editor.Check no longer runs a tsgo process synchronously on the serially-dispatched RPC goroutine (it head-of-line blocked every editor RPC for seconds per check). The engine already checks each .ts file on load/save; Check now reads that cached verdict and answers 'pending' while a check is in flight - clients poll. Statuses: ready/pending/not-ts/unsupported; 'not-ts' also disambiguates disabled files (previously indistinguishable from a clean verdict). - Diagnostics from other files (imports/references) now carry their file in the reply instead of being silently mis-anchored. - asyncapi: add the channel definitions both new operations referenced (the earlier insertion silently failed; operations pointed at nonexistent channels), update the Check reply schema. - TS_CHECK_* constants live in their own const block: the main block derives itemType from iota, which counts every preceding spec - the first attempt shifted SOURCE_ITEM_* by 4 and broke rule reloading. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AoTGMFABCCENSPaHThxAru
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AoTGMFABCCENSPaHThxAru
- Watchdog: a Go callback re-entering JS from inside a promise job (require(), runRules()) went depth 0->1->0 and disarmed the execution limit for the rest of the job; nested returns also drained the pending job queue mid-job, breaking run-to-completion. The job pump now counts as an active entry. Regression: TestExecutionTimeLimitSurvivesNestedEntryInJob. - Async rule errors: the JobErrorHandler documented as wired to the rule log was never wired - exceptions after the first await died on stderr only. Now per-heap (SetJobErrorHandler) and routed to the engine log as 'async rule error: ...'. - tsgo child: 15s watchdog kills a wedged (alive but stuck) child so the engine loop cannot block forever in a .ts load; response ids are verified (desync = transport error -> respawn); on-demand checks get a 60s timeout + PDEATHSIG; the reaper/ensureStarted ProcessState race is gone (liveness tracked via c.cmd under the mutex). - Editor.Check: files that fail transpile now get a terminal 'ready' verdict carrying the syntax error instead of 'pending' forever; .d.ts answers 'not-ts'; same-file diag matching by path suffix, not basename. - Transpiled TypeScript runs strict again: the stripped 'use strict' prologue is re-added inside the single-line wrapper (line numbers intact) - a typo'd assignment now throws instead of creating a global. - Stop(): drain the sync queue while waiting for the stop ack - with a full buffer the main loop could block on its own CallSync before reaching handleStop, deadlocking shutdown. - goThreadFinalize queues leftover stack values for the reap path instead of freeing under regMu (finalizer re-entry self-deadlock). - Duktape.version reports 10002 (the 1.0.2 being emulated), not 20600. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AoTGMFABCCENSPaHThxAru
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AoTGMFABCCENSPaHThxAru
# Conflicts: # debian/changelog
Member
Author
|
Split per review request into two stacked PRs: #223 (QuickJS engine + wild-corpus hardening, base master) and #224 (TypeScript support, stacked on #223). The |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Swaps the ES5-era Duktape 1.0.2 engine for QuickJS 2026-06-04 (latest Bellard release), bringing ES2023/ES2024 and most of ES2025 to rule scripts — classes with private fields, async/await + working promise microtasks,
Object.groupBy, iterator helpers, Set algebra,toSorted/findLast, arrow-function rule callbacks, BigInt, and more. Seesample-es2025.js.How
third_party/quickjs→ bellard/quickjs3d5e064, byte-identical to the official 2026-06-04 tarball). Patches, if ever needed, live as commits on a fork and rebase cleanly against upstream.internal/quickjsdukimplements the go-duktape API surface wbrules uses, on QuickJS — wired in with a one-linego.modreplace. wbrules engine code is untouched except stack-trace-format parsing inescontext.go. cgo compiles the submodule sources through five one-line wrapper files: no new build system, and armhf/arm64 cross-compilation rides the existingCGO_ENABLED=1 CC=<cross-gcc>flow.require()cache withmodule.static, Duktape's rc-error strings, plus a pending-job pump so promises resolve (Duktape 1.x had none).Validation
wbgo.so(single-processgo test ./wbrules/, 49.9 s). Three test-data updates for engine-behavior differences, each documented inPORT-QUICKJS.md.sample-bench.js)Memory at parity, reaction latency dominated by the MQTT/driver path either way, raw JS ~4× faster.
Notes for review
PORT-QUICKJS.mdis the full engineering log: ported semantics, the three behavior changes, packaging notes (plugin/binary-trimpathpairing,-fuse-ld=bfdfor arm64 on binutils ≥ 2.44), and benchmark methodology.git submodule update --initand, for cross-builds,crossbuild-essential-*+ foreign-arch libc6 (or the existing sbuild chroots)..gitmodulesthere.🤖 Generated with Claude Code
Update: TypeScript support + wild-corpus hardening
TypeScript rules (
6c9b649):.tsfiles in/etc/wb-rules/transpile on load via a persistent typescript-go child (~1 ms warm) and run immediately; type checking happens in the background and never delays execution ("run first, check later"). Findings land in the rules log (TS check: file:line:col) and as retained JSON on/wbrules/ts-check/<file>for editor integration. Tracebacks map back to.tssource lines through the emitted source map. Shipstsgo(/usr/lib/wb-rules/tsgo) andtypes/wb-rules.d.ts(builtin API declarations, also used by the homeui editor). New flags:-tsgo,-ts-types,-js-timeout.Wild-corpus validation (
cbe69b3): 663 user scripts harvested from wiki/support/GitHub were loaded through the engine — final sweep: 508 load OK, 5 need user modules not present, 150 fail for script-side reasons (forum-paste fragments, missing/etcconfigs), 0 crashes/hangs. The corpus is not distributed; every bug it exposed is pinned by a minimal synthetic regression test:syncDonehandshake)trackMqtt()before any publish crashed the engine MQTT client (lazy start on subscribe)while(true)) froze the engine forever → new watchdog viaJS_SetInterruptHandler(-js-timeout, default 60 s); stock Duktape wb-rules freezes identically'use strict'scripts threw on everydev/PersistentStoragewrite (proxy set traps didn't returntrue)exports/module.exportsnow in scope for plain rule filesrequire()of a missing module reportscannot find module "id"instead ofrc -100stack overflowon valid code: QuickJS's C-stack anchor vs Go goroutine↔OS-thread migration (anchor+operation fused into single cgo calls)-racenow viable: checkptr-clean shim, race-built plugin pairing documented)Validated on a WB8:
2.47.0~quickjs4deployed, TS demo rules live, checker output verified end to end.