Implement $fy: rule-based shell to mjs translation (JS + Rust)#133
Implement $fy: rule-based shell to mjs translation (JS + Rust)#133konard wants to merge 8 commits into
Conversation
Adding CLAUDE.md with task information for AI processing. This file will be removed when the task is complete. Issue: #1
- Add $fy virtual command that converts shell scripts to command-stream mjs format - Support stdin input for single commands and file input for full scripts - Handle shell operators: &&, ||, ; with proper JavaScript equivalents - Convert pipelines, sequences, variables, exports, and comments - Preserve script structure while translating to command-stream syntax - Add comprehensive test suite with 14 test cases - Include example scripts and debug utilities Features: - Single command conversion: echo "ls -la" | $fy - File conversion: $fy script.sh [output.mjs] - Shell operator translation (&&, ||, ;) - Pipeline preservation - Comment and structure preservation - Variable and export handling - Error handling for missing files 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
konard
changed the title
|
We should use https://github.com/link-foundation/meta-language, and may be logic from https://github.com/link-foundation/relative-meta-logic, in order to actually implement rule based translation, where we first formalize into meta language, and after it use substitution rules on top of meta language to actually finish the translation. Any missing features from https://github.com/link-foundation/meta-language must be reported, so we can fix that first, before continuing this pull request. Also may be you will learn something from github.com/link-assistant/formal-ai related to that specific task. So we use all the best practices possible for both JavaScript and Rust. |
|
🤖 AI Work Session Started Starting automated work session at 2026-07-25T19:11:32.340Z The PR has been converted to draft mode while work is in progress. This comment marks the beginning of an AI work session. Please wait for the session to finish, and provide your feedback. |
The first draft of $fy translated shell to JavaScript with ad-hoc string munging. This replaces it with the two-stage pipeline requested in review: 1. `formalizeShell` parses the script and formalizes it into a link-foundation/meta-language `LinkNetwork` of typed syntax links (`LinkType.Syntax`, language `Shell`). 2. `buildRuleSet` declares Shell -> JavaScript `TranslationRule`s and a small engine substitutes them over the network, rendering each node in the sub-language its parent asks for (statement, command, value, expression). Missing meta-language features found while doing this are reported in docs/meta-language-gaps.md so they can be fixed upstream. Also fixes two genuine bugs found on the way: - `parseSimpleCommand` treated reserved words as block terminators in any position, so `echo done | tr a-z A-Z` lost its argument. Reserved words are only reserved in command position. - `getStdinData` returned the stdio *mode* string as stdin *data*, so every virtual command saw the literal text `inherit` on stdin. experiments/fy-pipeline-repro.sh reproduces the parser bug and examples/fy-translate.mjs demonstrates the whole pipeline.
The reviewer asked for the best practices of both languages, so the
rule-based translator now exists in Rust as well: rust/src/fy mirrors
js/src/fy stage for stage (formalize into a meta-language links network,
then substitute through a TranslationRuleSet).
fixtures/fy/ holds one shell script and the module both implementations
must produce from it; rust/tests/fy.rs and js/tests/fy-tool.test.mjs
assert against those bytes, so the two cannot drift apart.
Also translate ${1:-default}: the brace expander only accepted names, so
positional and special parameters with a default fell through to the
untranslated \${...} escape hatch.
Reports two further meta-language findings: the Rust crate has no public
dynamic-arity link insertion (worked around by chunking), and
with_reference_capture exists in Rust 0.54 but not in JS 0.46.
Done. 1. Formalize → substitute
2. Missing meta-language features — reportedFull write-up with reproductions in
Gaps 1–3 block rule-based translation entirely, so Also confirmed on the Rust side by reading the crate: 3. JavaScript / Rust parity
|
konard
changed the title
The Node.js compatibility job imports js/src/$.mjs without installing dependencies, and the $fy command pulled meta-language in statically, so the module no longer loaded on Node: Module failed to load: Cannot find package 'meta-language' imported from js/src/fy/shell-formalizer.mjs Importing the translator inside the command handler keeps meta-language out of the library's import graph until $fy actually runs.
The Windows job failed on both new tests. A script checked out with CRLF
endings reached the lexer with the \r attached to each word, which broke
the parse ('rule `for` has no JavaScript:command template'), so both
translators now normalize line endings first. The golden fixture must
also arrive with LF for a byte-for-byte comparison to mean anything,
which .gitattributes now enforces.
The run-the-translation test passed a URL pathname as a module
specifier; on Windows that is '/D:/...', which is not importable, so it
passes the file URL instead.
🤖 Solution Draft LogThis log file contains the complete execution trace of the AI solution draft process. 💰 Cost: $29.524047📊 Context and tokens usage:Claude Opus 5: (7 sub-sessions)
Total: (16.2K new + 593.9K cache writes + 34.2M cache reads) input tokens, 256.4K output tokens, $29.524047 cost 🤖 Models used:
📎 Log file uploaded as Gist (10705KB)Now working session is ended, feel free to review and add any feedback on the solution draft. |
✅ Ready to mergeThis pull request is now ready to be merged:
Monitored by hive-mind with --auto-restart-until-mergeable flag |
1 participant
Summary
Implements
$fy(issue #1): a translator from shell scripts to command-streamJavaScript modules — in both the JavaScript and the Rust implementation.
Following review feedback, the translation is genuinely rule-based on top of
link-foundation/meta-language
rather than string rewriting:
LinkNetworkofLinkType.Syntaxnodesover
LinkType.Tokenleaves, with aLanguageProfiledeclaring the supportedsurface (
js/src/fy/shell-formalizer.mjs,rust/src/fy/formalizer.rs).TranslationRuleSetof ~40TranslationRule/TranslationTemplatepairs, one per construct, selected per link byLinkQuery, rewrites that network into JavaScript(
js/src/fy/translation-rules.mjs+rule-engine.mjs,rust/src/fy/rules.rs+engine.rs).All shell and JavaScript knowledge lives in the rule table; the engine has none.
What it translates
Pipelines,
&&/||/;,if/elif/else,while,until,for,case, functions,local,export, redirects,set -e, comments, and${NAME}/${NAME:-default}/$1/$@/$#/$?/$$/$(...)/ backtick expansions. Anything the rules cannot claim is reported as adiagnostic instead of being silently mistranslated.
JavaScript / Rust parity
fixtures/fy/sample.shexercises every supported construct andfixtures/fy/sample.mjsis the module both implementations produce from it,byte for byte.
rust/tests/fy.rsandjs/tests/fy-tool.test.mjseach assertagainst those files, so the two implementations cannot drift apart silently.
Missing meta-language features — reported
Ten gaps are documented with reproductions in
js/docs/meta-language-gaps.mdand filed upstream as link-foundation/meta-language#183. The blocking ones:
TranslationRuleSet.rendersubstitutes no placeholders, does not recurse, andapplies only the first matching rule; and on the Rust side there is no public
dynamic-arity link insertion.
rule-engine.mjs/engine.rsimplement themissing evaluation strategy over meta-language's own types and are written to
collapse into a single upstream call once those land.
Tests
js/tests/fy-tool.test.mjs— 30 tests: the formalizer's network shape, therule set, the CLI, the shared fixture, and an end-to-end check that runs a
translated script and compares its output with
shrunning the original.rust/src/fy/*unit tests (38) plusrust/tests/fy.rsfor the fixture.js/examples/fy-translate.mjsandrust/examples/fy_translate.rsdemonstrateboth halves of the pipeline.
Fixes #1