[vrenken]

Requested by Peter · project thread

The second of the three codegen PRs. It follows the design doc: a pest-style attribute macro in a companion crate, built on the tree shapes from #9.

#[lark_grammar(file = "calc.lark", parser = "lalr")]
#[lark_result(f64, "add", "mul", "number")]
pub mod calc {}

impl calc::Visitor for Evaluator {
    type Error = std::num::ParseFloatError;
    fn visit_add(&mut self, _: calc::AddNode<'_>, sum: f64, product: f64) -> Result<f64, Self::Error> { Ok(sum + product) }
    // ...
}
let value = Evaluator.visit_start(&calc::parse("1 + 2 * 3")?)?;

What the module gets

• The parser: parse, parse_{start} and lex, on a Lark built on first use. The grammar file is embedded with include_str!, so cargo rebuilds when it changes.
• Name constants: rules::ADD, terminals::NUMBER.
• Node wrappers: AddNode<'a> for each rule, alias and template. Each one is Copy and derefs to Tree, with an accessor per child:
  • a node wrapper or &Token for a child of one kind;
  • a generated enum for a child that can be several things, named after its ?rule (Shipping::ExpressShipping);
  • Option for [x] and x?;
  • an iterator with a plural name for x* and x+.
• Typed visitor (with #[lark_result]): one required method per mapped tree, plus visit_x_token per mapped terminal. Each method gets its children already visited and converted. Unmapped children come as their node, so only mapped trees are visited. It doesn't recurse.
  • The provided visit_x_tree visits a tree, and visit_{start} is added when all of a start rule's trees share a result type.
  • Mapping a ?rule sets the type of the children it stands for, and every tree it can be must then have that type (like LARK008). Children whose trees have different result types get a ...Value enum, so nothing falls back to an untyped object.
• Errors at the attribute: grammar errors, and names that aren't in the grammar or are mapped twice, are compile errors that point at the attribute.

The engine is etalii_lark::codegen::Generator, which a build.rs can use to write the same code to a file. The macro is a thin wrapper around it.

One deviation from the doc: I left out the separate untyped visitor trait. The node wrappers work with the crate's existing Visitor, and two traits called Visitor in one module would clash.

Tests

• etalii-lark-derive/tests:
  • the calculator grammar with variables and errors, including a visit of a 50,000-deep tree;
  • JSON, with ?value mapped as a whole;
  • purchase orders read through the node wrappers.
• codegen-tests (unpublished): generates and compiles the code for all 206 distinct grammars in the conformance suite, in two mappings. One maps everything to u64; the other maps the terminals to String, which exercises the value enums. This caught a keyword bug (as) and a name-clash bug: result types now resolve through aliases in the declaring scope, so a user type named like a generated enum still works.
• examples/purchase_orders.rs ports the .NET docs example to Rust, and CI runs it.

Release

• Both crates take their version from the workspace, and release-rust.yml sets it for both and runs cargo publish -p etalii-lark -p etalii-lark-derive.
• CI now packages the whole workspace.
• The first publish of etalii-lark-derive needs the token to have the publish-new scope. After that, trusted publishing can be set up for it too.

---

Generated by Claude Code