Composition and templating with native handlebars#1954
Open
alexmojaki wants to merge 13 commits into
Open
Conversation
Managed variables can now reference other variables and render
Handlebars templates against typed inputs.
Composition: `@{variable_name}@` references in serialized variable
values are expanded during resolution, before deserialization. The
resolver walks a small Handlebars-compatible subset of `@{}@` block
helpers (top-level `#if`/`#each`) and supports dotted-path access
(`@{user.name}@`). Resolution priority is shared between top-level
variables and child references via `_BaseVariable._lookup_serialized`:
context override → provider → registered code default. Reference
cycles, missing references, and depth limits surface through
`ComposedReference.error` and propagate as a single composition
warning at resolution time, with the resolver falling back to the
code default. Each resolved variable also carries a `composed_from`
trail (with reason/label/version per ref) onto the span attributes.
Template rendering: a new `logfire.template_var()` registers a
`TemplateVariable[T, InputsT]` whose `.get(inputs)` resolves the
variable, expands `@{ref}@` references, renders any `{{placeholder}}`
expressions using `pydantic_handlebars`, and deserialises to the
declared type. `inputs_type` generates a `template_inputs_schema`
that is included in the variable config and synced to the server.
`pydantic-handlebars` is an optional dependency installed via the
`logfire[variables]` extra on Python 3.10+; calling `template_var()`
without it raises immediately rather than silently degrading.
`Variable` is now a thin subclass of an internal `_BaseVariable`
that holds the shared resolution pipeline. The base class carries
no template-related surface area; `TemplateVariable` overrides
`to_config` to attach `template_inputs_schema`, and external diff/
sync code gates on `isinstance(variable, TemplateVariable)` via the
`get_template_inputs_schema(variable)` helper.
Write-time validation lives in `logfire.variables.template_validation`
and uses `pydantic_handlebars.check_template_compatibility` to detect
undeclared `{{field}}` references across the composition graph. Cycle
detection on the reference graph is also exposed for push-time use.
- Adds `docs/reference/advanced/managed-variables/templates-and-composition.md`
covering Handlebars `{{placeholder}}` rendering via `logfire.template_var()`
and `@{variable_name}@` composition references, including structured
values, cycle detection, and how templates and composition combine.
- Expands the managed-variables index with a templates intro and links
to the new page.
- Mentions `template_inputs_schema` and the `[variables]` extra in the
configuration reference and nav.
- Adds `examples/python/variable_composition_demo.py` exercising
composition, structured variables, template inputs, and
composition-time conditionals end-to-end.
- Skips Python doc examples whose source mentions `logfire.template_var`
when `pydantic-handlebars` is unavailable (matches the runtime
requirement on Python 3.9).
Replaces the regex-and-translate workaround in reference_syntax.py and
composition.py with calls into pydantic-handlebars >= 0.2.0's native
configurable-delimiter API.
## What changed
- pyproject.toml: bump `pydantic-handlebars` to `>=0.2.0` (in both the
`[variables]` extra and the dev group) and exempt it from the
`exclude-newer` filter alongside the other Pydantic packages.
- `_handlebars.get_environment()` returns a cached
`HandlebarsEnvironment(open_delim='@{', close_delim='}@')` for the
composition pass. `extract_composition_dependencies()` wraps
`pydantic_handlebars.extract_dependencies` with the same delimiters.
- `reference_syntax.render_once()` shrinks to a one-line call into that
environment. The sentinel-protect-then-regex-translate code path is
gone; `{{...}}` runtime placeholders survive because they're plain
content under the configured delimiters.
- `composition.find_references()` / `_collect_ref_names()` now route
through `extract_composition_dependencies` for AST-correct detection,
then a textual position scan supplies first-occurrence ordering.
## What you can now write
`@{...}@` accepts the full Handlebars syntax — block helpers with
dotted-path headers, `{{#each}}` parent-context references (`../`),
helper subexpressions, the lot. Previously the regex translator only
covered top-level identifiers in block headers, so e.g.
`@{#if user.active}@` silently dropped its condition. New regression
tests cover these in `TestExpandReferencesNativeHandlebarsSyntax` and
`TestFindReferencesNativeHandlebarsSyntax`.
The unresolved-dotted-reference protection in `_render_value` is kept
for behaviour compat (literal `@{name.field}@` is retained in the
output when `name` can't be resolved). That's the
"misrender retention" parity question — addressed separately.
- Use exact, trimmed equality against 'null' in expand_references rather than a case-insensitive startswith match. JSON null is always lowercase, and 'nullify' shouldn't have matched. - Add a test that exercises the 'dotted ref whose base is resolved' branch in _protect_unresolved_dotted_refs, which the regex narrowing in this PR left uncovered.
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
…ble-composition-clean
…le-composition-native-handlebars
…tic/logfire into feature/variable-composition-native-handlebars
alexmojaki
commented
May 22, 2026
dmontagu
added a commit
that referenced
this pull request
May 24, 2026
Six items Alex flagged as important-but-not-blocking on the original 1951/1952 stack, folded in before #1954 merges: ## Docs - `composition.py` module docstring + `expand_references` docstring no longer claim block helpers are restricted to top-level identifiers — the native handlebars path accepts `@{#if user.active}@` and helper sub-expression headers. - `docs/.../templates-and-composition.md` Control Flow section rewritten to match: drops the "must be top-level" caveat and adds rows for dotted-path conditions and `../` parent-scope access from inside a block. ## Caching - New `compile_composition_template(source)` in `_handlebars` wraps the parse step in an `lru_cache(maxsize=1024)` (#1952 r3289095058). `reference_syntax.render_once` now compiles once per distinct source and reuses the compiled program across resolutions — managed-variable values are typically stable, so the hit rate should be high. - `extract_composition_dependencies` no longer re-runs the `try: from pydantic_handlebars import …` block on every call — the import is moved into a `@cache`d helper that returns the function object directly. Matches the pattern of the existing `_get_template_compatibility_checker` (#1952 r3288194502). ## Warning text - `_composition_failure` renamed to `_fallback_to_default` and takes a `failure_stage` argument (`'composition'` or `'template rendering'`) so the `RuntimeWarning` text reflects the actual failed step. A pydantic-handlebars parse error during `{{...}}` rendering used to surface as `"composition failed"` even though composition succeeded (codex finding). - Updated `test_remote_render_error_records_exception` accordingly. ## Test naming - `test_override_render_failure_falls_back` renamed its locals: `bad_override` → `templated_config` (it's a *valid* template), and introduced `invalid_inputs` for the inputs that actually fail the pattern constraint. Alex r3289312130. ## Escape-detection coupling - Added a comment on the `_HAS_REFERENCE` regex flagging that the lookbehind encodes pydantic-handlebars' current escape semantics ("any preceding `\` escapes") and noting it'll need to count preceding backslashes if pydantic-handlebars adopts Handlebars.js's odd-vs-even-backslash spec behaviour (#1952 r3289062247).
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
This was referenced May 25, 2026
alexmojaki
commented
May 25, 2026
| templated_config = Config.model_construct(code='{{code}}') | ||
| invalid_inputs = Inputs(code='abc123') | ||
| with var.override(templated_config): | ||
| resolved = var.get(invalid_inputs) |
Collaborator
Author
There was a problem hiding this comment.
The point about the warning in #1951 (comment) remains
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
#1951 combined with #1952, i.e. #1952 directly against main, since I think that makes for cleaner review instead of the stack.