[TUI][Architecture] Pluggable renderer backends: preserve native output and allow external implementations #240

Decision / scope

Expose one stable TUI frame-renderer contract in MatterVis; keep MatterVis' existing implementation as the built-in default; permit independently installed external implementations selected by the operator. A separate repository (including tui-representation-research) must be able to implement a renderer and select it without forking, monkey-patching, or modifying MatterVis internals.

MatterVis continues to own source/view state, camera and orbit semantics, projection conventions, visibility/selection state, session lifecycle, and JSONL action transport. The renderer owns presentation of approved display information (depth notation, labels, occlusion, layout, glyphs, etc.). It must not edit the authoritative structure, execute an Agent policy, call a chemistry oracle, or grade answers.

Implement now. Do not wait for #163. This is a narrow Display-side extension point consistent with #163: Display(S,V,request) -> (S,V'); it does not add a second camera, a second action reducer, or Operate/Analysis behavior. Keep #163's future changes compatible with the contract, but do not make this issue contingent on completion of #163.

Downstream integration: research Issue #17. The sequencing is MatterVis renderer interface + built-in backend + independent external-backend smoke first; research harness/skill/renderer experiment refactor second.

Verified existing call sites

Code inspected at MatterVis main@4a3c6b9ab78c68e8a5bc784d1c3740a12d8b2f73 (2026-10-25):

• mat_viewer/tui/controller_depth.py::DepthControls.render_frame(): computes project_points, then directly calls compositor.compose_frame(). This is the stateful/session injection candidate.
• mat_viewer/tui/compositor.py::compose_frame(): currently chooses render_glyph_frame() versus molecule/network renderers. Preserve both as one native backend rather than rewriting their behavior in v1.
• mat_viewer/tui/cli.py::compose_static_frame() also calls compose_frame(). A hook only in DepthControls would leave static and active paths inconsistent; both must share one renderer dispatch.
• mat_viewer/tui/session.py::TerminalSession owns observe/orbit/align/pan/zoom/set_display/reset and emits mattervis.tui.session/v1 JSONL observations with frame.sha256 and state_fingerprint. These semantics must remain stable.
• mat_viewer/tui/crystal_ir.py::CrystalIR contains chemistry, metadata, molecule assignments and inferred information; loader_adapter.py can attach MolCrysKit chemistry records, and controller.observe() emits chemistry_warnings. Never pass raw CrystalIR or these analysis records to an untrusted/external renderer or benchmark model.

Layer ownership and API

source / existing MatterVis loader
          |
          v
operator-approved display IR (geometry + provided topology provenance only)
          |
MatterVis Session + View State + Camera + Projection + Action dispatcher
          |
          v
        RendererBackend.render(RenderRequest) -> RenderedFrame
          |                               |
          |-- mattervis.native-v1          |-- externally installed backend
          |   (compose_frame wrapper)      |   (e.g. research.depth-v1)
          v
bounded ASCII/Unicode frame + hash + renderer identity
          |
existing JSONL observation / research Agent harness

Define a public, versioned, typed render request and result, with a JSON-serializable schema and an equivalent Python Protocol/dataclasses. Proposed contract names are illustrative; freeze exact spellings in the implementation PR:

class RendererBackend(Protocol):
    backend_id: str
    api_version: str
    def render(self, request: RenderRequest) -> RenderedFrame: ...

RenderRequest (v1), immutable/public-safe

• display_ir: stable atom/site and displayed-copy IDs, element/species, approved Cartesian/fractional geometry, cell/PBC, provided bond endpoints and periodic image translations, explicit edge provenance, and separately declared enabled visibility/selection. Never pass CrystalIR itself.
• view: frozen camera orientation/basis/projection, target/zoom/pan, host-derived projected coordinates + depth when applicable, clipped/slice/focus/ghost state and viewport; rotation math and action semantics are not replaced by renderer plugins. Make any host-derived geometry available with a documented frame and units.
• render_options: charset, 100x36 or other bounded viewport, atom/molecule display level, depth/label/bond/cell/axis/boundary toggles and arbitrary namespaced, schema-validated backend options.
• request_schema_version, deterministic view-state and authorized-input digests. No task answer, scorer, CIP, IUPAC/name inference, MolCrysKit analysis object, gold, repair, structure-validity assertion, or benchmark split/label.

RenderedFrame (v1)

• Exact width x height lines of bounded text; machine ascii7 must be true 7-bit ASCII, no ANSI escapes or control characters; Unicode profile remains allowed where already supported. Reject malformed lines, size overflow, NaN-like metadata, or unexpected extra fields.
• Optional non-agent-visible layout/visibility diagnostics as a separate audit artifact if useful (e.g. omitted/clipped atoms, collisions); do not append scientific interpretations to the ASCII screen.
• Return frame text plus backend ID/version, config hash, implementation/package digest and content hash in host-owned run provenance. Preserve the existing frame SHA algorithm for identical bytes; document whether renderer identity appears in new optional observation fields or in a sidecar. Do not silently change state_fingerprint for default sessions.

For backend APIs, share the same cleaned IR across static and active TUI. A renderer may change glyphs and presentation, but must not mutate input IR/camera/session. Deterministic replay: same source IR + camera/view + config + backend digest => same frame bytes, unless a nondeterministic backend is explicitly disallowed from reproducible benchmarking.

Registration and selection

1. Ship mattervis.native-v1 as the default built-in adapter delegating to today's compose_frame (including atom vs molecule/network paths). No renderer flag => unchanged legacy behavior.
2. Provide an explicit registry for operator-installed renderers, e.g. a Python wheel exposing a mattervis.tui.renderers entry point. Resolve and validate it once per session; use explicit backend IDs and schema versions. Do not load arbitrary module:function paths supplied by a solver task, CIF metadata, a model-generated action or unknown CLI flags.
3. Allow TerminalSession(..., renderer=..., renderer_options=...) (or equivalent constructor-level RendererSpec) and CLI --renderer ID --renderer-config FILE for both stateful --session-format jsonl and static output. The CLI syntax is proposed, not currently implemented. Document a library API suitable for research without shell monkey patches.
4. Renderer selection/configuration is frozen for an episode and is not an Agent set_display action. Existing set_display, orbit, reset, viewport, and saved-view actions retain their meanings.
5. Missing, incompatible, unpinned or failing requested backend => explicit error, never silent fallback to native. Record a backend/format/config identity sufficient to replay. Keep a versioned manifest that research can bind into the experiment/attempt identity.
6. For trusted local plugins, an installed wheel/entry-point adapter is sufficient. Document that Python plugin loading executes code with host privileges: any untrusted contestant-supplied renderer must instead run in a process/container with isolated mounts/network and only approved display IR; this is the deployment boundary, not a claim that a Python Protocol is a sandbox. An optional JSONL subprocess adapter may be implemented without changing the high-level API if needed, but must not create another render semantics implementation.

Scientific/visibility safety

• Create a sanitized model-facing observation profile (opt-in, default behavior unchanged): frame and approved camera/action information only. Review title, scope, warnings, selection and error bodies, not just frame.text; the present default can include chemistry warnings or inferred metadata.
• Geometry-only mode must never call MolCrysKit/CIP/name/error classifiers during rendering. Any allowed bond graph must declare its origin (provided input vs inference), and inferred edges must be disabled when the experiment specifies bonds_off or provided_only. Do not silently repair a corrupted candidate topology.
• Externally supplied renderer can mathematically infer properties from full xyz. Therefore avoiding precomputed gold is necessary but not a proof of no shortcut: require canary/metamorphic non-leak tests, independent parity/information audits, and separate scorer access. No verified claim arises merely from satisfying this interface.
• The benchmark, not MatterVis, owns model prompt, action/token budget, task truth, and official scorer. MatterVis exposes the rendering mechanism, not a chemistry-answer tool.

Implementation sequence (same Issue; PRs may be phased)

Phase A: preserve baseline and introduce the seam

• Freeze golden frames/JSONL traces from current builtin in 80x24 and 100x36, ascii7/Unicode, depth on/off, molecule/framework, CIF/PBC and representative observe -> orbit -> set_display -> reset episodes.
• Add sanitized DisplayIR/RenderRequest builder, renderer protocol, native adapter and single dispatch used by both static and session. Keep legacy default byte-for-byte.

Phase B: prove external replacement

• Add explicit installed-plugin registration/configuration, selection in CLI and Python session API, input/output schema and capability validation.
• Supply an independent external package (a wheel or test fixture installed separately, not copied into the MatterVis source tree) implementing a simple visibly different ASCII depth/label policy. Exercise it without editing MatterVis code. Include a runnable example and pin/version instructions.

Phase C: isolation, audits and backward compatibility

• Introduce benchmark-safe observation export policy, hash and replay identity, non-leak tests, invalid-backend fail-closed tests, renderer switch equivalence of camera/action states and static-vs-active initial frame equality.
• Run the focused camera polar-orbit regression (Fix turntable pitch escape from polar views #227) and broader existing TUI/static/CLI tests. Document what was and was not scientifically validated.

Acceptance / Definition of done

• Existing native TUI CLI/API and default JSONL traces are behaviorally and bytewise unchanged for frozen fixtures; no camera/orbit regression.
• Both static and active paths use the same renderer selection and clean request contract; no duplicate rendering stack or state machine.
• An external separately installed backend changes the screen using the same source, viewport, camera and actions, without a MatterVis fork or edit.
• Missing/incorrect renderer ID, wrong API version, unsupported options, edited/pin-mismatched code, malformed frame or plugin crash fail closed; no silent native fallback.
• RenderRequest excludes chemistry/answer records and preserves atom, bond, image-translation identities/provenance; negative canary and poisoned-hidden-chemistry tests prove model-facing output invariance.
• ascii7 screen stays bounded, hashes/state/replay bind renderer code+options without breaking old fingerprints; deterministic replay passes.
• Different renderers are experiment factors; changing the renderer never changes task identity, action semantics, scoring oracle or allowed tool set.
• Developer documentation specifies the public Python/CLI contracts, plugin packaging example, trust/isolation limitations, pinning, tests and migration path.
• Conforms to [Refactor] Strictly separate Display / Operate / Analysis: preserve display APIs, implement structural operations, and remove mixed implementations #163's Display-only ownership without waiting for it. Do not classify a renderer tweak as a committed structural operation.

Non-goals: new Agent harness, benchmark task generation, a chemical answer/repair scorer, a real MLIP, rewriting molecule/compositor internals on day one, replacing existing MatterVis GUI/CLI, or closing #163.