Skip to content

Render images, with alt text as the fallback - #93

Draft
brainkim wants to merge 6 commits into
mainfrom
images
Draft

brainkim wants to merge 6 commits into
mainfrom
images

Conversation

@brainkim

Copy link
Copy Markdown
Member

Behavior

<img> renders. Before, it was an element with reflected attributes and no box: nothing on screen, not even its alt text. After:

  • It is a box. Its size is the width and height attributes or CSS width and height in cells; else the PNG's pixel size divided by the terminal's cell size, rounded up; else the size of its alt text.
  • In kitty, Ghostty, WezTerm and iTerm2 the box shows the picture. Everywhere else, and whenever the image cannot load, it shows the alt text, clipped to the box. An <img alt=""> with no image is an empty box, no border or placeholder glyph.
  • src may be a data: URL, a file: URL, or a path relative to the working directory. PNG only in this release.
  • load and error fire; naturalWidth, naturalHeight, complete, currentSrc, x, y and decode() behave as in a browser. Changing src reloads.

bun examples/images.ts from the repository root shows a PNG the script builds and hands over as a data: URL, docs/solitaire.png from a path, and a missing source falling back to its alt text.

Design

Two questions go to the terminal at startup, after the mode probes and their echo erase: CSI 16 t for the cell's pixel size (default 8 by 16 when unanswered) and the kitty graphics protocol's own query, which kitty, Ghostty and WezTerm answer. The wire reader learns to bound an APC reply the way it bounds the clipboard's OSC. iTerm2 has no query, so TERM_PROGRAM or LC_TERMINAL answers for it, only when the kitty question went unanswered. Until an answer lands, images are their alt text; the answer triggers a repaint.

A placement is a cell rect. The painter blanks the cells under it so text never paints over the pixels, and the screen writes the image sequence after the frame's rows with the cursor moved to the placement's corner first. Kitty gets one transmission per distinct image, in 4096-byte chunks, and a placement each frame it stays visible, and a deletion when it leaves or the document detaches, so no placement is left over the shell. iTerm2 has no placement ids and gets the whole file each frame. The intrinsic size comes from the PNG's IHDR chunk, no image library.

Change

  • dom.ts: HTMLImageElement builds a UA shadow tree holding the alt text, loads through node:fs/promises imported dynamically so the browser bundle still builds (only data: works there), and gets the members above with lib.dom's types.
  • exchange.ts: the two questions, the APC reply bounding, and negotiateImages, sent after scrubProbeEcho so the erase still follows the mode probes.
  • screen.ts and painter.ts: drawImage on the cell context, per-frame placements, the kitty and iTerm2 sequence writers, and a release on detach.
  • layout.ts: the image's box from attributes, natural size over the cell size, or alt text.
  • examples/images.ts, registered in the playground; an Images section in the layout guide.

Tests

tests/images.test.ts, 11 tests: the alt-text box, attribute sizing, a valid PNG built in the test firing load with the right natural size and the box scaling with a scripted cell-size reply, error on a missing file, kitty transmission and placement when the mock answers the query, no sequence and alt text when it does not, the iTerm2 form under its environment, and placement deletion on removal. Full suite 1636, drift exact, all 38 examples verified.

Conflicts with #92 in painter.ts; whichever merges second gets rebased.

🤖 Generated with Claude Code

https://claude.ai/code/session_015Bw3yEu4cfkuDkaZ3RSy4D

@brainkim
brainkim changed the base branch from main to gradients September 13, 2026 04:11
@brainkim
brainkim changed the base branch from gradients to main September 13, 2026 19:16
@brainkim
brainkim marked this pull request as draft September 13, 2026 22:27
@brainkim
brainkim force-pushed the images branch 3 times, most recently from 7038d95 to 4cfb203 Compare September 16, 2026 05:56
brainkim and others added 6 commits September 16, 2026 12:08
An image element now builds a UA shadow tree holding its alt text, loads
a PNG from a data: URL, a file: URL or a path, and reads the intrinsic
size out of the IHDR chunk. The box is the width and height attributes or
CSS, else the image over the terminal's cell size, else the alt text.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015Bw3yEu4cfkuDkaZ3RSy4D
Two more startup questions: XTWINOPS 16 for what a cell measures in
pixels, and the kitty graphics protocol's own query, which needs the wire
reader to bound an APC reply the way it already bounds the clipboard's
OSC. iTerm2 has no query, so its environment answers for it, and only
where the kitty question went unanswered.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015Bw3yEu4cfkuDkaZ3RSy4D
A placement is a cell rect the painter blanks and the screen writes
after the rows, with the cursor moved to its corner first: kitty gets
one transmission per image and a placement per frame it stays visible,
and a deletion when it goes; iTerm2 gets the whole file each frame,
since it has no placement ids. Detaching releases every placement the
terminal still holds, so none is left over the shell.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015Bw3yEu4cfkuDkaZ3RSy4D
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015Bw3yEu4cfkuDkaZ3RSy4D
Every image landed at its own pixel size wherever the cursor stood, and
the placement that was meant to show it went in after that stray copy.
The transmission asked kitty to display as well as store: a=T. Now it
only stores, and a placement leaves the cursor where it found it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015Bw3yEu4cfkuDkaZ3RSy4D
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant