diff --git a/CLAUDE.md b/CLAUDE.md index 7bbd835..b6fdd53 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -22,7 +22,7 @@ cp .env.example .env # YOUTUBE_API_KEY ./run.sh # https://localhost:8444/ uv run pytest tests/ -v # default suite, never hits the network node --test 'static/*.test.mjs' # pure frontend logic -uv run pytest tests/test_player_smoke.py -m browser -v # real Chromium; no quota +uv run pytest tests/ -m browser -v # real Chromium, all three suites; no quota uv run pytest -m live -v # one real search; spends 100 quota units uv run ruff check . && uv run ruff format . ``` @@ -64,13 +64,15 @@ npm run deploy # needs Docker; see docs/DEPLOY.md | `static/grid-logic.js` | Pure slot/reserve bookkeeping + config-change classification. Node-testable. | | `static/wallstate.js` | What *this browser* is watching: current query + history in `localStorage`. Every access is defensive. | | `static/socket.js` | WS connect with backoff; re-syncs on every (re)connect. | -| `static/wall-engine.js` | The actual player/DOM/YouTube-API engine (extracted from what used to be `player.js`): builds cells, pre-rolls, mute/audio targeting, zoom/pan, the context menu, the WS-driven config reconciliation. Exports `startWall({computeLayout})`; shared verbatim by `/` and `/layout`. | +| `static/wall-engine.js` | The actual player/DOM/YouTube-API engine (extracted from what used to be `player.js`): builds cells, pre-rolls, mute/audio targeting, zoom/pan, the context menu, the WS-driven config reconciliation. Exports `startWall({computeLayout, controlChannel})`; shared verbatim by `/` and `/layout`. Every interaction — a local DOM event or one relayed from `/layout-control` — funnels through the one `applyIntent()` dispatcher, and with `controlChannel` set it publishes a state snapshot over `BroadcastChannel` after each intent and on a 1s heartbeat. Both are opt-in: `/` passes no channel and behaves exactly as before. | | `static/player.js` | Bootstraps the `/` grid page via `wall-engine.js`'s `startWall()`. Two lines. | | `static/layout-fit.js` | Pure per-screen allocation/fit math for `/layout` — how a total video budget splits across the six real screens and how each screen's cells tile to approximate 16:9. No DOM, no fetch; node-tested like `grid-logic.js`. | | `static/layout-page.js` | Bootstraps `/layout`: loads `static/layout/screens.json`, builds a screens-based `computeLayout` from `layout-fit.js`'s `resolveLayout`, and calls `wall-engine.js`'s `startWall({computeLayout})`. | | `static/layout/screens.json` | Vendored, hand-copied snapshot of `../layout-driver/config/screens.yaml`'s geometry (canvas size, module size, per-screen grid/offset). No live link — see gotcha 39. | +| `static/layout-control.html` | The operator's window: the same header markup `/layout` now hides, plus a grid of plain rectangles — one per cell, no iframes. Loads no YouTube API at all. | +| `static/layout-control.js` | Drives that page. Relays every interaction to `/layout` as an intent over the `yt-matrix-layout-control` `BroadcastChannel`, and renders purely from the snapshots coming back — it fetches nothing and holds no truth of its own (gotcha 40). | | `static/config.js` | The live editor and its quota-cost indicator, including the Layout section (total/max_per_screen/per-screen overrides). | -| `scripts/build-dist.sh` | Assembles `dist/`: `player.html` → `index.html`, `config.html`, `layout.html`, `static/*.js`, `static/layout/screens.json`. Not the test `.mjs` files. | +| `scripts/build-dist.sh` | Assembles `dist/`: `player.html` → `index.html`, `config.html`, `layout.html`, `layout-control.html`, `static/*.js`, `static/layout/screens.json`. Not the test `.mjs` files. | | `tests/conftest.py` | Autouse guard: fails any default-suite test that reaches the live API. | ## Critical gotchas @@ -93,6 +95,18 @@ npm run deploy # needs Docker; see docs/DEPLOY.md or `resync()` — `resync()` also runs on every WebSocket reconnect, which is a network hiccup, not an intent. + **There is now a third trigger: a `newQuery` intent over the + `BroadcastChannel`**, sent by `/layout-control`'s own Query: button. It + reaches the same `requestNewQuery()` and spends the same 100 units. Two + things follow. The `generating` guard that stops a local double-click lives + in the button's `disabled` state, which a relayed click never sees — so + `applyIntent`'s `case "newQuery"` checks `generating` itself, and removing + that check restores a real double-spend. And `BroadcastChannel` is a genuine + broadcast, not a channel to one peer: **two open `/layout` tabs both act on + one control-page click**, which is two generations at 100 units each. One + broadcast page at a time is an operational rule, not something the code + enforces. + **The budget counter is an estimate, not a reading.** Google exposes no remaining-quota field on the YouTube Data API, and an API key cannot reach the Cloud quota APIs. `budget.py` assumes 100 units/search (documented) and @@ -282,11 +296,18 @@ npm run deploy # needs Docker; see docs/DEPLOY.md 14. **Browser bugs need a browser test.** #12 and #13 were both invisible to the Python suite and the node tests — one is script-ordering, the other a serialization gap between two languages. `tests/test_player_smoke.py` - (marked `browser`, covers `/`) and `tests/test_layout_smoke.py` (same - marker, covers `/layout`) are the only things that catch either. Run both - after any change to `wall-engine.js` — it drives both pages — and the - layout one after `layout-fit.js` or `layout-page.js` too. Run either after - a change to the config wire format. + (marked `browser`, covers `/`), `tests/test_layout_smoke.py` (same marker, + covers `/layout`) and `tests/test_layout_control_smoke.py` (same marker, + covers `/layout` and `/layout-control` together, two pages in one + `BrowserContext`) are the only things that catch either. Run **all three** + after any change to `wall-engine.js` — it drives all of it — the layout one + after `layout-fit.js` or `layout-page.js` too, and the control one after + `layout-control.js` or the snapshot/intent shapes. Run any of them after a + change to the config wire format. + + The control suite needs two pages in one context for a reason of its own: + `BroadcastChannel` connects tabs in the same browser profile and nothing + else, so a single-page test could not observe the feature at all. **The browser never loads `static/` — it loads `dist/`,** which `scripts/build-dist.sh` assembles by copying. Both the Worker's asset @@ -481,3 +502,95 @@ npm run deploy # needs Docker; see docs/DEPLOY.md *up front*, with no `onError` involved to catch a still and swap it out. Raising `total` well past `grid.cols × rows` therefore trades some of the motion filtering gotcha 16 exists for. + +40. **`/layout`'s chrome is `display: none`, and it must stay in the DOM.** + The NDI broadcaster captures that window, so the header and the context + menu may never paint — but `startWall()` opens with a dozen + `getElementById` lookups (`play`, `pause`, `mute`, `new-query`, `prompt`, + `follow`, `hover-unmute`, `status`, `audio`, `menu`, …) and every one of + them has to keep succeeding, because `/layout-control` drives those exact + elements remotely: an intent arrives, `applyIntent` runs the same code a + local click would, and the snapshot published afterwards is read straight + off `statusEl.textContent`, `muteButton.disabled` and friends. Delete the + markup instead of hiding it and those lookups return `null`, the engine + throws or silently no-ops partway through startup, and the wall renders + blank **with no console error** — the same class of invisible failure as + gotchas 12 and 13, and invisible to the Python and node suites for the same + reason (gotcha 14). Hiding is a CSS-only change on purpose; there is no + `/layout`-shaped branch anywhere in `wall-engine.js`. + + `#menu` is hidden alongside `header` for the same reason and by the same + rule. `contextmenu`'s `preventDefault()` only suppresses the *browser's* + menu; the app's own would still paint over the video and go out on the + feed. + + The direction of trust runs one way: `/layout` owns every piece of state + and `/layout-control` renders whatever the snapshot says. The control page + fetches nothing — no `/api/config`, no `/api/videos` — so there is no + second copy of the truth to drift. Giving it its own fetch to "make it + load faster" reintroduces exactly that. + +41. **A control-page gesture still starts and unmutes `/layout`, and this was + measured rather than assumed.** The worry is reasonable: Chrome's autoplay + policy gates audible playback on user activation, `BroadcastChannel` + confers no activation on the receiving document, and after `/layout`'s + header was hidden the buttons a human presses moved to a different + document entirely — so it looked like the wall might never be able to make + a sound again. + + It works. Probed against a real server with Playwright Chromium launched + **without** `--autoplay-policy=no-user-gesture-required`, headless and + headed, both identical: after pre-roll all eight players sat at state 2 + (paused, muted); a real click on `/layout-control`'s Play took all 8 to + state 1; a click on its Unmute took all 8 to `isMuted() === false` **and + left all 8 at state 1**; and with hover-to-unmute on, hovering cell 0 gave + `muted=[false, true × 7]` with every player still playing. + + The nuance that makes it safe: the policy gates *starting new audible + playback*, not clearing the mute flag on media that is already running. + `prerollCurrentSet()` (gotcha 18) starts every player muted, which needs no + gesture and is why it is written that way — so by the time any unmute is + relayed, the media element is already playing and past the gate. Gotcha 5's + "unmuting is only permitted off a user gesture" is about *starting* unmuted, + and remains true. + + `test_the_control_page_reflects_mute_state_from_the_broadcast_page`, in + `tests/test_layout_control_smoke.py`, is the standing proof: it launches with no + autoplay flag, plays and unmutes entirely from the control page, and + asserts the players are unmuted **and still at state 1**. That last + assertion is the point — a browser refusing the unmute could pause the + media instead of reporting muted. Do not add `--autoplay-policy` to that + test to "make it match the others": the absence of the flag is what it + tests. The menu test above it does use the flag, for the unrelated reason + that it needs playback started from the page's own script. + +42. **`POST /api/intent` is a second front door into `applyIntent()`, for a + controller that has no browser to share a `BroadcastChannel` with.** + `/layout-control` relays intents to `/layout` only because both are pages + in the *same* browser profile (gotcha 40's whole premise). An external + show-control/OSC router (e.g. `chasa`, in `../bardo-tools/chasa`) is a + separate process with no browser at all, so it goes in over HTTP instead: + `POST /api/intent` validates `type` against `WALL_WIDE_INTENT_TYPES` + (`ytmatrix/server.py`) and broadcasts `{"type": "intent", "intent": + payload}` over the same `/ws` connection `put_config` already uses for + config pushes. `wall-engine.js`'s socket `onMessage` calls `applyIntent()` + on it directly — one new branch, no new dispatch logic. + + Deliberately restricted to the wall-wide intents (`play`, `pause`, + `muteToggle`, `rewind`, `shuffle`, `resetView`, `newQuery`, + `hoverUnmuteToggle`, `followToggle`) — **not** the cell-indexed ones + (`cellWheel`, `cellDblclick`, `cellMenuAction`, `cellHoverEnter/Leave`, + `cellDragStart/Move/End`). Those carry an index tied to whichever page's + grid is open, and this broadcasts to *every* connected tab, `/` and + `/layout` alike — not just one wall. `newQuery` carries gotcha 29/2's own + quota-multiplication risk for the same reason: every open tab that + receives it independently spends its own 100-unit search. One wall/tab + open at a time when driving this from an external cue source is an + operational rule, not something the endpoint enforces. + + A Python REST client (chasa's `httpx`-based `rest` action) does not trust + this server's local mkcert certificate by default — `httpx` uses + `certifi`, not the macOS keychain, so it rejects the connection even + though a browser or `curl` accepts it. Confirmed live: setting + `SSL_CERT_FILE=$(mkcert -CAROOT)/rootCA.pem` in the caller's environment + fixes it with no code change on either side. diff --git a/docs/superpowers/plans/2026-08-29-layout-control-surface.md b/docs/superpowers/plans/2026-08-29-layout-control-surface.md new file mode 100644 index 0000000..c1f220d --- /dev/null +++ b/docs/superpowers/plans/2026-08-29-layout-control-surface.md @@ -0,0 +1,1693 @@ +# Layout Control Surface Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Add a second, ordinary browser tab (`/layout-control`) that mirrors `/layout`'s controls and per-cell interactions over `BroadcastChannel`, so an operator can drive the broadcast-clean `/layout` page (header hidden by CSS) without touching it directly. + +**Architecture:** Every local event handler in `wall-engine.js` is refactored to construct a small "intent" object and hand it to one `applyIntent()` dispatcher, instead of mutating state inline — the same function then runs whether the intent came from a real DOM event or arrived over a `BroadcastChannel`. `wall-engine.js` also gains an opt-in snapshot publisher (per-cell + global state) so a control page can render without any state of its own. `/layout-control` is a pure consumer: no independent fetch of config or geometry, everything comes from snapshots. + +**Tech Stack:** Vanilla JS ES modules (as the rest of `static/`), `BroadcastChannel` (native browser API, no library), Playwright for the two-page browser test. + +**Spec:** `docs/superpowers/specs/2026-08-29-layout-control-surface-design.md` + +## Global Constraints + +- **Zero behavior change on `/` and on `/layout` with no control tab open.** `tests/test_player_smoke.py` and `tests/test_layout_smoke.py` (both marked `browser`) must pass, unmodified, after every task in this plan. +- **`BroadcastChannel` only, no server involvement.** It connects tabs in the same browser/profile/machine, which is already required for the NDI broadcaster to work. No new backend route carries control traffic. +- Coordinates in intents that describe a pointer position are normalized (0..1, relative to the *sending* window's own cell rect) — the receiving side always converts using its own `getBoundingClientRect()`. Wheel `deltaY` travels as-is (only its sign matters to `nextZoom`). +- Every mutating context-menu action becomes a `cellMenuAction` intent; every read-only one (copy/open) executes locally wherever it was clicked, using cached snapshot data. +- `uv run ruff check . && uv run ruff format .` is not applicable to the JS-only tasks in this plan (no Python touched until none) but `node --check ` must be run on every JS file created or modified, and the two existing browser suites plus the new one must all pass before every commit that touches `wall-engine.js`. + +--- + +### Task 1: Extract named global-action functions + +**Files:** +- Modify: `static/wall-engine.js` + +**Interfaces:** +- Consumes: nothing new. +- Produces: `rewindAll()`, `toggleMute()`, `shuffleWall()`, `resetAllViews()` — each a pure extraction of an existing inline listener body, called by that same listener and by nothing else yet (Task 3 wires them into `applyIntent`). + +- [ ] **Step 1: Extract `rewindAll()`** + +Replace: +```js +rewindButton.addEventListener("click", () => { + for (const player of livePlayers()) { + try { + player.seekTo(config.playback.start_offset, true); + // seekTo resumes a player that is not already paused, so a paused wall + // would quietly start playing. Put it back. + if (!wantPlaying) player.pauseVideo(); + } catch { + // A player mid-teardown; skip it. + } + } +}); +``` +with: +```js +function rewindAll() { + for (const player of livePlayers()) { + try { + player.seekTo(config.playback.start_offset, true); + // seekTo resumes a player that is not already paused, so a paused wall + // would quietly start playing. Put it back. + if (!wantPlaying) player.pauseVideo(); + } catch { + // A player mid-teardown; skip it. + } + } +} + +rewindButton.addEventListener("click", rewindAll); +``` + +- [ ] **Step 2: Extract `toggleMute()`** + +Replace: +```js +muteButton.addEventListener("click", () => { + muted = !muted; + refreshMuteButton(); + // Unmuting eight players at once is only permitted off a user gesture -- + // this click is it. Doing it any other way leaves some players silent. + applyMuteStateToAll(); +}); +``` +with: +```js +function toggleMute() { + muted = !muted; + refreshMuteButton(); + // Unmuting eight players at once is only permitted off a user gesture -- + // this click is it. Doing it any other way leaves some players silent. + applyMuteStateToAll(); +} + +muteButton.addEventListener("click", toggleMute); +``` + +- [ ] **Step 3: Extract `shuffleWall()`** + +Replace: +```js +shuffleButton.addEventListener("click", () => { + const wasPlaying = wantPlaying; + slotState = shuffleSlots( + [...slotState.slots, ...slotState.reserves], + computeLayout(config).totalCells, + ); + rebuild(); + // Shuffling is not a new query, so it should not silently stop the wall. + // rebuild() clears wantPlaying via pre-roll; put it back if it was running. + wantPlaying = wasPlaying; +}); +``` +with: +```js +function shuffleWall() { + const wasPlaying = wantPlaying; + slotState = shuffleSlots( + [...slotState.slots, ...slotState.reserves], + computeLayout(config).totalCells, + ); + rebuild(); + // Shuffling is not a new query, so it should not silently stop the wall. + // rebuild() clears wantPlaying via pre-roll; put it back if it was running. + wantPlaying = wasPlaying; +} + +shuffleButton.addEventListener("click", shuffleWall); +``` + +- [ ] **Step 4: Extract `resetAllViews()`** + +Replace: +```js +resetViewButton.addEventListener("click", () => { + views.clear(); + for (const cell of gridEl.children) { + delete cell.dataset.zoomed; + applyCoverFit(cell); + } +}); +``` +with: +```js +function resetAllViews() { + views.clear(); + for (const cell of gridEl.children) { + delete cell.dataset.zoomed; + applyCoverFit(cell); + } +} + +resetViewButton.addEventListener("click", resetAllViews); +``` + +- [ ] **Step 5: Verify no behavior change** + +```bash +node --check static/wall-engine.js +node --test 'static/*.test.mjs' +uv run pytest tests/test_player_smoke.py -m browser -v +uv run pytest tests/test_layout_smoke.py -m browser -v +``` +Expected: identical pass counts to before this task (142 node tests unaffected since none touch `wall-engine.js` directly, 40 browser tests for `/`, 3 for `/layout`). + +- [ ] **Step 6: Commit** + +```bash +git add static/wall-engine.js +git commit -m "refactor: name the global header-control actions" +``` + +--- + +### Task 2: Extract named per-cell action functions + +**Files:** +- Modify: `static/wall-engine.js` + +**Interfaces:** +- Consumes: nothing new. +- Produces: `togglePlayForCell(index)`, `toggleLockedIndex(index)`, `restartCell(index)`, `resetCellZoom(index)` — each an extraction of logic currently duplicated or inlined between the `dblclick` listener and the context menu's `run` callbacks. + +- [ ] **Step 1: Add the four functions** + +Insert immediately before `function playerForCell(cell) {`: + +```js +function togglePlayForCell(index) { + const player = players[index]; + let state = -1; + try { + state = player?.getPlayerState?.() ?? -1; + } catch { + // A player mid-teardown; nothing useful to do. + } + if (state === 1) player?.pauseVideo?.(); + else player?.playVideo?.(); +} + +function toggleLockedIndex(index) { + lockedIndex = lockedIndex === index ? null : index; + for (const other of gridEl.children) delete other.dataset.locked; + const cell = gridEl.children[index]; + if (lockedIndex !== null && cell) cell.dataset.locked = "true"; + applyMuteStateToAll(); +} + +function restartCell(index) { + players[index]?.seekTo?.(config.playback.start_offset, true); +} + +function resetCellZoom(index) { + views.delete(index); + const cell = gridEl.children[index]; + if (!cell) return; + cell.dataset.zoomed = "false"; + applyCoverFit(cell); +} +``` + +- [ ] **Step 2: Use them in the context menu** + +In `menuItems(cell)`, replace the four `run` callbacks: + +```js +// Old: + { + label: playing ? "Pause this cell" : "Play this cell", + run: () => (playing ? player?.pauseVideo?.() : player?.playVideo?.()), + }, +``` +```js +// New: + { + label: playing ? "Pause this cell" : "Play this cell", + run: () => togglePlayForCell(index), + }, +``` + +```js +// Old: + { + label: lockedIndex === index ? "Unlock audio" : "Lock audio to this cell", + hint: "double-click", + run: () => { + lockedIndex = lockedIndex === index ? null : index; + for (const other of gridEl.children) delete other.dataset.locked; + if (lockedIndex !== null) cell.dataset.locked = "true"; + applyMuteStateToAll(); + }, + }, +``` +```js +// New: + { + label: lockedIndex === index ? "Unlock audio" : "Lock audio to this cell", + hint: "double-click", + run: () => toggleLockedIndex(index), + }, +``` + +```js +// Old: + { + label: "Restart this cell", + run: () => player?.seekTo?.(config.playback.start_offset, true), + }, +``` +```js +// New: + { + label: "Restart this cell", + run: () => restartCell(index), + }, +``` + +```js +// Old: + { + label: "Reset zoom", + hint: `${(viewFor(cell).zoom ?? 1).toFixed(2)}×`, + run: () => { + views.delete(index); + cell.dataset.zoomed = "false"; + applyCoverFit(cell); + }, + }, +``` +```js +// New: + { + label: "Reset zoom", + hint: `${(viewFor(cell).zoom ?? 1).toFixed(2)}×`, + run: () => resetCellZoom(index), + }, +``` + +- [ ] **Step 3: Use `toggleLockedIndex` in the dblclick listener** + +Replace: +```js +gridEl.addEventListener("dblclick", (event) => { + const cell = event.target.closest(".cell"); + if (!cell || cell.dataset.empty === "true") return; + const index = [...gridEl.children].indexOf(cell); + lockedIndex = lockedIndex === index ? null : index; + + for (const other of gridEl.children) delete other.dataset.locked; + if (lockedIndex !== null) cell.dataset.locked = "true"; + applyMuteStateToAll(); +}); +``` +with: +```js +gridEl.addEventListener("dblclick", (event) => { + const cell = event.target.closest(".cell"); + if (!cell || cell.dataset.empty === "true") return; + toggleLockedIndex([...gridEl.children].indexOf(cell)); +}); +``` + +- [ ] **Step 4: Verify no behavior change** + +```bash +node --check static/wall-engine.js +node --test 'static/*.test.mjs' +uv run pytest tests/test_player_smoke.py -m browser -v +uv run pytest tests/test_layout_smoke.py -m browser -v +``` +Expected: same pass counts as Task 1's verification — in particular `test_double_click_locks_audio_to_one_cell`, `test_double_clicking_another_cell_moves_the_lock`, `test_double_clicking_the_same_cell_turns_the_lock_off`, `test_reset_zoom_is_offered_and_works` and `test_right_click_offers_copy_url_at_time` in `tests/test_player_smoke.py` all still pass — they exercise exactly the code this task moved. + +- [ ] **Step 5: Commit** + +```bash +git add static/wall-engine.js +git commit -m "refactor: name the per-cell actions shared by dblclick and the context menu" +``` + +--- + +### Task 3: `applyIntent()` — the single dispatch point + +This is the highest-risk task in the plan: every local listener changes shape. The binding requirement is that the *effect* of every listener is unchanged; only how it's invoked changes (directly, or through one more function call). + +**Files:** +- Modify: `static/wall-engine.js` + +**Interfaces:** +- Consumes: `rewindAll`, `toggleMute`, `shuffleWall`, `resetAllViews` (Task 1); `togglePlayForCell`, `toggleLockedIndex`, `restartCell`, `resetCellZoom` (Task 2). +- Produces: `applyIntent(intent)` (async), `queueZoom(cell, index, width, height, deltaY, x, y)`, `applyCellWheel(index, deltaY, xFraction, yFraction)`, `applyCellPan(index, dxFraction, dyFraction)`, `applyCellMenuAction(index, action)`. The full intent vocabulary is exactly the one in the design spec's "Intents" section. + +- [ ] **Step 1: Extract `queueZoom` from the wheel listener, and add the three new dispatch helpers** + +Insert immediately before `gridEl.addEventListener("wheel", ...)` (i.e., right after `let pendingZoom = null;`): + +```js +function queueZoom(cell, index, width, height, deltaY, x, y) { + if (pendingZoom && pendingZoom.index === index) { + // Same cell, same frame: sum the deltas so no scrolling is lost, and + // track the cursor to wherever it ended up. + pendingZoom.deltaY += deltaY; + pendingZoom.x = x; + pendingZoom.y = y; + return; + } + // A different cell mid-frame: land the queued one first rather than + // dropping it. + if (pendingZoom) flushZoom(); + pendingZoom = { cell, index, width, height, deltaY, x, y }; + requestAnimationFrame(flushZoom); +} + +// x, y arrive normalized (0..1) -- from a real local event on THIS page's own +// cell, or relayed from /layout-control's differently-sized rectangle for +// the same cell. Either way this converts to this page's own real pixel +// space before reusing the exact zoom math a local wheel event already used. +function applyCellWheel(index, deltaY, xFraction, yFraction) { + const cell = gridEl.children[index]; + if (!cell || cell.dataset.empty === "true") return; + const bounds = cell.getBoundingClientRect(); + queueZoom(cell, index, bounds.width, bounds.height, deltaY, xFraction * bounds.width, yFraction * bounds.height); +} + +// dx, dy arrive as a fraction of the SENDER's own cell size (already an +// incremental delta, not an absolute position) -- rescaling by this page's +// own bounds is what makes a drag feel proportionally the same regardless of +// how big /layout-control's rectangle happens to be. +function applyCellPan(index, dxFraction, dyFraction) { + const cell = gridEl.children[index]; + if (!cell) return; + const bounds = cell.getBoundingClientRect(); + views.set( + index, + panBy(views.get(index) ?? IDENTITY_VIEW, dxFraction * bounds.width, dyFraction * bounds.height), + ); + applyCoverFit(cell); +} + +function applyCellMenuAction(index, action) { + switch (action) { + case "togglePlay": + togglePlayForCell(index); + break; + case "toggleLock": + toggleLockedIndex(index); + break; + case "restart": + restartCell(index); + break; + case "resetZoom": + resetCellZoom(index); + break; + case "replaceReserve": + handlePlayerError(index); + break; + default: + wlog(`applyCellMenuAction: unknown action ${action}`); + } +} + +// Every user action reachable from the header or a cell -- a real DOM event +// on THIS page, or one relayed from /layout-control over BroadcastChannel -- +// funnels through here. That symmetry is the whole point: a control-window +// message and a local click must produce identical effects, so this is the +// only place either kind is handled. publishSnapshot() (added when +// BroadcastChannel wiring lands) is a no-op until then. +async function applyIntent(intent) { + switch (intent.type) { + case "play": + startAll(); + break; + case "pause": + pauseAll(); + break; + case "muteToggle": + toggleMute(); + break; + case "rewind": + rewindAll(); + break; + case "shuffle": + shuffleWall(); + break; + case "resetView": + resetAllViews(); + break; + case "newQuery": + await requestNewQuery(intent.prompt ?? null); + break; + case "hoverUnmuteToggle": + hoverUnmuteCheckbox.checked = intent.checked; + setAudibleCell(null); + applyMuteStateToAll(); + break; + case "followToggle": + followCheckbox.checked = intent.checked; + break; + case "cellHoverEnter": + if (hoverUnmuteCheckbox.checked) setAudibleCell(intent.index); + break; + case "cellHoverLeave": + if (hoverUnmuteCheckbox.checked) setAudibleCell(null); + break; + case "cellWheel": + applyCellWheel(intent.index, intent.deltaY, intent.x, intent.y); + break; + case "cellDragStart": + // Purely a cursor affordance; kept for parity with a locally-driven drag. + gridEl.children[intent.index]?.setAttribute("data-dragging", "true"); + break; + case "cellDragMove": + applyCellPan(intent.index, intent.dx, intent.dy); + break; + case "cellDragEnd": + gridEl.children[intent.index]?.removeAttribute("data-dragging"); + break; + case "cellDblclick": + toggleLockedIndex(intent.index); + break; + case "cellMenuAction": + applyCellMenuAction(intent.index, intent.action); + break; + default: + wlog(`applyIntent: unknown intent type ${intent.type}`); + return; + } +} +``` + +- [ ] **Step 2: Rewire the wheel listener** + +Replace: +```js +gridEl.addEventListener( + "wheel", + (event) => { + const cell = event.target.closest(".cell"); + if (!cell || cell.dataset.empty === "true") return; + event.preventDefault(); + + const index = [...gridEl.children].indexOf(cell); + const bounds = cell.getBoundingClientRect(); + const x = event.clientX - bounds.left; + const y = event.clientY - bounds.top; + + if (pendingZoom && pendingZoom.index === index) { + // Same cell, same frame: sum the deltas so no scrolling is lost, and + // track the cursor to wherever it ended up. + pendingZoom.deltaY += event.deltaY; + pendingZoom.x = x; + pendingZoom.y = y; + return; + } + + // A different cell mid-frame: land the queued one first rather than + // dropping it. + if (pendingZoom) flushZoom(); + + pendingZoom = { + cell, + index, + width: bounds.width, + height: bounds.height, + deltaY: event.deltaY, + x, + y, + }; + requestAnimationFrame(flushZoom); + }, + { passive: false }, +); +``` +with: +```js +gridEl.addEventListener( + "wheel", + (event) => { + const cell = event.target.closest(".cell"); + if (!cell || cell.dataset.empty === "true") return; + event.preventDefault(); + const index = [...gridEl.children].indexOf(cell); + const bounds = cell.getBoundingClientRect(); + applyIntent({ + type: "cellWheel", + index, + deltaY: event.deltaY, + x: bounds.width ? (event.clientX - bounds.left) / bounds.width : 0, + y: bounds.height ? (event.clientY - bounds.top) / bounds.height : 0, + }); + }, + { passive: false }, +); +``` + +- [ ] **Step 3: Rewire drag (pointerdown / pointermove / endDrag)** + +Replace: +```js +gridEl.addEventListener("pointerdown", (event) => { + if (event.button !== 0) return; // left button only; right opens the menu + const cell = event.target.closest(".cell"); + if (!cell || cell.dataset.empty === "true") return; + drag = { + cell, + index: [...gridEl.children].indexOf(cell), + x: event.clientX, + y: event.clientY, + }; + cell.dataset.dragging = "true"; + try { + cell.setPointerCapture(event.pointerId); + } catch { + // Capture is a convenience; the document-level pointerup still ends it. + } +}); + +gridEl.addEventListener("pointermove", (event) => { + if (!drag) return; + const dx = event.clientX - drag.x; + const dy = event.clientY - drag.y; + drag.x = event.clientX; + drag.y = event.clientY; + views.set(drag.index, panBy(views.get(drag.index) ?? IDENTITY_VIEW, dx, dy)); + applyCoverFit(drag.cell); +}); + +function endDrag(event) { + if (!drag) return; + try { + drag.cell.releasePointerCapture(event.pointerId); + } catch { + // Already released, or never captured. + } + delete drag.cell.dataset.dragging; + drag = null; +} +``` +with: +```js +gridEl.addEventListener("pointerdown", (event) => { + if (event.button !== 0) return; // left button only; right opens the menu + const cell = event.target.closest(".cell"); + if (!cell || cell.dataset.empty === "true") return; + const index = [...gridEl.children].indexOf(cell); + drag = { cell, index, x: event.clientX, y: event.clientY }; + try { + cell.setPointerCapture(event.pointerId); + } catch { + // Capture is a convenience; the document-level pointerup still ends it. + } + applyIntent({ type: "cellDragStart", index }); +}); + +gridEl.addEventListener("pointermove", (event) => { + if (!drag) return; + const dx = event.clientX - drag.x; + const dy = event.clientY - drag.y; + const bounds = drag.cell.getBoundingClientRect(); + drag.x = event.clientX; + drag.y = event.clientY; + applyIntent({ + type: "cellDragMove", + index: drag.index, + dx: bounds.width ? dx / bounds.width : 0, + dy: bounds.height ? dy / bounds.height : 0, + }); +}); + +function endDrag(event) { + if (!drag) return; + try { + drag.cell.releasePointerCapture(event.pointerId); + } catch { + // Already released, or never captured. + } + applyIntent({ type: "cellDragEnd", index: drag.index }); + drag = null; +} +``` + +- [ ] **Step 4: Rewire dblclick and hover** + +Replace: +```js +gridEl.addEventListener("dblclick", (event) => { + const cell = event.target.closest(".cell"); + if (!cell || cell.dataset.empty === "true") return; + toggleLockedIndex([...gridEl.children].indexOf(cell)); +}); + +gridEl.addEventListener("pointerover", (event) => { + if (!hoverUnmuteCheckbox.checked) return; + const cell = event.target.closest(".cell"); + if (!cell || cell.dataset.empty === "true") return; + setAudibleCell([...gridEl.children].indexOf(cell)); +}); + +// pointerleave on the grid, not per cell: moving between adjacent cells would +// otherwise blip the audio off and on again between them. +gridEl.addEventListener("pointerleave", () => { + if (!hoverUnmuteCheckbox.checked) return; + setAudibleCell(null); +}); +``` +with: +```js +gridEl.addEventListener("dblclick", (event) => { + const cell = event.target.closest(".cell"); + if (!cell || cell.dataset.empty === "true") return; + applyIntent({ type: "cellDblclick", index: [...gridEl.children].indexOf(cell) }); +}); + +gridEl.addEventListener("pointerover", (event) => { + if (!hoverUnmuteCheckbox.checked) return; + const cell = event.target.closest(".cell"); + if (!cell || cell.dataset.empty === "true") return; + applyIntent({ type: "cellHoverEnter", index: [...gridEl.children].indexOf(cell) }); +}); + +// pointerleave on the grid, not per cell: moving between adjacent cells would +// otherwise blip the audio off and on again between them. +gridEl.addEventListener("pointerleave", () => { + if (!hoverUnmuteCheckbox.checked) return; + applyIntent({ type: "cellHoverLeave" }); +}); +``` + +- [ ] **Step 5: Rewire the header buttons and checkboxes** + +Replace: +```js +playButton.addEventListener("click", startAll); +pauseButton.addEventListener("click", pauseAll); + +rewindButton.addEventListener("click", rewindAll); + +muteButton.addEventListener("click", toggleMute); +``` +with: +```js +playButton.addEventListener("click", () => applyIntent({ type: "play" })); +pauseButton.addEventListener("click", () => applyIntent({ type: "pause" })); +rewindButton.addEventListener("click", () => applyIntent({ type: "rewind" })); +muteButton.addEventListener("click", () => applyIntent({ type: "muteToggle" })); +``` + +Replace: +```js +hoverUnmuteCheckbox.addEventListener("change", () => { + // Leaving the mode must not strand a cell audible or the whole wall silent. + setAudibleCell(null); + applyMuteStateToAll(); +}); +``` +with: +```js +hoverUnmuteCheckbox.addEventListener("change", () => + applyIntent({ type: "hoverUnmuteToggle", checked: hoverUnmuteCheckbox.checked }), +); +``` + +Replace: +```js +shuffleButton.addEventListener("click", shuffleWall); +``` +with: +```js +shuffleButton.addEventListener("click", () => applyIntent({ type: "shuffle" })); +``` + +Replace: +```js +resetViewButton.addEventListener("click", resetAllViews); +``` +with: +```js +resetViewButton.addEventListener("click", () => applyIntent({ type: "resetView" })); +``` + +Replace: +```js +newQueryButton.addEventListener("click", async () => { + const prompt = promptInput.value.trim(); + promptInput.disabled = true; + try { + await requestNewQuery(prompt || null); + } finally { + promptInput.disabled = false; + } +}); +``` +with: +```js +newQueryButton.addEventListener("click", async () => { + const prompt = promptInput.value.trim(); + promptInput.disabled = true; + try { + await applyIntent({ type: "newQuery", prompt: prompt || null }); + } finally { + promptInput.disabled = false; + } +}); +``` + +`followCheckbox` has no existing listener to replace — this task does not add one (nothing currently reacts live to it; `/layout-control`'s own checkbox will drive it via a `followToggle` intent once Task 6 exists, and `applyIntent`'s `followToggle` case already handles that from Step 1). + +- [ ] **Step 6: Verify no behavior change** + +```bash +node --check static/wall-engine.js +node --test 'static/*.test.mjs' +uv run pytest tests/ -v +uv run pytest tests/test_player_smoke.py -m browser -v +uv run pytest tests/test_layout_smoke.py -m browser -v +``` +Expected: every suite passes at its pre-existing count — this is the proof that routing every listener through one more function call changed nothing observable. Pay particular attention to `test_scroll_wheel_zooms_toward_the_pointer`, `test_zooming_out_pulls_back_to_the_whole_frame`, `test_drag_pans_a_zoomed_cell`, and `test_dragging_cannot_open_a_gap` — these exercise exactly the normalize-then-rescale round trip Step 2/3 introduced, and on `/` that round trip must be a no-op (dividing and re-multiplying by the same cell's own bounds). + +- [ ] **Step 7: Commit** + +```bash +git add static/wall-engine.js +git commit -m "refactor: route every local interaction through one applyIntent dispatcher" +``` + +--- + +### Task 4: Snapshot publishing + `BroadcastChannel` wiring + +**Files:** +- Modify: `static/wall-engine.js` + +**Interfaces:** +- Consumes: `applyIntent` (Task 3), `computeLayout` (existing). +- Produces: `startWall({ computeLayout, controlChannel })` — `controlChannel` is `null` by default (nothing new activates) or a channel-name string. `buildSnapshot()`, `publishSnapshot()`. + +- [ ] **Step 1: Add the option and the snapshot machinery** + +Change the `startWall` signature: +```js +// Old: +export function startWall({ computeLayout = defaultComputeLayout } = {}) { +``` +```js +// New: +export function startWall({ computeLayout = defaultComputeLayout, controlChannel = null } = {}) { +``` + +Insert, immediately before `function refreshControls() {`: + +```js +// null until startWall({ controlChannel }) opens one -- publishSnapshot() +// below is a no-op until then, so every call site can call it unconditionally +// without checking whether a control page exists. +let broadcastChannel = null; + +function buildSnapshot() { + const layout = computeLayout(config); + const cells = slotState.slots.map((videoId, index) => { + const player = players[index]; + let currentTime = 0; + let playing = false; + try { + currentTime = player?.getCurrentTime?.() ?? 0; + playing = (player?.getPlayerState?.() ?? -1) === 1; + } catch { + // A player mid-teardown; snapshot with the defaults above. + } + return { + index, + videoId: videoId ?? null, + title: videoId ? (titles.get(videoId) ?? videoId) : null, + empty: !videoId, + zoom: views.get(index)?.zoom ?? 1, + locked: lockedIndex === index, + audible: isAudible(index, currentAudioTarget()), + playing, + currentTime, + // Percentages, same object shape buildCells() already applies to this + // page's own cell -- /layout-control positions its rectangle from this + // directly, so the two pages can never disagree about geometry. + rect: layout.cellRect ? layout.cellRect(index) : null, + }; + }); + + return { + type: "snapshot", + global: { + status: statusEl.textContent, + statusState: statusEl.dataset.state ?? "", + audioIndicatorText: audioEl.textContent, + audioLocked: audioEl.dataset.locked === "true", + muted, + prerolled, + wantPlaying, + playDisabled: playButton.disabled, + hoverUnmuteChecked: hoverUnmuteCheckbox.checked, + followChecked: followCheckbox.checked, + newQueryVisible: !newQueryButton.hidden, + newQueryDisabled: newQueryButton.disabled, + reservesLeft: slotState.reserves.length, + }, + cells, + }; +} + +function publishSnapshot() { + if (!broadcastChannel) return; + try { + broadcastChannel.postMessage(buildSnapshot()); + } catch { + // A channel can throw if it has already been closed; nothing useful to do. + } +} +``` + +- [ ] **Step 2: Publish after the moments that matter, plus a heartbeat** + +At the end of `applyIntent`'s switch (immediately before its closing `}`), i.e. change: +```js +// Old: + default: + wlog(`applyIntent: unknown intent type ${intent.type}`); + return; + } +} +``` +```js +// New: + default: + wlog(`applyIntent: unknown intent type ${intent.type}`); + return; + } + publishSnapshot(); +} +``` + +At the end of `finishPreroll`, change: +```js +// Old: + // A new set starts paused unless the user asked it to follow the play state. + if (wantPlaying && followCheckbox.checked) { + startAll(); + } else { + wantPlaying = false; + refreshControls(); + setStatus(`${statusPrefix} · ready — press Play`); + } +} +``` +```js +// New: + // A new set starts paused unless the user asked it to follow the play state. + if (wantPlaying && followCheckbox.checked) { + startAll(); + } else { + wantPlaying = false; + refreshControls(); + setStatus(`${statusPrefix} · ready — press Play`); + } + publishSnapshot(); +} +``` + +- [ ] **Step 3: Open the channel and subscribe, only when asked** + +Immediately before the final `connectSocket({` call at the bottom of `startWall`, insert: + +```js +if (controlChannel) { + broadcastChannel = new BroadcastChannel(controlChannel); + broadcastChannel.addEventListener("message", (event) => { + applyIntent(event.data); + }); + // A heartbeat, not the only source of truth: applyIntent and finishPreroll + // already publish on every discrete change. This just guarantees a + // late-joining or reconnecting control tab is never more than a second + // stale, without instrumenting every low-level mutation site. + setInterval(publishSnapshot, 1000); +} + +``` + +- [ ] **Step 4: Verify no behavior change on `/` or default `/layout`** + +Neither `player.js` nor `layout-page.js` passes `controlChannel` yet (that's Task 5), so `broadcastChannel` stays `null` everywhere in the app today and `publishSnapshot()` is a no-op. + +```bash +node --check static/wall-engine.js +node --test 'static/*.test.mjs' +uv run pytest tests/ -v +uv run pytest tests/test_player_smoke.py -m browser -v +uv run pytest tests/test_layout_smoke.py -m browser -v +``` +Expected: identical pass counts to Task 3's verification. + +- [ ] **Step 5: Commit** + +```bash +git add static/wall-engine.js +git commit -m "feat: opt-in BroadcastChannel snapshot publishing and intent intake" +``` + +--- + +### Task 5: `/layout` hides its chrome and opts in + +**Files:** +- Modify: `static/layout.html` +- Modify: `static/layout-page.js` + +**Interfaces:** +- Consumes: `startWall`'s `controlChannel` option (Task 4). +- Produces: nothing consumed elsewhere in this plan except the literal channel name string, which Task 6 must match exactly: `"yt-matrix-layout-control"`. + +- [ ] **Step 1: Hide the header, keep it in the DOM** + +In `static/layout.html`'s ` + + +
+ + + + + + + + + + + + loading… + ← broadcast +
+
+ + + + diff --git a/static/layout-control.js b/static/layout-control.js new file mode 100644 index 0000000..23704d3 --- /dev/null +++ b/static/layout-control.js @@ -0,0 +1,327 @@ +import { videoUrl, formatTimecode } from "./grid-logic.js"; + +const CHANNEL_NAME = "yt-matrix-layout-control"; +const STALE_AFTER_MS = 2500; + +const channel = new BroadcastChannel(CHANNEL_NAME); + +const gridEl = document.getElementById("grid"); +const statusEl = document.getElementById("status"); +const playButton = document.getElementById("play"); +const pauseButton = document.getElementById("pause"); +const muteButton = document.getElementById("mute"); +const newQueryButton = document.getElementById("new-query"); +const followCheckbox = document.getElementById("follow"); +const promptInput = document.getElementById("prompt"); +const hoverUnmuteCheckbox = document.getElementById("hover-unmute"); +const menuEl = document.getElementById("menu"); +const resetViewButton = document.getElementById("reset-view"); +const shuffleButton = document.getElementById("shuffle"); +const audioEl = document.getElementById("audio"); +const rewindButton = document.getElementById("rewind"); + +function send(intent) { + channel.postMessage(intent); +} + +// How long a message this page wrote itself ("copied URL", "clipboard +// blocked") is protected from the next snapshot. /layout's heartbeat arrives +// every second, so without this window a copy confirmation can be painted over +// before it has been read -- and it is the only feedback the action gives. +const LOCAL_STATUS_HOLD_MS = 1500; + +let localStatusAt = 0; + +function setStatus(text, state = "") { + statusEl.textContent = text; + statusEl.dataset.state = state; +} + +// Status this page owns, rather than /layout's own line relayed through. +function setLocalStatus(text, state = "") { + localStatusAt = Date.now(); + setStatus(text, state); +} + +// BroadcastChannel only bridges tabs/windows within the SAME browser process +// and profile -- it cannot reach a /layout running in a different browser, a +// different profile, or (the production case) the broadcaster's own headed- +// but-off-screen Chrome instance. Say so loudly here, since the only other +// symptom is this page sitting on "waiting" forever with nothing to click. +console.log( + `[layout-control] connecting on BroadcastChannel("${CHANNEL_NAME}"). ` + + "This only reaches a /layout tab open in this SAME browser process/profile " + + "-- not a different browser, a different profile, or the NDI broadcaster's " + + "own off-screen Chrome instance, which is a separate process even if it is " + + "the same browser application.", +); +setStatus("waiting for /layout to connect… (must be open in this same browser)", "busy"); + +// The last snapshot's cells, kept only so the context menu can be built +// without a round trip -- every mutating action still goes back over the +// channel as an intent. +let latestCells = []; +// The last snapshot's `global` block, for the same reason: the reserve count +// the menu reports lives there. +let latestGlobal = {}; +let staleTimer = null; + +function renderFromSnapshot(snapshot) { + latestCells = snapshot.cells; + latestGlobal = snapshot.global; + const g = snapshot.global; + if (Date.now() - localStatusAt >= LOCAL_STATUS_HOLD_MS) setStatus(g.status, g.statusState); + audioEl.textContent = g.audioIndicatorText; + audioEl.dataset.locked = String(g.audioLocked); + muteButton.textContent = g.muted ? "Unmute" : "Mute"; + muteButton.dataset.muted = String(g.muted); + playButton.disabled = g.playDisabled; + hoverUnmuteCheckbox.checked = g.hoverUnmuteChecked; + followCheckbox.checked = g.followChecked; + newQueryButton.hidden = !g.newQueryVisible; + newQueryButton.disabled = g.newQueryDisabled; + + if (gridEl.children.length !== snapshot.cells.length) { + gridEl.replaceChildren(); + for (let i = 0; i < snapshot.cells.length; i += 1) { + const cell = document.createElement("div"); + cell.className = "cell"; + const label = document.createElement("span"); + label.className = "label"; + cell.appendChild(label); + gridEl.appendChild(cell); + } + } + snapshot.cells.forEach((cellData, index) => { + const cell = gridEl.children[index]; + if (!cell) return; + if (cellData.rect) Object.assign(cell.style, cellData.rect); + cell.dataset.empty = cellData.empty ? "true" : "false"; + cell.dataset.audible = String(cellData.audible); + cell.dataset.locked = String(cellData.locked); + if (cellData.videoId) cell.dataset.videoId = cellData.videoId; + else delete cell.dataset.videoId; + cell.querySelector(".label").textContent = cellData.title ?? ""; + }); + + clearTimeout(staleTimer); + staleTimer = setTimeout(() => { + setStatus( + "no update from /layout in a while — is it open in this same browser?", + "error", + ); + }, STALE_AFTER_MS); +} + +channel.addEventListener("message", (event) => { + if (event.data?.type === "snapshot") renderFromSnapshot(event.data); +}); + +playButton.addEventListener("click", () => send({ type: "play" })); +pauseButton.addEventListener("click", () => send({ type: "pause" })); +muteButton.addEventListener("click", () => send({ type: "muteToggle" })); +rewindButton.addEventListener("click", () => send({ type: "rewind" })); +shuffleButton.addEventListener("click", () => send({ type: "shuffle" })); +resetViewButton.addEventListener("click", () => send({ type: "resetView" })); +hoverUnmuteCheckbox.addEventListener("change", () => + send({ type: "hoverUnmuteToggle", checked: hoverUnmuteCheckbox.checked }), +); +followCheckbox.addEventListener("change", () => + send({ type: "followToggle", checked: followCheckbox.checked }), +); +promptInput.addEventListener("keydown", (event) => { + if (event.key !== "Enter" || newQueryButton.disabled || newQueryButton.hidden) return; + event.preventDefault(); + newQueryButton.click(); +}); +newQueryButton.addEventListener("click", () => { + const prompt = promptInput.value.trim(); + // Disable optimistically. /layout's dispatcher is the real guard against a + // double 100-unit spend, but its answer only arrives with the next snapshot + // -- up to a heartbeat away -- and a button that stays live for a second + // after a click invites the second click. + newQueryButton.disabled = true; + send({ type: "newQuery", prompt: prompt || null }); +}); + +function cellIndexOf(target) { + const cell = target.closest(".cell"); + return cell ? [...gridEl.children].indexOf(cell) : -1; +} + +gridEl.addEventListener("pointerover", (event) => { + const index = cellIndexOf(event.target); + if (index >= 0) send({ type: "cellHoverEnter", index }); +}); +gridEl.addEventListener("pointerleave", () => send({ type: "cellHoverLeave" })); + +gridEl.addEventListener( + "wheel", + (event) => { + const cell = event.target.closest(".cell"); + if (!cell) return; + event.preventDefault(); + const bounds = cell.getBoundingClientRect(); + send({ + type: "cellWheel", + index: [...gridEl.children].indexOf(cell), + deltaY: event.deltaY, + x: bounds.width ? (event.clientX - bounds.left) / bounds.width : 0, + y: bounds.height ? (event.clientY - bounds.top) / bounds.height : 0, + }); + }, + { passive: false }, +); + +let drag = null; +gridEl.addEventListener("pointerdown", (event) => { + if (event.button !== 0) return; + const cell = event.target.closest(".cell"); + if (!cell) return; + const index = [...gridEl.children].indexOf(cell); + drag = { cell, index, x: event.clientX, y: event.clientY }; + try { + cell.setPointerCapture(event.pointerId); + } catch { + // Convenience only. + } + send({ type: "cellDragStart", index }); +}); +gridEl.addEventListener("pointermove", (event) => { + if (!drag) return; + const dx = event.clientX - drag.x; + const dy = event.clientY - drag.y; + const bounds = drag.cell.getBoundingClientRect(); + drag.x = event.clientX; + drag.y = event.clientY; + send({ + type: "cellDragMove", + index: drag.index, + dx: bounds.width ? dx / bounds.width : 0, + dy: bounds.height ? dy / bounds.height : 0, + }); +}); +function endDrag(event) { + if (!drag) return; + try { + drag.cell.releasePointerCapture(event.pointerId); + } catch { + // Already released. + } + send({ type: "cellDragEnd", index: drag.index }); + drag = null; +} +document.addEventListener("pointerup", endDrag); +document.addEventListener("pointercancel", endDrag); + +gridEl.addEventListener("dblclick", (event) => { + const index = cellIndexOf(event.target); + if (index >= 0) send({ type: "cellDblclick", index }); +}); + +// --- context menu, built from the last snapshot ---------------------------- +// +// Copy/open actions execute right here -- the snapshot already carries +// everything they need. Everything that mutates a real player only the +// broadcast page owns goes back over the channel as a cellMenuAction intent. + +async function copyText(text, label) { + try { + await navigator.clipboard.writeText(text); + setLocalStatus(`copied ${label}`, "busy"); + } catch { + setLocalStatus("clipboard blocked by the browser", "error"); + } +} + +function menuItemsFor(cellData) { + if (!cellData || cellData.empty) return []; + const { index, videoId, title, currentTime, locked, playing, zoom } = cellData; + const name = title ?? videoId; + const relay = (action) => () => send({ type: "cellMenuAction", index, action }); + return [ + { + label: "Copy video URL at time", + hint: formatTimecode(currentTime), + run: () => copyText(videoUrl(videoId, currentTime), `URL at ${formatTimecode(currentTime)}`), + }, + { label: "Copy video URL", run: () => copyText(videoUrl(videoId), "URL") }, + { label: "Copy video ID", hint: videoId, run: () => copyText(videoId, "video ID") }, + { label: "Copy title", run: () => copyText(name, "title") }, + { + label: "Open on YouTube at time", + run: () => window.open(videoUrl(videoId, currentTime), "_blank", "noopener"), + }, + { label: playing ? "Pause this cell" : "Play this cell", run: relay("togglePlay") }, + { + label: locked ? "Unlock audio" : "Lock audio to this cell", + hint: "double-click", + run: relay("toggleLock"), + }, + { label: "Restart this cell", run: relay("restart") }, + { label: "Reset zoom", hint: `${(zoom ?? 1).toFixed(2)}×`, run: relay("resetZoom") }, + { + label: "Replace with next reserve", + hint: `${latestGlobal.reservesLeft ?? 0} left`, + run: relay("replaceReserve"), + }, + ]; +} + +function closeMenu() { + menuEl.hidden = true; +} + +function openMenu(index, x, y) { + const cellData = latestCells[index]; + const items = menuItemsFor(cellData); + if (items.length === 0) return; + + menuEl.replaceChildren(); + const head = document.createElement("div"); + head.className = "head"; + head.textContent = cellData.title ?? cellData.videoId; + menuEl.appendChild(head); + + for (const item of items) { + const button = document.createElement("button"); + const label = document.createElement("span"); + label.textContent = item.label; + button.appendChild(label); + if (item.hint) { + const hint = document.createElement("span"); + hint.className = "hint"; + hint.textContent = item.hint; + button.appendChild(hint); + } + button.addEventListener("click", () => { + item.run(); + closeMenu(); + }); + menuEl.appendChild(button); + } + + menuEl.hidden = false; + menuEl.style.left = `${x}px`; + menuEl.style.top = `${y}px`; + const rect = menuEl.getBoundingClientRect(); + if (rect.right > window.innerWidth) { + menuEl.style.left = `${Math.max(0, window.innerWidth - rect.width - 4)}px`; + } + if (rect.bottom > window.innerHeight) { + menuEl.style.top = `${Math.max(0, window.innerHeight - rect.height - 4)}px`; + } +} + +gridEl.addEventListener("contextmenu", (event) => { + const index = cellIndexOf(event.target); + if (index < 0) return; + event.preventDefault(); + openMenu(index, event.clientX, event.clientY); +}); +document.addEventListener("click", (event) => { + if (!menuEl.hidden && !menuEl.contains(event.target)) closeMenu(); +}); +document.addEventListener("keydown", (event) => { + if (event.key === "Escape") closeMenu(); +}); diff --git a/static/layout-page.js b/static/layout-page.js index dc3f2e6..5d11282 100644 --- a/static/layout-page.js +++ b/static/layout-page.js @@ -56,4 +56,4 @@ function computeLayout(config) { } await loadScreens(); -startWall({ computeLayout }); +startWall({ computeLayout, controlChannel: "yt-matrix-layout-control" }); diff --git a/static/layout.html b/static/layout.html index 2f93021..734b66c 100644 --- a/static/layout.html +++ b/static/layout.html @@ -20,6 +20,16 @@ padding: 8px 14px; border-bottom: 1px solid #26262b; flex: none; overflow: hidden; } + /* Hidden, not removed: wall-engine.js's getElementById lookups for every + header control must keep succeeding, since /layout-control drives them + remotely. This is what keeps the NDI broadcast clean -- nothing here is + a JS change. + + #menu goes with it. contextmenu's preventDefault() only stops the + browser's own menu; the app's custom one would still paint over the wall + and straight into the broadcast feed. Right-clicking is done on + /layout-control, which relays the chosen action as an intent. */ + header, #menu { display: none; } button { font: inherit; font-weight: 600; padding: 7px 13px; border-radius: 6px; border: 1px solid #3a3a42; background: #1c1c21; color: #e8e8ea; cursor: pointer; diff --git a/static/wall-engine.js b/static/wall-engine.js index 128b4d6..89b61ad 100644 --- a/static/wall-engine.js +++ b/static/wall-engine.js @@ -139,7 +139,7 @@ function defaultComputeLayout(config) { // in startWall() so /layout could reuse it was meant to be a pure extraction, // and re-indenting every line would have turned that into a diff nobody could // audit line-by-line. Leave it flush left. -export function startWall({ computeLayout = defaultComputeLayout } = {}) { +export function startWall({ computeLayout = defaultComputeLayout, controlChannel = null } = {}) { const gridEl = document.getElementById("grid"); const statusEl = document.getElementById("status"); @@ -537,6 +537,80 @@ function rebuild() { prerollCurrentSet(++generation); } +// null until startWall({ controlChannel }) opens one -- publishSnapshot() +// below is a no-op until then, so every call site can call it unconditionally +// without checking whether a control page exists. +let broadcastChannel = null; + +function buildSnapshot() { + // The heartbeat below fires on a timer, which can beat the first resync() to + // the punch -- and computeLayout(config) is the first thing this does. + // Nothing useful to publish yet; the next tick will have it. + if (!config) return null; + const layout = computeLayout(config); + const cells = slotState.slots.map((videoId, index) => { + const player = players[index]; + let currentTime = 0; + let playing = false; + try { + currentTime = player?.getCurrentTime?.() ?? 0; + playing = (player?.getPlayerState?.() ?? -1) === 1; + } catch { + // A player mid-teardown; snapshot with the defaults above. + } + return { + index, + videoId: videoId ?? null, + title: videoId ? (titles.get(videoId) ?? videoId) : null, + empty: !videoId, + zoom: views.get(index)?.zoom ?? 1, + locked: lockedIndex === index, + audible: isAudible(index, currentAudioTarget()), + playing, + currentTime, + // Percentages, same object shape buildCells() already applies to this + // page's own cell -- /layout-control positions its rectangle from this + // directly, so the two pages can never disagree about geometry. + rect: layout.cellRect ? layout.cellRect(index) : null, + }; + }); + + return { + type: "snapshot", + global: { + status: statusEl.textContent, + statusState: statusEl.dataset.state ?? "", + audioIndicatorText: audioEl.textContent, + audioLocked: audioEl.dataset.locked === "true", + muted, + prerolled, + wantPlaying, + playDisabled: playButton.disabled, + hoverUnmuteChecked: hoverUnmuteCheckbox.checked, + followChecked: followCheckbox.checked, + newQueryVisible: !newQueryButton.hidden, + newQueryDisabled: newQueryButton.disabled, + reservesLeft: slotState.reserves.length, + }, + cells, + }; +} + +function publishSnapshot() { + if (!broadcastChannel) return; + try { + const snapshot = buildSnapshot(); + if (!snapshot) return; + broadcastChannel.postMessage(snapshot); + } catch (error) { + // A closed channel is the expected case, but this catch also covers + // anything buildSnapshot() itself threw -- and a control page silently + // frozen on a stale snapshot is exactly the sort of failure that costs an + // afternoon. Say so. + wlog("publishSnapshot failed", error); + } +} + function refreshControls() { playButton.disabled = !prerolled; window.__wantPlaying = wantPlaying; @@ -657,6 +731,7 @@ function finishPreroll(token) { refreshControls(); setStatus(`${statusPrefix} · ready — press Play`); } + publishSnapshot(); } function startAll() { @@ -685,6 +760,47 @@ function pauseAll() { } } +function rewindAll() { + for (const player of livePlayers()) { + try { + player.seekTo(config.playback.start_offset, true); + // seekTo resumes a player that is not already paused, so a paused wall + // would quietly start playing. Put it back. + if (!wantPlaying) player.pauseVideo(); + } catch { + // A player mid-teardown; skip it. + } + } +} + +function toggleMute() { + muted = !muted; + refreshMuteButton(); + // Unmuting eight players at once is only permitted off a user gesture -- + // this click is it. Doing it any other way leaves some players silent. + applyMuteStateToAll(); +} + +function shuffleWall() { + const wasPlaying = wantPlaying; + slotState = shuffleSlots( + [...slotState.slots, ...slotState.reserves], + computeLayout(config).totalCells, + ); + rebuild(); + // Shuffling is not a new query, so it should not silently stop the wall. + // rebuild() clears wantPlaying via pre-roll; put it back if it was running. + wantPlaying = wasPlaying; +} + +function resetAllViews() { + views.clear(); + for (const cell of gridEl.children) { + delete cell.dataset.zoomed; + applyCoverFit(cell); + } +} + /** * Restart each video shortly before it ends. * @@ -964,7 +1080,7 @@ newQueryButton.addEventListener("click", async () => { const prompt = promptInput.value.trim(); promptInput.disabled = true; try { - await requestNewQuery(prompt || null); + await applyIntent({ type: "newQuery", prompt: prompt || null }); } finally { promptInput.disabled = false; } @@ -993,6 +1109,42 @@ async function copyText(text, label) { } } +function togglePlayForCell(index) { + const player = players[index]; + let state = -1; + try { + state = player?.getPlayerState?.() ?? -1; + } catch { + // A player mid-teardown; nothing useful to do. + } + if (state === 1) player?.pauseVideo?.(); + else player?.playVideo?.(); +} + +function toggleLockedIndex(index) { + // Locking audio to a cell with no player would leave the wall silent with + // an indicator claiming otherwise. The local dblclick handler already + // refuses an empty cell; a relayed cellDblclick reaches here directly. + if (gridEl.children[index]?.dataset.empty === "true") return; + lockedIndex = lockedIndex === index ? null : index; + for (const other of gridEl.children) delete other.dataset.locked; + const cell = gridEl.children[index]; + if (lockedIndex !== null && cell) cell.dataset.locked = "true"; + applyMuteStateToAll(); +} + +function restartCell(index) { + players[index]?.seekTo?.(config.playback.start_offset, true); +} + +function resetCellZoom(index) { + views.delete(index); + const cell = gridEl.children[index]; + if (!cell) return; + cell.dataset.zoomed = "false"; + applyCoverFit(cell); +} + function playerForCell(cell) { const index = [...gridEl.children].indexOf(cell); return { index, player: index >= 0 ? players[index] : null }; @@ -1040,30 +1192,21 @@ function menuItems(cell) { }, { label: playing ? "Pause this cell" : "Play this cell", - run: () => (playing ? player?.pauseVideo?.() : player?.playVideo?.()), + run: () => togglePlayForCell(index), }, { label: lockedIndex === index ? "Unlock audio" : "Lock audio to this cell", hint: "double-click", - run: () => { - lockedIndex = lockedIndex === index ? null : index; - for (const other of gridEl.children) delete other.dataset.locked; - if (lockedIndex !== null) cell.dataset.locked = "true"; - applyMuteStateToAll(); - }, + run: () => toggleLockedIndex(index), }, { label: "Restart this cell", - run: () => player?.seekTo?.(config.playback.start_offset, true), + run: () => restartCell(index), }, { label: "Reset zoom", hint: `${(viewFor(cell).zoom ?? 1).toFixed(2)}×`, - run: () => { - views.delete(index); - cell.dataset.zoomed = "false"; - applyCoverFit(cell); - }, + run: () => resetCellZoom(index), }, { label: "Replace with next reserve", @@ -1135,38 +1278,17 @@ window.addEventListener("resize", closeMenu); // Eight players starting at once is exactly what browsers throttle; this click // is the user gesture that makes them all start. It is disabled until the set // has pre-rolled, so there is no window where pressing it starts only some. -playButton.addEventListener("click", startAll); -pauseButton.addEventListener("click", pauseAll); - -rewindButton.addEventListener("click", () => { - for (const player of livePlayers()) { - try { - player.seekTo(config.playback.start_offset, true); - // seekTo resumes a player that is not already paused, so a paused wall - // would quietly start playing. Put it back. - if (!wantPlaying) player.pauseVideo(); - } catch { - // A player mid-teardown; skip it. - } - } -}); - -muteButton.addEventListener("click", () => { - muted = !muted; - refreshMuteButton(); - // Unmuting eight players at once is only permitted off a user gesture -- - // this click is it. Doing it any other way leaves some players silent. - applyMuteStateToAll(); -}); +playButton.addEventListener("click", () => applyIntent({ type: "play" })); +pauseButton.addEventListener("click", () => applyIntent({ type: "pause" })); +rewindButton.addEventListener("click", () => applyIntent({ type: "rewind" })); +muteButton.addEventListener("click", () => applyIntent({ type: "muteToggle" })); // Hover to unmute: point at a cell to hear only that one. The iframe has // pointer-events:none, so the cell itself receives the hover -- the same CSS // the context menu depends on. -hoverUnmuteCheckbox.addEventListener("change", () => { - // Leaving the mode must not strand a cell audible or the whole wall silent. - setAudibleCell(null); - applyMuteStateToAll(); -}); +hoverUnmuteCheckbox.addEventListener("change", () => + applyIntent({ type: "hoverUnmuteToggle", checked: hoverUnmuteCheckbox.checked }), +); // Scroll to zoom, anchored on the pointer. passive:false because the page // must not scroll underneath -- the wall is a fixed-height layout and a @@ -1203,41 +1325,167 @@ function flushZoom() { cell.dataset.zoomed = view.zoom > 1.001 ? "true" : "false"; } +function queueZoom(cell, index, width, height, deltaY, x, y) { + if (pendingZoom && pendingZoom.index === index) { + // Same cell, same frame: sum the deltas so no scrolling is lost, and + // track the cursor to wherever it ended up. + pendingZoom.deltaY += deltaY; + pendingZoom.x = x; + pendingZoom.y = y; + return; + } + // A different cell mid-frame: land the queued one first rather than + // dropping it. + if (pendingZoom) flushZoom(); + pendingZoom = { cell, index, width, height, deltaY, x, y }; + requestAnimationFrame(flushZoom); +} + +// x, y arrive normalized (0..1) -- from a real local event on THIS page's own +// cell, or relayed from /layout-control's differently-sized rectangle for +// the same cell. Either way this converts to this page's own real pixel +// space before reusing the exact zoom math a local wheel event already used. +function applyCellWheel(index, deltaY, xFraction, yFraction) { + const cell = gridEl.children[index]; + if (!cell || cell.dataset.empty === "true") return; + const bounds = cell.getBoundingClientRect(); + queueZoom(cell, index, bounds.width, bounds.height, deltaY, xFraction * bounds.width, yFraction * bounds.height); +} + +// dx, dy arrive as a fraction of the SENDER's own cell size (already an +// incremental delta, not an absolute position) -- rescaling by this page's +// own bounds is what makes a drag feel proportionally the same regardless of +// how big /layout-control's rectangle happens to be. +function applyCellPan(index, dxFraction, dyFraction) { + const cell = gridEl.children[index]; + // Same guard applyCellWheel has: an empty cell has no iframe to move, and + // storing a view for it would survive into whatever reserve lands there. + if (!cell || cell.dataset.empty === "true") return; + const bounds = cell.getBoundingClientRect(); + views.set( + index, + panBy(views.get(index) ?? IDENTITY_VIEW, dxFraction * bounds.width, dyFraction * bounds.height), + ); + applyCoverFit(cell); +} + +function applyCellMenuAction(index, action) { + switch (action) { + case "togglePlay": + togglePlayForCell(index); + break; + case "toggleLock": + toggleLockedIndex(index); + break; + case "restart": + restartCell(index); + break; + case "resetZoom": + resetCellZoom(index); + break; + case "replaceReserve": + handlePlayerError(index); + break; + default: + wlog(`applyCellMenuAction: unknown action ${action}`); + } +} + +// Every user action reachable from the header or a cell -- a real DOM event +// on THIS page, or one relayed from /layout-control over BroadcastChannel -- +// funnels through here. That symmetry is the whole point: a control-window +// message and a local click must produce identical effects, so this is the +// only place either kind is handled. publishSnapshot() at the end is a no-op +// unless startWall was given a controlChannel, so every path can call it. +async function applyIntent(intent) { + switch (intent.type) { + case "play": + startAll(); + break; + case "pause": + pauseAll(); + break; + case "muteToggle": + toggleMute(); + break; + case "rewind": + rewindAll(); + break; + case "shuffle": + shuffleWall(); + break; + case "resetView": + resetAllViews(); + break; + case "newQuery": + // `generating` is already set synchronously by requestNewQuery and + // cleared in its finally, so this is the same guard the local button's + // disabled state gives a local click. A relayed intent never sees that + // disabled state -- /layout-control's own button only learns it is + // disabled when the next snapshot arrives, up to a heartbeat later -- + // so the dispatcher itself has to hold the line. Two fast clicks over + // the channel would otherwise be two generations at 100 units each. + if (generating) { + wlog("ignoring newQuery: one is already in flight"); + break; + } + await requestNewQuery(intent.prompt ?? null); + break; + case "hoverUnmuteToggle": + hoverUnmuteCheckbox.checked = intent.checked; + setAudibleCell(null); + applyMuteStateToAll(); + break; + case "followToggle": + followCheckbox.checked = intent.checked; + break; + case "cellHoverEnter": + if (hoverUnmuteCheckbox.checked) setAudibleCell(intent.index); + break; + case "cellHoverLeave": + if (hoverUnmuteCheckbox.checked) setAudibleCell(null); + break; + case "cellWheel": + applyCellWheel(intent.index, intent.deltaY, intent.x, intent.y); + break; + case "cellDragStart": + // Purely a cursor affordance; kept for parity with a locally-driven drag. + gridEl.children[intent.index]?.setAttribute("data-dragging", "true"); + break; + case "cellDragMove": + applyCellPan(intent.index, intent.dx, intent.dy); + break; + case "cellDragEnd": + gridEl.children[intent.index]?.removeAttribute("data-dragging"); + break; + case "cellDblclick": + toggleLockedIndex(intent.index); + break; + case "cellMenuAction": + applyCellMenuAction(intent.index, intent.action); + break; + default: + wlog(`applyIntent: unknown intent type ${intent.type}`); + return; + } + publishSnapshot(); +} + gridEl.addEventListener( "wheel", (event) => { const cell = event.target.closest(".cell"); if (!cell || cell.dataset.empty === "true") return; event.preventDefault(); - const index = [...gridEl.children].indexOf(cell); const bounds = cell.getBoundingClientRect(); - const x = event.clientX - bounds.left; - const y = event.clientY - bounds.top; - - if (pendingZoom && pendingZoom.index === index) { - // Same cell, same frame: sum the deltas so no scrolling is lost, and - // track the cursor to wherever it ended up. - pendingZoom.deltaY += event.deltaY; - pendingZoom.x = x; - pendingZoom.y = y; - return; - } - - // A different cell mid-frame: land the queued one first rather than - // dropping it. - if (pendingZoom) flushZoom(); - - pendingZoom = { - cell, + applyIntent({ + type: "cellWheel", index, - width: bounds.width, - height: bounds.height, deltaY: event.deltaY, - x, - y, - }; - requestAnimationFrame(flushZoom); + x: bounds.width ? (event.clientX - bounds.left) / bounds.width : 0, + y: bounds.height ? (event.clientY - bounds.top) / bounds.height : 0, + }); }, { passive: false }, ); @@ -1250,28 +1498,29 @@ gridEl.addEventListener("pointerdown", (event) => { if (event.button !== 0) return; // left button only; right opens the menu const cell = event.target.closest(".cell"); if (!cell || cell.dataset.empty === "true") return; - drag = { - cell, - index: [...gridEl.children].indexOf(cell), - x: event.clientX, - y: event.clientY, - }; - cell.dataset.dragging = "true"; + const index = [...gridEl.children].indexOf(cell); + drag = { cell, index, x: event.clientX, y: event.clientY }; try { cell.setPointerCapture(event.pointerId); } catch { // Capture is a convenience; the document-level pointerup still ends it. } + applyIntent({ type: "cellDragStart", index }); }); gridEl.addEventListener("pointermove", (event) => { if (!drag) return; const dx = event.clientX - drag.x; const dy = event.clientY - drag.y; + const bounds = drag.cell.getBoundingClientRect(); drag.x = event.clientX; drag.y = event.clientY; - views.set(drag.index, panBy(views.get(drag.index) ?? IDENTITY_VIEW, dx, dy)); - applyCoverFit(drag.cell); + applyIntent({ + type: "cellDragMove", + index: drag.index, + dx: bounds.width ? dx / bounds.width : 0, + dy: bounds.height ? dy / bounds.height : 0, + }); }); function endDrag(event) { @@ -1281,7 +1530,7 @@ function endDrag(event) { } catch { // Already released, or never captured. } - delete drag.cell.dataset.dragging; + applyIntent({ type: "cellDragEnd", index: drag.index }); drag = null; } @@ -1293,25 +1542,9 @@ document.addEventListener("pointercancel", endDrag); // there is plenty behind the wall -- and reshuffling what we already paid for // costs nothing. Not persisted: a reload restores the server's ranked order, // which is relevance, country spread and stills-to-the-back. -shuffleButton.addEventListener("click", () => { - const wasPlaying = wantPlaying; - slotState = shuffleSlots( - [...slotState.slots, ...slotState.reserves], - computeLayout(config).totalCells, - ); - rebuild(); - // Shuffling is not a new query, so it should not silently stop the wall. - // rebuild() clears wantPlaying via pre-roll; put it back if it was running. - wantPlaying = wasPlaying; -}); +shuffleButton.addEventListener("click", () => applyIntent({ type: "shuffle" })); -resetViewButton.addEventListener("click", () => { - views.clear(); - for (const cell of gridEl.children) { - delete cell.dataset.zoomed; - applyCoverFit(cell); - } -}); +resetViewButton.addEventListener("click", () => applyIntent({ type: "resetView" })); // Double-click to hold the audio on one cell. Again on the same cell turns it // off; on a different cell it moves there. Unlike hover, this survives the @@ -1319,40 +1552,53 @@ resetViewButton.addEventListener("click", () => { gridEl.addEventListener("dblclick", (event) => { const cell = event.target.closest(".cell"); if (!cell || cell.dataset.empty === "true") return; - const index = [...gridEl.children].indexOf(cell); - lockedIndex = lockedIndex === index ? null : index; - - for (const other of gridEl.children) delete other.dataset.locked; - if (lockedIndex !== null) cell.dataset.locked = "true"; - applyMuteStateToAll(); + applyIntent({ type: "cellDblclick", index: [...gridEl.children].indexOf(cell) }); }); gridEl.addEventListener("pointerover", (event) => { if (!hoverUnmuteCheckbox.checked) return; const cell = event.target.closest(".cell"); if (!cell || cell.dataset.empty === "true") return; - setAudibleCell([...gridEl.children].indexOf(cell)); + applyIntent({ type: "cellHoverEnter", index: [...gridEl.children].indexOf(cell) }); }); // pointerleave on the grid, not per cell: moving between adjacent cells would // otherwise blip the audio off and on again between them. gridEl.addEventListener("pointerleave", () => { if (!hoverUnmuteCheckbox.checked) return; - setAudibleCell(null); + applyIntent({ type: "cellHoverLeave" }); }); refreshMuteButton(); refreshControls(); +if (controlChannel) { + broadcastChannel = new BroadcastChannel(controlChannel); + broadcastChannel.addEventListener("message", (event) => { + applyIntent(event.data); + }); + // A heartbeat, not the only source of truth: applyIntent and finishPreroll + // already publish on every discrete change. This just guarantees a + // late-joining or reconnecting control tab is never more than a second + // stale, without instrumenting every low-level mutation site. + setInterval(publishSnapshot, 1000); +} + connectSocket({ onReconnect: resync, - // Config is the only thing broadcast. Nothing pushes a video set any more: - // the server does not know what query any given browser is watching, so a - // wall only ever changes its own videos -- on its own resync, or off its own - // New query. Deciding whether a config change means this browser has to - // refetch is therefore the client's job, done right here. + // Config and relayed control intents (POST /api/intent, e.g. from an + // external OSC/show-control router) are the only things broadcast. Nothing + // pushes a video set any more: the server does not know what query any + // given browser is watching, so a wall only ever changes its own videos -- + // on its own resync, or off its own New query. Deciding whether a config + // change means this browser has to refetch is therefore the client's job, + // done right here. onMessage: (message) => { wlog(`socket message type=${message.type}`); + if (message.type === "intent") { + applyIntent(message.intent); + return; + } if (message.type !== "config") return; const previous = config; const change = classifyConfigChange(previous, message.config); diff --git a/tests/test_layout_control_smoke.py b/tests/test_layout_control_smoke.py new file mode 100644 index 0000000..3e62508 --- /dev/null +++ b/tests/test_layout_control_smoke.py @@ -0,0 +1,246 @@ +"""Browser smoke test for /layout-control: an interaction on the control +page must land on the real broadcast page, over BroadcastChannel. + +Marked `browser`, excluded from the default suite. Both pages come from one +Playwright BrowserContext -- BroadcastChannel only connects tabs in the same +browser profile, which is exactly the constraint this feature is built +around (the NDI broadcaster already only ever captures a local window). +""" + +from __future__ import annotations + +import asyncio +import os +import socket +import subprocess +import sys +import time +from pathlib import Path + +import httpx +import pytest +import yaml +from playwright.sync_api import sync_playwright + +from ytmatrix import cache, youtube +from ytmatrix.store import FileStore + +REPO_ROOT = Path(__file__).resolve().parent.parent + +pytestmark = pytest.mark.browser + +CONFIG = { + "query": "golden cover", + "grid": {"cols": 4, "rows": 2}, + "search": { + "order": "relevance", + "video_duration": "any", + "safe_search": "moderate", + "relevance_language": "en", + }, + "playback": {"muted": True, "autoplay_on_change": True, "start_offset": 0, "loop": True}, + "cache": {"ttl_hours": 24}, + "query_generation": {"enabled": True}, +} + + +def _find_free_port() -> int: + with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock: + sock.bind(("127.0.0.1", 0)) + return sock.getsockname()[1] + + +@pytest.fixture(scope="session") +def _fresh_dist(): + subprocess.run( + ["bash", str(REPO_ROOT / "scripts" / "build-dist.sh")], + cwd=str(REPO_ROOT), + check=True, + capture_output=True, + ) + + +@pytest.fixture +def running_server(tmp_path, _fresh_dist): + config_path = tmp_path / "config.yaml" + config_path.write_text(yaml.safe_dump(CONFIG)) + + cache_dir = tmp_path / "cache" + params = youtube.build_params("golden cover", "relevance", "any", "moderate", "en") + base = [ + "43DOm50YWaI", + "1qwq1UCG9c4", + "AuaABDWFs_8", + "0-xSuAexKTw", + "huqUnIVAjHg", + "GnDfJC1vPlQ", + "R7EH2TKJHYQ", + "uSAPVDS2LUo", + ] + ids = [base[i % len(base)] for i in range(50)] + asyncio.run( + cache.write( + FileStore(cache_dir), + params, + [{"video_id": v, "title": v, "channel": "c"} for v in ids], + ) + ) + + port = _find_free_port() + env = { + **os.environ, + "YOUTUBE_API_KEY": "SMOKE_TEST_KEY_UNUSED", + "YTMATRIX_HOST": "127.0.0.1", + "YTMATRIX_PORT": str(port), + "YTMATRIX_CONFIG_PATH": str(config_path), + "YTMATRIX_CACHE_DIR": str(cache_dir), + "YTMATRIX_RUNTIME_DIR": str(tmp_path / "runtime"), + } + process = subprocess.Popen( + [sys.executable, "-m", "ytmatrix.main"], + cwd=str(REPO_ROOT), + env=env, + stdout=subprocess.PIPE, + stderr=subprocess.STDOUT, + ) + try: + deadline = time.monotonic() + 20 + while time.monotonic() < deadline: + try: + if ( + httpx.get( + f"https://localhost:{port}/healthz", verify=False, timeout=1.0 + ).status_code + == 200 + ): + break + except httpx.HTTPError: + pass + time.sleep(0.3) + else: + raise RuntimeError("server did not become healthy in time") + yield f"https://localhost:{port}" + finally: + process.terminate() + process.wait(timeout=5) + + +def test_a_wheel_on_the_control_page_zooms_the_real_cell(running_server): + with sync_playwright() as p: + browser = p.chromium.launch() + context = browser.new_context(ignore_https_errors=True) + broadcast = context.new_page() + control = context.new_page() + + broadcast.goto(f"{running_server}/layout", wait_until="load") + broadcast.wait_for_function( + "document.querySelectorAll('.cell iframe').length === 8", timeout=20_000 + ) + control.goto(f"{running_server}/layout-control", wait_until="load") + control.wait_for_selector(".cell", timeout=20_000) + + def broadcast_zoom(nth=0): + return broadcast.evaluate( + f"""() => {{ + const cells = document.querySelectorAll('.cell'); + const f = cells[{nth}].querySelector('iframe').getBoundingClientRect(); + const c = cells[{nth}].getBoundingClientRect(); + return f.width / c.width; + }}""" + ) + + before = broadcast_zoom() + + box = control.locator(".cell").first.bounding_box() + control.mouse.move(box["x"] + box["width"] / 2, box["y"] + box["height"] / 2) + for _ in range(8): + control.mouse.wheel(0, -120) + + broadcast.wait_for_function( + f"""() => {{ + const cells = document.querySelectorAll('.cell'); + const f = cells[0].querySelector('iframe').getBoundingClientRect(); + const c = cells[0].getBoundingClientRect(); + return f.width / c.width > {before + 0.2}; + }}""", + timeout=10_000, + ) + browser.close() + + +def test_a_menu_action_on_the_control_page_pauses_the_real_player(running_server): + with sync_playwright() as p: + browser = p.chromium.launch(args=["--autoplay-policy=no-user-gesture-required"]) + context = browser.new_context(ignore_https_errors=True) + broadcast = context.new_page() + control = context.new_page() + + broadcast.goto(f"{running_server}/layout", wait_until="load") + broadcast.wait_for_function("window.__prerolled === true", timeout=40_000) + broadcast.evaluate("window.__players.forEach(p => p && p.playVideo())") + # Specifically cell 0, which is the one the menu action below targets. + # "some player is playing" would pass with cell 0 never having started, + # and then the assertion at the end would be vacuously true. + broadcast.wait_for_function( + "window.__players[0] && window.__players[0].getPlayerState() === 1", timeout=10_000 + ) + + control.goto(f"{running_server}/layout-control", wait_until="load") + control.wait_for_selector('.cell[data-empty="false"]', timeout=20_000) + + control.locator(".cell").first.click(button="right") + control.wait_for_selector("#menu:not([hidden])", timeout=5_000) + control.locator("#menu button", has_text="Pause this cell").first.click() + + broadcast.wait_for_function( + "window.__players[0] && window.__players[0].getPlayerState() !== 1", timeout=10_000 + ) + browser.close() + + +def test_the_control_page_reflects_mute_state_from_the_broadcast_page(running_server): + """Play and unmute both survive the gesture moving to another document. + + Launched with NO `--autoplay-policy=no-user-gesture-required`, unlike the + menu test above -- that is the point. The buttons an operator actually + presses now live on /layout-control, and BroadcastChannel confers no user + activation on the receiving document, so /layout starts and unmutes eight + players having never been clicked. This is the test that says that works; + see CLAUDE.md gotcha 41 for why it is allowed to. + """ + with sync_playwright() as p: + browser = p.chromium.launch() + context = browser.new_context(ignore_https_errors=True) + broadcast = context.new_page() + control = context.new_page() + + broadcast.goto(f"{running_server}/layout", wait_until="load") + broadcast.wait_for_function("window.__prerolled === true", timeout=40_000) + + control.goto(f"{running_server}/layout-control", wait_until="load") + # The mute button already reads "Unmute" in the page's static HTML, so + # waiting on its text proves nothing. The status line starting out as + # this page's own placeholder and then changing IS proof a snapshot + # arrived -- only renderFromSnapshot ever replaces it. + control.wait_for_function( + "document.getElementById('status').textContent !== 'waiting for /layout to connect…'", + timeout=10_000, + ) + + control.click("#play") + broadcast.wait_for_function( + "window.__players.every(p => p && p.getPlayerState() === 1)", timeout=20_000 + ) + + control.click("#mute") + broadcast.wait_for_function("window.__players.every(p => !p.isMuted())", timeout=10_000) + # Still playing, specifically. A browser that refused the unmute would + # not necessarily report muted -- it could pause the media instead, and + # a wall that goes silent-and-frozen is the failure worth naming. + broadcast.wait_for_function( + "window.__players.every(p => p && p.getPlayerState() === 1)", timeout=10_000 + ) + control.wait_for_function( + "document.getElementById('mute').textContent.trim() === 'Mute'", timeout=10_000 + ) + browser.close() diff --git a/tests/test_layout_smoke.py b/tests/test_layout_smoke.py index 2da28ce..26538ad 100644 --- a/tests/test_layout_smoke.py +++ b/tests/test_layout_smoke.py @@ -198,7 +198,7 @@ def test_pre_roll_and_mute_work_the_same_as_the_grid_page(running_server): page.wait_for_function("window.__prerolled === true", timeout=40_000) assert page.evaluate("window.__players.every(p => p.isMuted())"), "should start muted" - page.click("#play") - page.click("#mute") + page.evaluate("document.querySelector('#play').click()") + page.evaluate("document.querySelector('#mute').click()") page.wait_for_function("window.__players.every(p => !p.isMuted())", timeout=15_000) browser.close() diff --git a/tests/test_player_smoke.py b/tests/test_player_smoke.py index a601888..3d685e7 100644 --- a/tests/test_player_smoke.py +++ b/tests/test_player_smoke.py @@ -248,6 +248,39 @@ def test_pause_stops_every_player(running_server): browser.close() +def test_a_posted_intent_reaches_the_wall_over_websocket(running_server): + """POST /api/intent is the second front door into applyIntent() -- for a + controller with no browser to share a BroadcastChannel with (e.g. chasa, + an OSC router). This proves the whole relay: HTTP POST -> server + broadcast -> this page's own /ws connection -> applyIntent() -> a real + effect on the players, with no button ever clicked. + """ + with sync_playwright() as p: + browser = p.chromium.launch(args=["--autoplay-policy=no-user-gesture-required"]) + page = browser.new_context(ignore_https_errors=True).new_page() + page.goto(running_server, wait_until="load") + page.wait_for_function( + "window.__players?.every(p => typeof p?.pauseVideo === 'function')", timeout=25_000 + ) + page.wait_for_function("window.__prerolled === true", timeout=40_000) + page.evaluate(""" + window.__pauseCalls = 0; + for (const player of window.__players) { + const original = player.pauseVideo.bind(player); + player.pauseVideo = () => { window.__pauseCalls += 1; return original(); }; + } + """) + page.click("#play") + + response = httpx.post( + f"{running_server}api/intent", json={"type": "pause"}, verify=False, timeout=5.0 + ) + assert response.status_code == 200 + + page.wait_for_function("window.__pauseCalls >= 8", timeout=15_000) + browser.close() + + def test_a_new_set_stays_paused_until_pre_rolled(running_server): """Nothing starts until every cell has buffered, and not even then.""" with sync_playwright() as p: diff --git a/tests/test_server.py b/tests/test_server.py index caf1b3b..3bbf2b8 100644 --- a/tests/test_server.py +++ b/tests/test_server.py @@ -209,6 +209,40 @@ def test_a_cosmetic_change_broadcasts_config_only(app_env): assert first["config"]["playback"]["start_offset"] == 45 +def test_a_wall_wide_intent_is_broadcast_to_every_connection(app_env): + app, _, _ = app_env + with TestClient(app) as client, client.websocket_connect("/ws") as ws: + response = client.post("/api/intent", json={"type": "muteToggle"}) + assert response.status_code == 200 + message = json.loads(ws.receive_text()) + assert message == {"type": "intent", "intent": {"type": "muteToggle"}} + + +def test_an_intent_with_extra_fields_is_relayed_verbatim(app_env): + """hoverUnmuteToggle/followToggle carry a `checked` flag; newQuery carries + an optional `prompt`. The endpoint does not know or care about a given + type's extra fields -- it relays whatever the caller sent, and the + client-side applyIntent() dispatcher is what interprets them.""" + app, _, _ = app_env + with TestClient(app) as client, client.websocket_connect("/ws") as ws: + client.post("/api/intent", json={"type": "hoverUnmuteToggle", "checked": True}) + message = json.loads(ws.receive_text()) + assert message == {"type": "intent", "intent": {"type": "hoverUnmuteToggle", "checked": True}} + + +def test_an_unknown_intent_type_is_rejected_with_422(app_env): + app, _, _ = app_env + with TestClient(app) as client: + response = client.post("/api/intent", json={"type": "cellMenuAction", "index": 0}) + assert response.status_code == 422 + + +def test_a_missing_intent_type_is_rejected_with_422(app_env): + app, _, _ = app_env + with TestClient(app) as client: + assert client.post("/api/intent", json={}).status_code == 422 + + def test_quota_exhaustion_falls_back_to_stale_cache(app_env, monkeypatch): app, _, store = app_env ids = seed_cache(store) diff --git a/ytmatrix/main.py b/ytmatrix/main.py index 6cf0257..77fa07c 100644 --- a/ytmatrix/main.py +++ b/ytmatrix/main.py @@ -78,6 +78,10 @@ async def _config_page() -> FileResponse: async def _layout_page() -> FileResponse: return FileResponse(dist / "layout.html") + @app.get("/layout-control", include_in_schema=False) + async def _layout_control_page() -> FileResponse: + return FileResponse(dist / "layout-control.html") + app.mount("/", StaticFiles(directory=dist, html=True), name="dist") uvicorn.run( diff --git a/ytmatrix/server.py b/ytmatrix/server.py index c579561..14cb228 100644 --- a/ytmatrix/server.py +++ b/ytmatrix/server.py @@ -30,6 +30,23 @@ logger = logging.getLogger(__name__) +# /api/intent's allowlist. Deliberately excludes every cell-indexed intent +# (cellWheel, cellDblclick, cellMenuAction, cellHoverEnter/Leave, +# cellDragStart/Move/End) -- those carry a cell index tied to whichever +# page's grid is open, and this endpoint broadcasts to every connected tab, +# not just one wall. See post_intent's docstring. +WALL_WIDE_INTENT_TYPES = { + "play", + "pause", + "muteToggle", + "rewind", + "shuffle", + "resetView", + "newQuery", + "hoverUnmuteToggle", + "followToggle", +} + class BudgetExceededError(RuntimeError): """The self-imposed daily ceiling would be crossed by another search.""" @@ -794,6 +811,40 @@ async def new_query(request: Request, payload: dict | None = None) -> dict: extra_timings={"gemini": gemini_secs}, ) + @app.post("/api/intent") + async def post_intent(payload: dict) -> dict: + """Relay a wall-wide control intent to every connected browser tab. + + For an external controller (a show-control/OSC router, e.g. chasa) + that cannot join a browser's own BroadcastChannel the way + /layout-control does -- this is a second front door into the same + applyIntent() dispatcher in wall-engine.js, over the /ws connection + every wall already holds open for config pushes. + + Deliberately restricted to WALL_WIDE_INTENT_TYPES: a cell-indexed + intent (cellWheel, cellDblclick, cellMenuAction, ...) carries an + index tied to whichever page's grid is open, and this broadcasts to + every connected tab -- / and /layout alike -- not just one wall. + + newQuery carries the same quota-multiplication risk documented for + /layout-control's own Query button (gotcha 29/2): every open tab + that receives this independently calls requestNewQuery, each one a + 100-unit search. One wall/tab open at a time when driving this from + an external cue source is an operational rule, not something this + endpoint enforces. + """ + intent_type = payload.get("type") + if intent_type not in WALL_WIDE_INTENT_TYPES: + raise HTTPException( + status_code=422, + detail=( + f"unknown intent type {intent_type!r}; must be one of " + f"{sorted(WALL_WIDE_INTENT_TYPES)}" + ), + ) + await manager.broadcast({"type": "intent", "intent": payload}) + return {"status": "ok"} + @app.websocket("/ws") async def websocket_endpoint(websocket: WebSocket) -> None: await manager.connect(websocket)