diff --git a/internal/interaction-harness/README.md b/internal/interaction-harness/README.md index b7f583829fd4a..fbed412ff6877 100644 --- a/internal/interaction-harness/README.md +++ b/internal/interaction-harness/README.md @@ -147,8 +147,15 @@ later in WebKit on a loaded machine). When the cast saw the element at rest before the block (wait for `cast.frameCount()` to pass 0 before starting the animation), judging starts at its first presented change: the judged part still ends with the block and still needs four intervals, and `onScreenFromMs` and the -summary report when it came. An element already moving is judged from the -block's start. WebKit's casts are 2 px quantized: the frames come at the +summary report when it came. So is an element that moved, then held from +before the block until its first presented change inside it: WebKit presents +a just-started compositor animation late (a sheet's release held on screen for +130 to 300 ms while its timeline ran), and its cast reaches Node about 0.7 s +late on a loaded runner, too late to wait on before blocking. Only a hold that +starts before the block, on two frames at least, is excused, and every frame +after its first change is judged, so a freeze later in the block still fails. +Any other element already moving is judged from the block's start. WebKit's +casts are 2 px quantized: the frames come at the viewport's size, but the page in them is captured at half resolution, so positions move in 2 CSS px steps (Chromium's in 1). With under two steps of motion per frame, a frame delivered a few ms early shows no step and reads as @@ -156,6 +163,47 @@ not moved. The proof reports `stepPx` and `pxPerFrame` and notes motion under two steps per frame, which never fails it. Move a proof's element at least two steps a frame: the self-tests move 560 px in 1,600 ms, about 4.8 px a frame. +On a two-core runner the software compositor and the screencast share the core +the block spins on, so every compositor animation presents fewer frames there: +Lightbox's double-tap zoom, pinned to two cores, missed a display-rate bound in +7 of 10 runs while it moved on every frame the compositor presented. Judge such +a proof against a control instead of a rate: `startBusyControl(page)` shows a +small `CONTROL_RGB` box in the top layer (a manual popover, so above a modal +dialog opened before it; `raise()` after one opened later) whose transform +moves it along the top edge for as long as the page runs. A proof of a +dialog's backdrop mounts the control inside the dialog with +`startBusyControl(page, {within: dialog})`, a plain fixed box with no top-layer +entry: in WebKit, with the popover above BottomSheet's open modal dialog, the +sheet's dim held for 155 to 225 ms into a block while the control moved, and +inside the dialog it moves in step with the sheet. `waitForAnimation` and +`blockMainThread`'s `during` never count the control's own animation, so on +a dialog they still wait for the backdrop's. Track it from the +same screencast (`trackMarker(page, frames, {rgb: CONTROL_RGB})`) and pass it +as `control`: the component passes when it moved on every frame the control +moved and its longest stall is no longer than the control's. Both are read from +the same screencast frames, so a compositor animation misses none. A window in +which the control moved on fewer than four frames fails, since the machine +presented too little to judge. A run of frames with no new position counts as +moved when the change that ends it is a single cast step: the component moves +under one step a frame there, as on an easing tail, which the cast cannot show +on every frame. The run must also read as a slowing curve, at most one frame +longer than the component's run before it. A freeze ends in a catch-up of more +than one step, a stall in a frame-counted animation is longer than its runs +before it, and a run that no change ends stays a miss. The component still has +to show a new position on at least four of the control's frames. The cast step (`stepPx`) +is read from the value's changes in the judged part, so a position the page +commits off the grid once the block ends does not shrink it. +A main-thread animation still fails: the control moves and it does not. + +An animation that lasts a fraction of a second, such as a 0.2 s fade, leaves a +block only a few frames to judge: in WebKit, or on a loaded machine, the control +can present fewer than four, and one run cannot decide. Play it several times +and judge the runs pooled (`pooledBusyProof(runs)`, each run with its own +tracked frames, control and window): the control must move on at least four +frames over all the runs, the component on every one of them, and in each run +its longest stall may be no longer than that run's control's, since a stall +belongs to its run. + On a loaded machine Chromium's screencast can skip frames the compositor still presents. `startCompositorTrace(page)`, started with the screencast and stopped after the block, records when the compositor presented each frame diff --git a/internal/interaction-harness/src/busy.control.test.ts b/internal/interaction-harness/src/busy.control.test.ts new file mode 100644 index 0000000000000..f18b732b53311 --- /dev/null +++ b/internal/interaction-harness/src/busy.control.test.ts @@ -0,0 +1,486 @@ +// Copyright (c) Meta Platforms, Inc. and affiliates. + +/** + * @file busy.control.test.ts + * @input Synthetic tracked frames (no browser) + * @output The busy-main-thread verdict against a control: a component that + * moves on the control's frames passes through a stall the compositor itself + * makes, and a frozen one, or one the control outpaces by a single frame, + * still fails. Unchanged frames under one cast step a frame count as + * moved; a freeze, whose catch-up is more than one step, and a stall + * longer than the component's runs before it do not. Runs + * of a block too short to judge alone are judged pooled: the control over + * all of them, each run's stall against that run's control. + * @position Unit test of ./busy's control judgment; the frames are + * busy.test.ts's fixtures, and the browser proof with a real compositor + * control is in harness.interaction.browser.spec.ts. + */ + +import {describe, expect, it} from 'vitest'; +import {busyWindowProof, pooledBusyProof} from './busy'; +import type {TrackedFrame} from './frames'; + +const FRAME = 1000 / 60; +const BLOCK = {from: 1000, to: 1700}; + +/** A marker moving 3 px per frame from t = 800 to 1900, minus `dropped` frame indices. */ +function moving(dropped: readonly number[] = []): TrackedFrame[] { + const frames: TrackedFrame[] = []; + for (let i = 0; 800 + i * FRAME <= 1900; i += 1) { + if (dropped.includes(i)) { + continue; + } + frames.push({ + t: 800 + i * FRAME, + box: {x: 0, y: 3 * i, width: 10, height: 10}, + } as TrackedFrame); + } + return frames; +} + +/** Index of the first frame at or after `t`. */ +const at = (t: number) => Math.ceil((t - 800) / FRAME); + +/** `moving()` with each frame in `repeats` presenting the previous frame's position. */ +function repeating(repeats: readonly number[]): TrackedFrame[] { + return moving().map((f, i) => + repeats.includes(i) + ? ({...f, box: {...f.box!, y: 3 * (i - 1)}} as TrackedFrame) + : f, + ); +} + +/** `frames` as the control's: the same frames and times, moving along x. */ +function asControl(frames: readonly TrackedFrame[]): TrackedFrame[] { + return frames.map( + f => + ({ + ...f, + box: f.box == null ? null : {x: f.box.y, y: 0, width: 12, height: 12}, + }) as TrackedFrame, + ); +} + +/** + * A frame-counted loop: `step` px a frame, standing still from `lag` ms into + * each window to its end, then going on without catching up. + */ +function frameCounted( + windows: readonly {from: number; to: number}[], + lag: number, + step = 1, +): TrackedFrame[] { + let y = 0; + return moving().map((f, i) => { + if (i > 0 && !windows.some(w => f.t >= w.from + lag && f.t < w.to)) { + y += step; + } + return {...f, box: {...f.box!, y}} as TrackedFrame; + }); +} + +describe('busyWindowProof against a control', () => { + it('passes a component that moves on every frame the control does, through a stall the compositor makes for both, which fails without the control', () => { + // Three frames in a row presented nothing, for the control too: the + // compositor shared the core the block spins on (a two-core runner). + const gap = [at(1300), at(1300) + 1, at(1300) + 2]; + const component = moving(gap); + const control = asControl(moving(gap)); + const alone = busyWindowProof(component, BLOCK); + expect(alone.verdict.failures.join('\n')).toMatch(/longest stall/); + const against = busyWindowProof(component, BLOCK, {control}); + expect(against.verdict.failures, against.summary).toEqual([]); + expect(against.control).toMatchObject({moved: against.control?.movedWith}); + expect(against.summary).toMatch( + /against the control: moved on \d+ of its \d+ frames/, + ); + }); + + it('still fails a frozen component while the control moves', () => { + const frozen = moving().map( + f => + ({ + ...f, + box: { + ...f.box!, + y: f.t >= BLOCK.from ? 3 * at(BLOCK.from) : f.box!.y, + }, + }) as TrackedFrame, + ); + const proof = busyWindowProof(frozen, BLOCK, { + control: asControl(moving()), + }); + expect(proof.verdict.pass).toBe(false); + expect(proof.verdict.failures.join('\n')).toMatch( + /moved on 0 of the \d+ frames the control moved/, + ); + expect(proof.verdict.failures.join('\n')).toMatch( + /longer than the control's/, + ); + }); + + it('fails a component the control outpaces: it repeats positions on frames the control moves', () => { + const repeats = [0, 1, 2, 3].map(k => at(1100) + 6 * k); + const proof = busyWindowProof(repeating(repeats), BLOCK, { + control: asControl(moving()), + }); + expect(proof.verdict.failures.join('\n')).toMatch( + /moved on \d+ of the \d+ frames the control moved/, + ); + }); + + it('fails a component that misses a single frame the control moved', () => { + // One repeated position: a frame the control moved on and the component + // did not, which a compositor animation never shows, since both are read + // from the same screencast frames. + const proof = busyWindowProof(repeating([at(1300)]), BLOCK, { + control: asControl(moving()), + }); + expect(proof.verdict.failures.join('\n')).toMatch( + /moved on \d+ of the \d+ frames the control moved/, + ); + expect(proof.control!.moved - proof.control!.movedWith).toBe(1); + }); + + it('counts unchanged frames on a tail under one cast step a frame as moved: each run of them ends in a single step', () => { + // 3 px a frame, then half a pixel a frame from 1400 ms: the whole-pixel + // positions change every other frame, one step at a time. + const k = at(1400); + const tail = moving().map((f, i) => + i < k + ? f + : ({ + ...f, + box: {...f.box!, y: 3 * k + Math.floor((i - k) / 2)}, + } as TrackedFrame), + ); + const proof = busyWindowProof(tail, BLOCK, {control: asControl(moving())}); + expect(proof.verdict.failures, proof.summary).toEqual([]); + expect(proof.control?.belowStep).toBeGreaterThan(0); + expect(proof.summary).toMatch(/under one cast step a frame/); + }); + + it('still fails a slow component frozen on frames the control moved on: the change that ends the run catches up more than one step', () => { + // Half a pixel a frame throughout, frozen for 12 frames from 1200 ms, then + // back where its clock says: the change that ends the run is 6 steps. + const k = at(1200); + const slow = moving().map( + (f, i) => + ({ + ...f, + box: { + ...f.box!, + y: Math.floor((i >= k && i < k + 12 ? k - 1 : i) / 2), + }, + }) as TrackedFrame, + ); + const proof = busyWindowProof(slow, BLOCK, {control: asControl(moving())}); + expect(proof.verdict.failures.join('\n')).toMatch( + /moved on \d+ of the \d+ frames the control moved/, + ); + }); + + it('still fails a component that stops for good while the control moves: no change ends its run', () => { + const k = at(1300); + const stopped = moving().map((f, i) => + i < k ? f : ({...f, box: {...f.box!, y: 3 * k}} as TrackedFrame), + ); + const proof = busyWindowProof(stopped, BLOCK, { + control: asControl(moving()), + }); + expect(proof.verdict.failures.join('\n')).toMatch( + /moved on \d+ of the \d+ frames the control moved/, + ); + }); + + it('still fails a frame-counted component that stands still through the block and resumes one step a frame: it shows too little motion to judge', () => { + // 1 px a frame, counted per frame rather than by the clock: it stands + // still from the block's start to its end, then goes on one step a + // frame, so the change that ends the run is a single step. + const s0 = at(BLOCK.from); + const s1 = at(BLOCK.to); + const counted = moving().map( + (f, i) => + ({ + ...f, + box: { + ...f.box!, + y: i < s0 ? i : i <= s1 ? s0 - 1 : s0 - 1 + (i - s1), + }, + }) as TrackedFrame, + ); + const proof = busyWindowProof(counted, BLOCK, { + control: asControl(moving()), + }); + expect(proof.verdict.failures.join('\n')).toMatch( + /too little motion in this window to judge/, + ); + }); + + it('still fails a frame-counted loop that stalls for a few frames inside the block: its run is longer than the runs before it', () => { + // 1 px a frame, so each step is one cast step, standing still for three + // frames from 1300 ms and going on from there without catching up. + const proof = busyWindowProof( + frameCounted([{from: 1300, to: 1350}], 0), + BLOCK, + {control: asControl(moving())}, + ); + expect(proof.verdict.failures.join('\n')).toMatch( + /moved on \d+ of the \d+ frames the control moved/, + ); + expect(proof.control?.belowStep).toBe(0); + }); + + it('reads the finest cast step when the judged part shows no change, so a larger jump never passes for one step', () => { + const frozen = moving().map( + f => + ({...f, box: {...f.box!, y: f.t < BLOCK.to ? 0 : 2}}) as TrackedFrame, + ); + const proof = busyWindowProof(frozen, BLOCK, { + control: asControl(moving()), + }); + expect(proof.stepPx).toBe(0.5); + expect(proof.control?.belowStep).toBe(0); + }); + + it('reads the cast step from changes, so a position committed off the grid once the block ends does not shrink it', () => { + // An ease-out on a 2 px grid, as WebKit casts one, that ends on 401 once + // the block is over: its tail shows runs of unchanged frames that end in + // a single 2 px step. + const eased = moving().map( + (f, i) => + ({ + ...f, + box: { + ...f.box!, + y: + f.t > BLOCK.to + 20 + ? 401 + : 2 * Math.round((400 - 400 * 0.93 ** i) / 2), + }, + }) as TrackedFrame, + ); + const proof = busyWindowProof(eased, BLOCK, { + control: asControl(moving()), + }); + expect(proof.stepPx).toBe(2); + expect(proof.verdict.failures, proof.summary).toEqual([]); + expect(proof.control?.belowStep).toBeGreaterThan(0); + }); + + it('fails a window in which the control itself presented too little to judge', () => { + const still = asControl( + moving().map(f => ({...f, box: {...f.box!, y: 0}}) as TrackedFrame), + ); + const proof = busyWindowProof(moving(), BLOCK, {control: still}); + expect(proof.verdict.failures.join('\n')).toMatch( + /the control moved on 0 frames in the judged part/, + ); + }); +}); + +/** A 90 ms block from `from`: its judged part holds three 60 Hz frames. */ +const short = (from: number) => ({from, to: from + 90}); + +describe('pooledBusyProof', () => { + it('judges a block too short for one run over pooled runs: one run cannot decide, three runs of a component that moves with the control pass', () => { + const control = asControl(moving()); + const single = busyWindowProof(moving(), short(1000), {control}); + expect(single.verdict.failures.join('\n')).toMatch( + /the control moved on 3 frames in the judged part, under 4/, + ); + const runs = [1000, 1300, 1600].map(from => ({ + tracked: moving(), + control, + window: short(from), + })); + const pooled = pooledBusyProof(runs); + expect(pooled.verdict.failures, pooled.summary).toEqual([]); + expect(pooled.control).toEqual({moved: 9, movedWith: 9}); + expect(pooled.runs).toHaveLength(3); + expect(pooled.summary).toMatch( + /^3 runs pooled: moved on 9 of the control's 9 frames; run 1: /, + ); + }); + + it('still fails a component that is frozen in every run while the control moves', () => { + const frozen = moving().map( + f => ({...f, box: {...f.box!, y: 0}}) as TrackedFrame, + ); + const pooled = pooledBusyProof( + [1000, 1300, 1600].map(from => ({ + tracked: frozen, + control: asControl(moving()), + window: short(from), + })), + ); + expect(pooled.verdict.pass).toBe(false); + expect(pooled.verdict.failures.join('\n')).toMatch( + /moved on 0 of the 9 frames the control moved over 3 runs/, + ); + expect(pooled.verdict.failures.join('\n')).toMatch( + /run 1: longest stall .* longer than the control's/, + ); + }); + + it("judges each run's longest stall against that run's control, never pooled", () => { + // Two frames in a row present no new position, inside run 2 only. + const k = at(1320); + const stalled = moving().map((f, i) => + i === k || i === k + 1 + ? ({...f, box: {...f.box!, y: 3 * (k - 1)}} as TrackedFrame) + : f, + ); + const pooled = pooledBusyProof( + [1000, 1300, 1600].map(from => ({ + tracked: stalled, + control: asControl(moving()), + window: short(from), + })), + ); + const failures = pooled.verdict.failures.join('\n'); + expect(failures).toMatch( + /run 2: longest stall .* longer than the control's/, + ); + expect(failures).not.toMatch(/run [13]: longest stall/); + }); + + it('fails a pool in which the component misses a single frame the control moved', () => { + const pooled = pooledBusyProof( + [1000, 1300, 1600].map(from => ({ + tracked: repeating([at(1320)]), + control: asControl(moving()), + window: short(from), + })), + ); + expect(pooled.verdict.failures.join('\n')).toMatch( + /moved on \d+ of the \d+ frames the control moved over 3 runs/, + ); + expect(pooled.control.moved - pooled.control.movedWith).toBe(1); + }); + + it("counts a pooled run's unchanged frames under one cast step a frame as moved, as one window does", () => { + const k = at(1000); + const tail = moving().map((f, i) => + i < k + ? f + : ({ + ...f, + box: {...f.box!, y: 3 * k + Math.floor((i - k) / 2)}, + } as TrackedFrame), + ); + const pooled = pooledBusyProof( + [1000, 1300, 1600].map(from => ({ + tracked: tail, + control: asControl(moving()), + window: short(from), + })), + ); + expect(pooled.verdict.failures, pooled.summary).toEqual([]); + }); + + it('still fails a pooled frame-counted loop that the cast shows moving early in each judged part', () => { + // Six 70 ms blocks; the cast shows the loop's last frames up to 35 ms into + // each, then it stands still to the block's end and goes on one step a + // frame: a 1 px loop's steps are one cast step, and a 2 px loop's read + // as one when its changes are all 2 px. + const windows = [1000, 1150, 1300, 1450, 1600, 1750].map(from => ({ + from, + to: from + 70, + })); + for (const step of [1, 2]) { + const pooled = pooledBusyProof( + windows.map(window => ({ + tracked: frameCounted(windows, 35, step), + control: asControl(moving()), + window, + })), + ); + expect(pooled.verdict.failures.join('\n'), `${step} px`).toMatch( + /moved on \d+ of the \d+ frames the control moved over 6 runs/, + ); + } + }); + + it('judges the control over the pool too: runs in which it presented too little fail', () => { + const still = asControl( + moving().map(f => ({...f, box: {...f.box!, y: 0}}) as TrackedFrame), + ); + const pooled = pooledBusyProof( + [1000, 1300].map(from => ({ + tracked: moving(), + control: still, + window: short(from), + })), + ); + expect(pooled.verdict.failures.join('\n')).toMatch( + /the control moved on 0 frames over 2 runs, under 4/, + ); + }); + + it('judges one long run as busyWindowProof judges it against the control', () => { + const control = asControl(moving()); + const pooled = pooledBusyProof([ + {tracked: moving(), control, window: BLOCK}, + ]); + const single = busyWindowProof(moving(), BLOCK, {control}); + expect(pooled.verdict).toEqual(single.verdict); + expect(pooled.control).toEqual({ + moved: single.control?.moved, + movedWith: single.control?.movedWith, + }); + }); + + it('throws on no runs', () => { + expect(() => pooledBusyProof([])).toThrow(/no runs/); + }); +}); + +describe('a hold that starts before the block', () => { + // A sheet's release in WebKit: moving until 120 ms before the block, held + // (the just-started animation not presented yet) until 100 ms into it, + // then moving on every frame. + const holdFrom = BLOCK.from - 120; + const holdTo = BLOCK.from + 100; + const sheet = ({freezeAfterFirst = false} = {}) => { + let y = 0; + let changed = false; + return moving().map(f => { + if (f.t < holdFrom) { + y += 3; + } else if (f.t >= holdTo && (!freezeAfterFirst || !changed)) { + y += 3; + changed = true; + } + return {...f, box: {...f.box!, y}} as TrackedFrame; + }); + }; + const control = asControl(moving()); + + it('is judged from its first presented change inside the block', () => { + const proof = busyWindowProof(sheet(), BLOCK, {axis: 'y', control}); + expect(proof.verdict.failures, proof.summary).toEqual([]); + expect(proof.onScreenFromMs).toBeGreaterThanOrEqual(100); + }); + + it('still fails when the element changes once inside the block and then freezes', () => { + const proof = busyWindowProof(sheet({freezeAfterFirst: true}), BLOCK, { + axis: 'y', + control, + }); + expect(proof.verdict.pass, proof.summary).toBe(false); + }); + + it('is not excused when it starts inside the block', () => { + let y = 0; + const frames = moving().map(f => { + if (f.t < BLOCK.from + 50 || f.t >= BLOCK.from + 250) { + y += 3; + } + return {...f, box: {...f.box!, y}} as TrackedFrame; + }); + const proof = busyWindowProof(frames, BLOCK, {axis: 'y', control}); + expect(proof.verdict.pass, proof.summary).toBe(false); + expect(proof.onScreenFromMs).toBeNull(); + }); +}); diff --git a/internal/interaction-harness/src/busy.ts b/internal/interaction-harness/src/busy.ts index 55fd9d670d52a..918834f7cd164 100644 --- a/internal/interaction-harness/src/busy.ts +++ b/internal/interaction-harness/src/busy.ts @@ -4,8 +4,11 @@ * @file busy.ts * @input Uses a Playwright `Page` * @output `blockMainThread` — a synchronous busy loop on the page's main thread - * that reports exactly when it ran — and `busyWindowProof`, which reads frame - * evidence for that window. + * that reports exactly when it ran —, `startBusyControl` — a plain + * compositor-only control animation recorded in the same window —, + * `busyWindowProof`, which reads frame evidence for that window, against + * the control when it has one, and `pooledBusyProof`, which judges several + * short blocks as one. * @position The busy-main-thread proof: an animation that runs on the * compositor keeps advancing at display rate while JavaScript holds the main * thread; one driven from the main thread (rAF, style writes, layout @@ -18,6 +21,138 @@ import type {Locator, Page} from '@playwright/test'; import {frameStats, type FrameStats, type TrackedFrame} from './frames'; +/** The control's colour, apart from the component's marker (`MARKER_RGB`). */ +export const CONTROL_RGB: readonly [number, number, number] = [0, 255, 255]; + +/** The control box's side, and how far it travels along the top edge, CSS px. */ +const CONTROL_SIDE = 12; +const CONTROL_TRAVEL = 240; + +/** + * Starts a plain compositor-only control animation: a `CONTROL_RGB` box in the + * page's top-left corner whose transform moves it back and forth along the top + * edge (`CONTROL_TRAVEL` px every 400 ms, linear) for as long as the page + * runs. It shows what the compositor presented in a busy window on this + * machine, so a proof can judge a component against it rather than against a + * frame rate: track it from the same screencast (`trackMarker(page, frames, + * {rgb: CONTROL_RGB})`) and pass it to `busyWindowProof` as `control`. + * + * The box is a manual popover, so it shows in the top layer, above a modal + * dialog the page opened before it; `raise()` shows it again on top of one + * opened after. With `within`, it is mounted inside that element instead (an + * open modal dialog), as a plain fixed box with no top-layer entry, for a + * proof of the dialog's `::backdrop`: in WebKit, with the popover above + * BottomSheet's open modal dialog, the sheet's dim held for 155 to 225 ms + * into a block while the control moved. `waitForAnimation` and + * `blockMainThread`'s `during` never count the control's own animation, so + * they still wait for the dialog's. Start it before the block: it + * resolves once the animation is running and past its first frame (on the + * compositor; see `blockMainThread`). Keep the component's marker out of the + * strip it crosses (the top `CONTROL_SIDE` px, `CONTROL_SIDE + + * CONTROL_TRAVEL` px wide). + */ +export async function startBusyControl( + page: Page, + { + rgb = CONTROL_RGB, + within, + }: {rgb?: readonly [number, number, number]; within?: Locator} = {}, +): Promise<{ + readonly raise: () => Promise; + readonly remove: () => Promise; +}> { + if (within != null) { + await within.evaluate(host => + host.setAttribute('data-harness-control-host', ''), + ); + } + await page.evaluate( + ({color, side, travel}) => + new Promise((resolve, reject) => { + const host = document.querySelector('[data-harness-control-host]'); + host?.removeAttribute('data-harness-control-host'); + const box = document.createElement('div'); + box.setAttribute('data-harness-control', ''); + box.style.cssText = `position:fixed;inset:auto;left:0;top:0;margin:0;padding:0;border:0;overflow:visible;width:${side}px;height:${side}px;background:rgb(${color.join(',')});z-index:2147483647;pointer-events:none;`; + if (host == null) { + box.setAttribute('popover', 'manual'); + document.body.appendChild(box); + box.showPopover(); + } else { + host.appendChild(box); + } + const animation = box.animate( + [ + {transform: 'translateX(0px)'}, + {transform: `translateX(${travel}px)`}, + ], + { + duration: 400, + iterations: Infinity, + direction: 'alternate', + easing: 'linear', + }, + ); + // The harness's own waits run on the real timers (see blockMainThread). + const gate = ( + window as unknown as { + __harnessTimerGate?: {frame?: typeof requestAnimationFrame}; + } + ).__harnessTimerGate; + const frame = gate?.frame ?? window.requestAnimationFrame.bind(window); + const deadline = performance.now() + 5_000; + const tick = () => { + if (!animation.pending && Number(animation.currentTime ?? 0) >= 16) { + resolve(); + } else if (performance.now() > deadline) { + reject(new Error('startBusyControl: the control did not start')); + } else { + frame(tick); + } + }; + frame(tick); + }), + {color: [...rgb], side: CONTROL_SIDE, travel: CONTROL_TRAVEL}, + ); + return { + raise: () => + page.evaluate(() => { + for (const box of document.querySelectorAll( + '[data-harness-control][popover]', + )) { + box.hidePopover(); + box.showPopover(); + } + }), + remove: () => + page.evaluate(() => { + for (const box of document.querySelectorAll('[data-harness-control]')) { + box.remove(); + } + }), + }; +} + +/** What the control presented in the judged part (see `busyWindowProof`). */ +export interface BusyControlRead { + /** Frames in the judged part that presented a new control position. */ + readonly moved: number; + /** Of those, the frames that also presented a new component position. */ + readonly movedWith: number; + /** The control's longest time without a new position, the edges included. */ + readonly longestStallMs: number; + /** + * Of `movedWith`, the frames counted as moved because the component's motion + * there was under one cast step per frame (see `busyWindowProof`). + */ + readonly belowStep: number; + /** + * The component's longest time without a new position, read the same way: + * a run of unchanged frames under one cast step per frame is motion. + */ + readonly componentStallMs: number; +} + export interface BusyWindow { /** Epoch ms when the loop started and ended, on the page's own clock. */ readonly from: number; @@ -43,8 +178,9 @@ export interface BusyWindow { * holds that hand-off, and even a compositor animation freezes. To catch a * settle near its start, call it before the release that starts the settle and * await it after: the page is already waiting when the animation begins. Any - * animation inside `during` counts, so pick an element that runs nothing - * else meanwhile. + * animation inside `during` counts but the busy-window control's own + * (`startBusyControl`, mounted inside a dialog with `within`), so pick an + * element that runs nothing else meanwhile. * * An animation counts once it is no longer pending and has run at least one * frame (`minProgressMs` below 16 ms is raised to 16). One created in the @@ -94,11 +230,21 @@ export async function blockMainThread( const frame = gate?.frame ?? window.requestAnimationFrame.bind(window); const later = gate?.timeout ?? window.setTimeout.bind(window); const deadline = performance.now() + wait; + // The busy-window control animates on its own, inside a dialog with + // `within`: it never stands for the animation under test. + const isControl = (a: Animation) => { + const target = (a.effect as KeyframeEffect | null)?.target ?? null; + return ( + target instanceof Element && + target.closest('[data-harness-control]') != null + ); + }; const tick = () => { const running = element .getAnimations({subtree: true}) .some( a => + !isControl(a) && a.playState === 'running' && !a.pending && Number(a.currentTime ?? 0) >= minProgress, @@ -149,7 +295,8 @@ export interface BusyProof extends FrameStats { readonly onScreenFromMs: number | null; /** * The cast's position step, CSS px: the largest of 0.5, 1 and 2 that every - * tracked position is a multiple of (WebKit's casts are 2: see the README). + * change of the tracked value in the judged part is a multiple of, or 0.5 + * when it does not change there (WebKit's casts step 2: see the README). */ readonly stepPx: number; /** The tracked value's mean motion per screencast interval in the judged part, px. */ @@ -206,6 +353,8 @@ export interface BusyProof extends FrameStats { * frames were delivered. */ readonly movedFraction: number; + /** With a `control`: what it presented in the judged part; null without one. */ + readonly control: BusyControlRead | null; /** The judgment against the limits (see `busyWindowProof`). */ readonly verdict: BusyVerdict; /** One line with the raw numbers, for an annotation or a failure message. */ @@ -244,10 +393,17 @@ export interface BusyProof extends FrameStats { * tracked value was seen at rest before the judged part (a frame there, and no * change in the 250 ms before it), judging starts at its first presented * change inside the block, which the - * proof reports as `onScreenFromMs`. The judged part still ends with the - * block and the `minJudgedFrames` floor applies to it, so a start the block - * itself delays fails; an animation that was already moving is judged from - * the block's start, and a stall there fails as before. + * proof reports as `onScreenFromMs`. So does a hold that starts before the + * block (two frames at least at one position, the first before it) and ends + * at the first presented change inside it: WebKit presents a just-started + * compositor animation late (a sheet's release held on screen for 130 to + * 300 ms while its timeline ran, and the screencast reaches Node about 0.7 s + * late on a loaded runner, too late to wait on before blocking). Every frame + * after that first change is judged, so a freeze later in the block fails. + * The judged part still ends with the block and the `minJudgedFrames` floor + * applies to it, so a start the block itself delays fails; any other + * animation that was already moving is judged from the block's start, and a + * stall there fails as before. * * `stepPx` and `pxPerFrame` say whether a frame's motion is resolvable at * all: under two position steps per frame, a frame delivered early can show @@ -262,6 +418,30 @@ export interface BusyProof extends FrameStats { * that opens or closes the judged part, or holds a delivered frame that did * not move, is judged from the screencast as before, so a frozen element * fails whatever else the compositor draws. + * + * With a `control` (`startBusyControl`'s box, tracked from the same + * screencast), the window is judged against what the compositor presented in + * it rather than against a frame rate: on a two-core runner the software + * compositor and the screencast share the core the block spins on, so every + * compositor animation presents fewer frames there, the control too. The + * component passes when: + * - the control moved on at least `minJudgedFrames` frames of the judged part + * (fewer: the machine presented too little to judge, which fails); + * - the component moved on every frame the control moved: both are read from + * the same screencast frames, so a compositor animation misses none; + * - its longest stall is no longer than the control's. + * A run of frames with no new component position counts as moved when the + * change that ends it is a single cast step (`stepPx`): the component moves + * under one step per frame there, as on an easing tail, which the cast cannot + * show on every frame. The run also has to read as a slowing curve, at most + * one frame longer than the component's run before it. A freeze ends in a + * catch-up of more than one step, a stall in a frame-counted animation is + * longer than its runs before it, and a run that no change ends stays a miss. + * The component still has to show a new position on at least + * `minJudgedFrames` of the control's frames. + * A main-thread animation still fails: the control moves and it does not. + * `maxMedianFrameMs`, when given, still applies; `minMovedFraction` and the + * absolute stall, frame and motion limits do not. */ export function busyWindowProof( tracked: readonly TrackedFrame[], @@ -276,6 +456,7 @@ export function busyWindowProof( minMovedFraction = 0.9, minJudgedFrames = 4, presented, + control, }: { axis?: 'x' | 'y' | 'width' | 'height'; marginMs?: number; @@ -316,6 +497,11 @@ export function busyWindowProof( * judged alone. */ presented?: readonly number[]; + /** + * The control's frames (`startBusyControl`, tracked from the same + * screencast with `CONTROL_RGB`): judge the component against them. + */ + control?: readonly TrackedFrame[]; } = {}, ): BusyProof { let end = window.to - marginMs; @@ -369,8 +555,40 @@ export function busyWindowProof( const movedBefore = lead.some( (value, i) => i > leadFrom && Math.abs(value - lead[i - 1]) > 0.25, ); + // Held from before the block to its first presented change inside it: in + // WebKit a just-started compositor animation is presented late (a sheet's + // release held on screen for 130 to 300 ms while its timeline ran, and the + // screencast reaches Node about 0.7 s late on a loaded runner, too late to + // wait on before blocking), so a hold that STARTS before the block is the + // engine's, and judging starts at that first change. Only such a hold: from + // that change on every frame is judged, so a freeze later in the block + // still fails. The hold shows on two frames at least, the first before the + // block. + let holdStart = lead.length - 1; + while ( + holdStart > 0 && + Math.abs(lead[holdStart - 1] - lead[lead.length - 1]) <= 0.25 + ) { + holdStart -= 1; + } + const heldFrames = + lead.length - + holdStart + + values.filter( + v => + v.value != null && + v.t < (inWindow[0] ?? -Infinity) && + Math.abs(v.value - lead[lead.length - 1]) <= 0.25, + ).length; + const heldFromBeforeBlock = + lead.length > 0 && + holdStart > 0 && + prior[holdStart].t < window.from && + heldFrames >= 2; const onScreenFrom = - prior.length > 0 && !movedBefore && inWindow.length > 0 + prior.length > 0 && + (!movedBefore || heldFromBeforeBlock) && + inWindow.length > 0 ? inWindow[0] : null; const judged = {from: onScreenFrom ?? inner.from, to: inner.to}; @@ -479,12 +697,29 @@ export function busyWindowProof( ? expectedFrames : deliveredFrames; const movedFraction = (moved + recorderSkipped) / Math.max(1, judgedFrames); - // The cast's position step, and the motion per interval it has to resolve. - const positions = tracked.flatMap(f => (f.box == null ? [] : [f.box[axis]])); + // The cast's position step for the tracked value: the largest of 2, 1 and + // 0.5 CSS px that every change between its consecutive positions in the + // judged part is a multiple of. Changes rather than positions: a position + // the page commits once the block ends can sit off that grid. + const partValues = tracked.flatMap(f => + f.box != null && f.t >= judged.from && f.t <= judged.to + ? [f.box[axis]] + : [], + ); + const stepChanges = partValues + .slice(1) + .map((p, i) => Math.abs(p - partValues[i])) + .filter(d => d > 0.25); + // With no change to read, the finest step: a coarser one would let a larger + // jump pass for a single step. const stepPx = - [2, 1, 0.5].find(step => - positions.every(p => Math.abs(p / step - Math.round(p / step)) < 0.01), - ) ?? 0.5; + stepChanges.length === 0 + ? 0.5 + : ([2, 1, 0.5].find(step => + stepChanges.every( + d => Math.abs(d / step - Math.round(d / step)) < 0.01, + ), + ) ?? 0.5); const judgedValues = tracked.flatMap(f => f.box != null && f.t >= judged.from && f.t <= judged.to ? [{t: f.t, value: f.box[axis]}] @@ -512,20 +747,123 @@ export function busyWindowProof( `${pxPerFrame.toFixed(1)} px/frame is under two of the cast's ${stepPx} px position steps: a frame delivered early can show no step and read as not moved`, ); } + // The control's own new positions in the judged part, against its frame + // before the part, and of those the frames the component moved on too: + // both are tracked from the same screencast frames, so their times match. + let controlRead: BusyControlRead | null = null; + if (control != null) { + const baseline = [...control] + .reverse() + .find(f => f.box != null && f.t < inner.from); + let last: number | null = baseline?.box == null ? null : baseline.box.x; + const controlChanges: number[] = []; + for (const f of control) { + if (f.box == null || f.t < inner.from || f.t > judged.to) { + continue; + } + if (last != null && Math.abs(f.box.x - last) > 0.25) { + if (f.t > judged.from || onScreenFrom == null) { + controlChanges.push(f.t); + } + } + last = f.box.x; + } + const controlEdges = [judged.from, ...controlChanges, judged.to]; + // A frame with no new component position, in a run of such frames whose + // next change is a single cast step: the component moves under one step + // per frame there, which the cast cannot show on every frame, so the + // frame counts as moved. A freeze ends in a catch-up of more than one + // step, and a run that no change ends stays a miss. + const series = tracked.flatMap(f => + f.box == null ? [] : [{t: f.t, value: f.box[axis]}], + ); + const seriesIndex = new Map(series.map((f, i) => [f.t, i])); + const same = (a: number, b: number) => + Math.abs(series[a].value - series[b].value) <= 0.25; + // The first frame of the run of equal positions that holds frame i. + const runStart = (i: number): number => { + let s = i; + while (s > 0 && same(s - 1, i)) { + s -= 1; + } + return s; + }; + const underStep = (t: number): boolean => { + const i = seriesIndex.get(t); + if (i == null || changed.has(t)) { + return false; + } + const start = runStart(i); + let k = i + 1; + while (k < series.length && same(k, i)) { + k += 1; + } + if ( + k >= series.length || + Math.abs(series[k].value - series[i].value) >= 1.5 * stepPx + ) { + return false; + } + // A slowing curve lengthens its runs a frame at a time: a run more than + // one frame longer than the one before it is a stall. + const unchanged = k - 1 - start; + const before = start > 0 ? start - 1 - runStart(start - 1) : 0; + return unchanged <= before + 1; + }; + const componentEdges = [ + judged.from, + ...delivered.filter(t => changed.has(t) || underStep(t)), + judged.to, + ]; + controlRead = { + moved: controlChanges.length, + movedWith: controlChanges.filter(t => changed.has(t) || underStep(t)) + .length, + belowStep: controlChanges.filter(underStep).length, + longestStallMs: Math.max( + ...controlEdges.slice(1).map((t, i) => t - controlEdges[i]), + ), + componentStallMs: Math.max( + ...componentEdges.slice(1).map((t, i) => t - componentEdges[i]), + ), + }; + } const failures: string[] = []; - if (stats.frames === 0) { + if (stats.frames === 0 && controlRead == null) { failures.push('no frame presented in the judged part'); } else if (stats.medianFrameMs > maxMedianFrameMs) { failures.push( `median frame interval ${stats.medianFrameMs.toFixed(1)} ms > ${maxMedianFrameMs} ms (${stats.frames} frames presented)`, ); } - if (longestStallFrames > maxStallFrames) { + if (controlRead != null) { + if (controlRead.moved < minJudgedFrames) { + failures.push( + `the control moved on ${controlRead.moved} frames in the judged part, under ${minJudgedFrames}: the compositor presented too little in this window to judge`, + ); + } + const visible = controlRead.movedWith - controlRead.belowStep; + if (visible < minJudgedFrames) { + failures.push( + `the component showed a new position on ${visible} of the control's frames, under ${minJudgedFrames}: too little motion in this window to judge`, + ); + } + if (controlRead.movedWith < controlRead.moved) { + failures.push( + `moved on ${controlRead.movedWith} of the ${controlRead.moved} frames the control moved (${((controlRead.movedWith / Math.max(1, controlRead.moved)) * 100).toFixed(0)}%)`, + ); + } + if (controlRead.componentStallMs > controlRead.longestStallMs) { + failures.push( + `longest stall ${controlRead.componentStallMs.toFixed(1)} ms, longer than the control's ${controlRead.longestStallMs.toFixed(1)} ms`, + ); + } + } else if (longestStallFrames > maxStallFrames) { failures.push( `longest stall ${longestStallMs.toFixed(1)} ms = ${longestStallFrames} intervals > ${maxStallFrames}`, ); } - if (expectedFrames < minJudgedFrames) { + if (controlRead == null && expectedFrames < minJudgedFrames) { failures.push( `${(judged.to - judged.from).toFixed(0)} ms judged = ${expectedFrames} frames < ${minJudgedFrames}: the animation under test was not moving during the block`, ); @@ -534,7 +872,10 @@ export function busyWindowProof( 1, Math.floor((1 - minMovedFraction) * judgedFrames + 1e-9), ); - if (judgedFrames - moved - recorderSkipped > allowedMisses) { + if ( + controlRead == null && + judgedFrames - moved - recorderSkipped > allowedMisses + ) { failures.push( `${moved} of ${judgedFrames} expected frames moved${recorderSkipped > 0 ? ` (and ${recorderSkipped} the screencast skipped)` : ''} (${(movedFraction * 100).toFixed(0)}% < ${(minMovedFraction * 100).toFixed(0)}%)`, ); @@ -554,6 +895,9 @@ export function busyWindowProof( (onScreenFromMs == null ? '' : `, on screen from +${onScreenFromMs.toFixed(0)} ms (nothing moved before the block)`) + + (controlRead == null + ? '' + : `; against the control: moved on ${controlRead.movedWith} of its ${controlRead.moved} frames${controlRead.belowStep > 0 ? ` (${controlRead.belowStep} under one cast step a frame)` : ''}, its longest stall ${controlRead.longestStallMs.toFixed(1)} ms`) + notes.map(note => `; note: ${note}`).join(''); return { ...stats, @@ -573,6 +917,109 @@ export function busyWindowProof( longestStallFrames, moved, movedFraction, + control: controlRead, + verdict: {pass: failures.length === 0, failures}, + summary, + }; +} + +/** One run of a pooled busy proof (see `pooledBusyProof`). */ +export interface BusyRun { + /** The component's frames, tracked from the run's screencast. */ + readonly tracked: readonly TrackedFrame[]; + /** The control's frames (`startBusyControl`), from the same screencast. */ + readonly control: readonly TrackedFrame[]; + /** The run's block (`blockMainThread`). */ + readonly window: BusyWindow; +} + +export interface PooledBusyProof { + /** + * Each run judged on its own against its control: read its numbers, not its + * verdict (one short run can hold too few frames to judge). + */ + readonly runs: readonly BusyProof[]; + /** + * The control's new positions over every run's judged part, and those the + * component moved on too. + */ + readonly control: {readonly moved: number; readonly movedWith: number}; + /** The judgment over the pool (see `pooledBusyProof`). */ + readonly verdict: BusyVerdict; + /** One line with the pooled numbers, then each run's summary. */ + readonly summary: string; +} + +/** + * A busy proof over several runs of the same short animation, against the + * control. An animation that lasts a fraction of a second leaves a block only + * a few frames to judge: in WebKit, or on a loaded machine, the control can + * present fewer than `minJudgedFrames` in one run, and one run cannot decide. + * Pooled, the runs are judged as one long window is: + * - the control moved on at least `minJudgedFrames` frames over all the runs; + * - the component moved on every one of those frames; + * - in each run, its longest stall is no longer than that run's control's: a + * stall belongs to its run and is never pooled. + * `maxMedianFrameMs`, when given, applies to each run. A main-thread + * animation still fails: in every run the control moves and it does not. + */ +export function pooledBusyProof( + runs: readonly BusyRun[], + options: Omit< + NonNullable[2]>, + 'control' | 'minMovedFraction' + > = {}, +): PooledBusyProof { + if (runs.length === 0) { + throw new Error('pooledBusyProof: no runs to judge'); + } + const {minJudgedFrames = 4, maxMedianFrameMs = Infinity} = options; + const proofs = runs.map(run => + busyWindowProof(run.tracked, run.window, { + ...options, + control: run.control, + }), + ); + const reads = proofs.map(proof => proof.control as BusyControlRead); + const moved = reads.reduce((sum, read) => sum + read.moved, 0); + const movedWith = reads.reduce((sum, read) => sum + read.movedWith, 0); + const failures: string[] = []; + if (moved < minJudgedFrames) { + failures.push( + `the control moved on ${moved} frames over ${runs.length} runs, under ${minJudgedFrames}: the compositor presented too little in these windows to judge`, + ); + } + const visible = + movedWith - reads.reduce((sum, read) => sum + read.belowStep, 0); + if (visible < minJudgedFrames) { + failures.push( + `the component showed a new position on ${visible} of the control's frames over ${runs.length} runs, under ${minJudgedFrames}: too little motion in these windows to judge`, + ); + } + if (movedWith < moved) { + failures.push( + `moved on ${movedWith} of the ${moved} frames the control moved over ${runs.length} runs (${((movedWith / Math.max(1, moved)) * 100).toFixed(0)}%)`, + ); + } + proofs.forEach((proof, i) => { + if (proof.frames > 0 && proof.medianFrameMs > maxMedianFrameMs) { + failures.push( + `run ${i + 1}: median frame interval ${proof.medianFrameMs.toFixed(1)} ms > ${maxMedianFrameMs} ms`, + ); + } + const read = reads[i]; + if (read.componentStallMs > read.longestStallMs) { + failures.push( + `run ${i + 1}: longest stall ${read.componentStallMs.toFixed(1)} ms, longer than the control's ${read.longestStallMs.toFixed(1)} ms`, + ); + } + }); + const summary = + `${runs.length} runs pooled: moved on ${movedWith} of the control's ${moved} frames` + + proofs.map((proof, i) => `; run ${i + 1}: ${proof.summary}`).join(''); + return { + runs: proofs, + control: {moved, movedWith}, verdict: {pass: failures.length === 0, failures}, summary, }; diff --git a/internal/interaction-harness/src/curves.ts b/internal/interaction-harness/src/curves.ts index 52d187ea0a6a3..e4dd987203a55 100644 --- a/internal/interaction-harness/src/curves.ts +++ b/internal/interaction-harness/src/curves.ts @@ -981,12 +981,23 @@ export async function waitForAnimation( (target === element || element.contains(target)) ); }; + // The busy-window control (busy.ts) animates on its own, inside a + // dialog with `within`: it never answers for the component. + const isControl = (a: Animation) => { + const target = + (a.effect as KeyframeEffect | null)?.target ?? null; + return ( + target instanceof Element && + target.closest('[data-harness-control]') != null + ); + }; const after = watchedFrom ?? recorder.armedAt(); // Document-timeline animations only, as for holdAnimations: a scroll- // or view-timeline one that reached its end is not a finished curve. const recorded = recorder.log.filter( e => e.seenAt >= after && + !isControl(e.animation) && (inside(e.animation) || e.ancestors.includes(element)) && e.animation.timeline instanceof DocumentTimeline, ); @@ -1004,6 +1015,7 @@ export async function waitForAnimation( .map(e => e.animation) ).filter( a => + !isControl(a) && a.playState === 'running' && a.timeline instanceof DocumentTimeline, ); diff --git a/internal/interaction-harness/src/harness.interaction.browser.spec.ts b/internal/interaction-harness/src/harness.interaction.browser.spec.ts index d5ecffece8e7b..5bd7dd066a9b9 100644 --- a/internal/interaction-harness/src/harness.interaction.browser.spec.ts +++ b/internal/interaction-harness/src/harness.interaction.browser.spec.ts @@ -32,6 +32,7 @@ */ import {expect, test, type Page} from '@playwright/test'; +import type {CastFrame, TrackedFrame} from './frames'; import { animationStoryboard, blockMainThread, @@ -39,6 +40,7 @@ import { compareCurve, countBetween, cubicBezier, + CONTROL_RGB, curveOf, drag, flick, @@ -53,6 +55,7 @@ import { parseTransform, PARTIAL_LIFT_SKIP, pinch, + pooledBusyProof, readInputLog, readLongTasks, releaseAnimations, @@ -61,6 +64,7 @@ import { springStep, startCompositorTrace, startScreencast, + startBusyControl, Storyboard, tap, thawTimers, @@ -98,6 +102,73 @@ const BUSY_FIXTURE = ` { + // A dimming backdrop over a black page: its brightness at a point is the + // value judged, read from the same frames as the control. + await page.route('http://backdrop.test/', route => + route.fulfill({ + contentType: 'text/html', + body: ``, + }), + ); + await page.goto('http://backdrop.test/'); + const dialog = page.locator('#sheet'); + // The dim runs from the dialog's open, endlessly and wrapping rather than + // turning, so it is on screen and changing on every frame by the block. + await dialog.evaluate(el => { + (el as HTMLDialogElement).showModal(); + el.animate([{opacity: 0.2}, {opacity: 1}], { + duration: 900, + easing: 'linear', + iterations: Infinity, + pseudoElement: '::backdrop', + }); + }); + const control = await startBusyControl(page, {within: dialog}); + const cast = await startScreencast(page); + await page.waitForTimeout(200); + const window = await blockMainThread(page, 600, {during: dialog}); + await page.waitForTimeout(200); + const castFrames = await cast.stop(); + await control.remove(); + const controlFrames = await trackMarker(page, castFrames, { + rgb: CONTROL_RGB, + }); + const dim = await brightnessAt(page, castFrames, {x: 40, y: 600}); + const proof = busyWindowProof(dim, window, { + axis: 'x', + control: controlFrames, + }); + expect(proof.verdict.failures, proof.summary).toEqual([]); + }); + + test("waitForAnimation and blockMainThread's during never count the control's own animation inside a dialog", async ({ + page, + }) => { + await page.setContent( + 'still', + ); + await page.evaluate(() => + (document.getElementById('d') as HTMLDialogElement).showModal(), + ); + const dialog = page.locator('#d'); + const control = await startBusyControl(page, {within: dialog}); + // The dialog animates nothing of its own: both gates keep waiting. + await expect( + blockMainThread(page, 100, {during: dialog, timeout: 600}), + ).rejects.toThrow(/no animation started/); + await expect(waitForAnimation(dialog, {timeout: 600})).rejects.toThrow(); + // Its own animation answers both. + await dialog.evaluate(el => + el.animate([{opacity: 1}, {opacity: 0.5}], {duration: 2000}), + ); + await waitForAnimation(dialog); + await blockMainThread(page, 100, {during: dialog}); + await control.remove(); + }); + + test('a hold that starts before the block is judged from its first presented change inside it; a single change and then a freeze still fails', async ({ + page, + }) => { + await busyFixture(page); + const box = page.locator('#box'); + await markElement(box, {mode: 'outline'}); + const control = await startBusyControl(page); + // On the compositor: a move, a 300 ms hold (as WebKit holds a sheet's + // just-started release on screen), then a move; or one jump inside the + // block and nothing after it. + const run = async ( + keyframes: Keyframe[], + options: KeyframeAnimationOptions, + ) => { + const cast = await startScreencast(page); + await page.waitForTimeout(150); + await box.evaluate( + (el, [frames, timing]) => { + el.getAnimations().forEach(a => a.cancel()); + el.animate(frames, timing); + }, + [keyframes, options] as const, + ); + // The block starts inside the hold, which began before it. + const window = await blockMainThread(page, 600, { + during: box, + minProgressMs: 700, + }); + await page.waitForTimeout(150); + const frames = await cast.stop(); + return busyWindowProof(await trackMarker(page, frames), window, { + axis: 'y', + control: await trackMarker(page, frames, {rgb: CONTROL_RGB}), + }); + }; + const held = await run( + [ + {transform: 'translateY(0px)', offset: 0}, + {transform: 'translateY(100px)', offset: 0.3}, + {transform: 'translateY(100px)', offset: 0.45}, + {transform: 'translateY(560px)', offset: 1}, + ], + {duration: 2000, easing: 'linear'}, + ); + expect(held.verdict.failures, held.summary).toEqual([]); + const jumpThenFreeze = await run( + [{transform: 'translateY(0px)'}, {transform: 'translateY(50px)'}], + {duration: 800, easing: 'steps(1, end)', fill: 'forwards'}, + ); + expect(jumpThenFreeze.verdict.pass, jumpThenFreeze.summary).toBe(false); + await control.remove(); + }); + + test('a block too short to judge alone is judged over pooled runs: a compositor animation passes, and a rAF one fails, timed or frame-counted', async ({ + page, + }, testInfo) => { + await busyFixture(page); + const box = page.locator('#box'); + await markElement(box, {mode: 'outline'}); + const control = await startBusyControl(page); + + // Six 70 ms blocks: each judged part (20 ms margins, 30 ms) holds at most + // three frames even at WebKit's 14 ms cast, under the four one window + // needs, as a short fade's does. + const runs = async ( + driver: 'compositor' | 'raf' | 'counted 1' | 'counted 2', + ) => { + await box.evaluate((el, how) => { + el.getAnimations().forEach(a => a.cancel()); + el.style.transform = 'none'; + if (how === 'compositor') { + // Endless, wrapping from 560 px back to 0 rather than turning, so + // every frame changes: two frames straddling a turnaround evenly + // would read one position. + el.animate( + [{transform: 'translateY(0px)'}, {transform: 'translateY(560px)'}], + {duration: 1600, easing: 'linear', iterations: Infinity}, + ); + } else if (how === 'raf') { + const start = performance.now(); + // The same endless run on the main thread. + const step = (now: number) => { + const p = ((now - start) / 1600) % 1; + el.style.transform = `translateY(${560 * p}px)`; + requestAnimationFrame(step); + }; + requestAnimationFrame(step); + } else { + // A fixed step a frame, counted rather than timed: after a block it + // goes on from where it stood instead of catching up. + const px = how === 'counted 1' ? 1 : 2; + let y = 0; + let direction = 1; + const step = () => { + if (y + direction * px > 560 || y + direction * px < 0) { + direction = -direction; + } + y += direction * px; + el.style.transform = `translateY(${y}px)`; + requestAnimationFrame(step); + }; + requestAnimationFrame(step); + } + }, driver); + const cast = await startScreencast(page); + await page.waitForTimeout(200); + const windows = []; + for (let i = 0; i < 6; i += 1) { + windows.push(await blockMainThread(page, 70)); + await page.waitForTimeout(150); + } + const frames = await cast.stop(); + const tracked = await trackMarker(page, frames); + const controlFrames = await trackMarker(page, frames, {rgb: CONTROL_RGB}); + return windows.map(window => ({tracked, control: controlFrames, window})); + }; + + const compositor = await runs('compositor'); + const single = busyWindowProof( + compositor[0].tracked, + compositor[0].window, + {axis: 'y', control: compositor[0].control}, + ); + expect(single.verdict.failures.join('\n'), single.summary).toMatch( + /too little in this window to judge/, + ); + const pooled = pooledBusyProof(compositor, {axis: 'y'}); + testInfo.annotations.push({ + type: 'busy-pooled compositor', + description: pooled.summary, + }); + expect(pooled.verdict.failures, pooled.summary).toEqual([]); + + // The mutant: a main-thread animation freezes in every block. + const raf = pooledBusyProof(await runs('raf'), {axis: 'y'}); + testInfo.annotations.push({ + type: 'busy-pooled raf', + description: raf.summary, + }); + expect(raf.verdict.pass, raf.summary).toBe(false); + expect(raf.verdict.failures.join('\n')).toMatch( + /moved on \d+ of the \d+ frames the control moved over 6 runs/, + ); + + // A frame-counted loop stalls in every block too and then steps on, one + // cast step a frame at 1 px: its stall is longer than its runs before it. + for (const counted of ['counted 1', 'counted 2'] as const) { + const proof = pooledBusyProof(await runs(counted), {axis: 'y'}); + testInfo.annotations.push({ + type: `busy-pooled ${counted}`, + description: proof.summary, + }); + expect(proof.verdict.pass, `${counted}: ${proof.summary}`).toBe(false); + } + await control.remove(); + }); + test('a block never starts on an animation that is not on the compositor yet', async ({ page, }, testInfo) => { @@ -475,6 +900,9 @@ test.describe('frame evidence', () => { await box.evaluate(el => { el.style.willChange = 'transform'; }); + // Judged against what the compositor presented in the same window: a + // loaded runner's compositor skips frames a display-rate bound counts. + const control = await startBusyControl(page); const cast = await startScreencast(page); // A rAF loop registered before the helper's tick runs ahead of it in every // frame; on its 12th frame it creates the animation, and the helper's tick @@ -499,8 +927,13 @@ test.describe('frame evidence', () => { minProgressMs: 0, }); await waitForRest(box); - const tracked = await trackMarker(page, await cast.stop()); - const proof = busyWindowProof(tracked, window, {axis: 'y'}); + const frames = await cast.stop(); + await control.remove(); + const tracked = await trackMarker(page, frames); + const proof = busyWindowProof(tracked, window, { + axis: 'y', + control: await trackMarker(page, frames, {rgb: CONTROL_RGB}), + }); testInfo.annotations.push({ type: 'busy-main-thread pending animation', description: proof.summary,