integration/out/comparison.json scored 0% on both arms — not a verdict on HTLM, a broken instrument: ~30-40% of tasks structurally unwinnable against a flagged, automation-detected headless Chromium (Cloudflare/CAPTCHA/access-denied), plus a real crash (PlanShortlist.shortlist_indices unbounded, minimax emits huge index lists, JSON truncates, pydantic fails, 4/20 htlm tasks lost). Swap in the anti-detection camofox browser (Camoufox/Firefox fork, REST API on :9377, own snapshot/action format — needs a bridge, not a drop-in for browser-use's BrowserSession) so the live A/B (llm vs htlm grounder, from #7) measures grounding, not bot-detection.
Plan: .agents/plans/2026-07-16-live-proof-of-work.md (Track A)
Deliverables:
integration/camofox_adapter.py — snapshot_to_page (camofox accessibility refs → integer index, camofox roles → map_mind2web._map_role categories) + action_to_camofox (action → REST call).integration/run_bench_camofox.py (or --browser {chromium,camofox} flag on run_bench.py) — same loop/planner/judge, camofox session; keep the llm/htlm A/B intact.integration/planner.py:89 crash fix — bound shortlist_indices (max_length=64, matching the prompt's stated cap) and/or hard-truncate in Planner.shortlist. Land regardless of the rest of this track.- Bucket/skip judge-flagged
impossible_task/captcha rows so the reported number reflects winnable tasks only.
Success = a non-degenerate live success number (not both-arms-0%) that separates the htlm and llm arms, written to integration/out/ with a short results note. Background/secondary to #8 — offline 91.2% stays the primary claim.
integration/out/comparison.jsonscored 0% on both arms — not a verdict on HTLM, a broken instrument: ~30-40% of tasks structurally unwinnable against a flagged, automation-detected headless Chromium (Cloudflare/CAPTCHA/access-denied), plus a real crash (PlanShortlist.shortlist_indicesunbounded, minimax emits huge index lists, JSON truncates, pydantic fails, 4/20 htlm tasks lost). Swap in the anti-detection camofox browser (Camoufox/Firefox fork, REST API on :9377, own snapshot/action format — needs a bridge, not a drop-in for browser-use'sBrowserSession) so the live A/B (llm vs htlm grounder, from #7) measures grounding, not bot-detection.Plan:
.agents/plans/2026-07-16-live-proof-of-work.md(Track A)Deliverables:
integration/camofox_adapter.py—snapshot_to_page(camofox accessibility refs → integerindex, camofox roles →map_mind2web._map_rolecategories) +action_to_camofox(action → REST call).integration/run_bench_camofox.py(or--browser {chromium,camofox}flag onrun_bench.py) — same loop/planner/judge, camofox session; keep the llm/htlm A/B intact.integration/planner.py:89crash fix — boundshortlist_indices(max_length=64, matching the prompt's stated cap) and/or hard-truncate inPlanner.shortlist. Land regardless of the rest of this track.impossible_task/captcha rows so the reported number reflects winnable tasks only.Success = a non-degenerate live success number (not both-arms-0%) that separates the htlm and llm arms, written to
integration/out/with a short results note. Background/secondary to #8 — offline 91.2% stays the primary claim.