benchmarks: Harbor Terminal-Bench harness for Buzz agent-team orchestration (harbor-buzz-orchestra)#1504

Open

tlongwell-block wants to merge 31 commits into

mainfrom

wren/harbor-buzz-orchestra

Commits on Jul 3, 2026

feat(benchmarks): scaffold Harbor Buzz orchestra agent
npub12gtutshhh76rx0jx697f32f9tffd4hhp3hx58fp4x6u4uemkm7sqf8f757
committed
feat(benchmarks): carry NIP-OA agent attestations
npub12gtutshhh76rx0jx697f32f9tffd4hhp3hx58fp4x6u4uemkm7sqf8f757
committed
feat(benchmarks): add Buzz trial provisioner testbed

npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d
and
Tyler Longwell
committed
feat(benchmarks): add M1 personas, manifest, and benchmark schema

npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d
and
Tyler Longwell
committed
feat(benchmarks): add serialized Harbor terminal broker
npub12gtutshhh76rx0jx697f32f9tffd4hhp3hx58fp4x6u4uemkm7sqf8f757
committed
feat(benchmarks): wire Buzz ACP subprocess runtime
npub12gtutshhh76rx0jx697f32f9tffd4hhp3hx58fp4x6u4uemkm7sqf8f757
committed
feat(benchmarks): add M1 local endpoint launch config

npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d
and
Tyler Longwell
committed
benchmark: allow pinned Buzz binary paths
npub12gtutshhh76rx0jx697f32f9tffd4hhp3hx58fp4x6u4uemkm7sqf8f757
committed
fix(benchmarks): isolate agent hints and bound rounds
npub12gtutshhh76rx0jx697f32f9tffd4hhp3hx58fp4x6u4uemkm7sqf8f757
committed
feat(benchmarks): M1 personas publish via buzz_exec CLI surface

npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d
and
Tyler Longwell
committed
Add production Buzz CLI agent surface
npub12gtutshhh76rx0jx697f32f9tffd4hhp3hx58fp4x6u4uemkm7sqf8f757
committed
fix(benchmarks): update M1 manifest persona SHA-256 pins

npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d
and
Tyler Longwell
committed
fix(benchmarks): worker persona must @mention its assigner in reports

npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d
and
Tyler Longwell
committed
bench: prepare offline verifier dependencies
npub12gtutshhh76rx0jx697f32f9tffd4hhp3hx58fp4x6u4uemkm7sqf8f757
committed
bench: add laptop advisory verifier mode
npub12gtutshhh76rx0jx697f32f9tffd4hhp3hx58fp4x6u4uemkm7sqf8f757
committed
fix(benchmarks): personas must not invent absolute paths outside the task workdir

npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d
and
Tyler Longwell
committed
Gate Buzz trials on agent readiness
npub12gtutshhh76rx0jx697f32f9tffd4hhp3hx58fp4x6u4uemkm7sqf8f757
committed
fix(benchmarks): orchestrator must not invent constraints the task omits

npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d
and
Tyler Longwell
committed
bench: align M1 probe and recover silent worker reports
npub12gtutshhh76rx0jx697f32f9tffd4hhp3hx58fp4x6u4uemkm7sqf8f757
committed
docs(benchmarks): document and validate Harbor harness
npub12gtutshhh76rx0jx697f32f9tffd4hhp3hx58fp4x6u4uemkm7sqf8f757
committed
ci(benchmarks): use hosted uv cache offline
npub12gtutshhh76rx0jx697f32f9tffd4hhp3hx58fp4x6u4uemkm7sqf8f757
committed
ci(benchmarks): install Harbor dependencies with pip
npub12gtutshhh76rx0jx697f32f9tffd4hhp3hx58fp4x6u4uemkm7sqf8f757
committed
bench: generalize verifier preparation to real TB tasks

npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d
and
Tyler Longwell
committed
bench: add Sonnet+Haiku Terminal-Bench team condition

npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d
and
Tyler Longwell
committed
bench: one-command leaderboard runner

npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d
and
Tyler Longwell
committed
bench: production-faithful trial seeding — user identity, composed system prompts

npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d
and
Tyler Longwell
committed

Commits on Jul 4, 2026

bench: just benchmark — one-command stack bring-up, leaderboard defaults, live GUI

npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d
and
Tyler Longwell
committed
bench: run the production Buzz agent stack inside the task container

npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d
and
Tyler Longwell
committed
bench: diagnose cross-clone Postgres credential collisions at stack bring-up

npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d
and
Tyler Longwell
committed
bench: print the benchmark user's nsec at startup for GUI onboarding

npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d
and
Tyler Longwell
committed
bench: isolate the --gui app under its own bundle identifier

npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d
and
Tyler Longwell
committed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

benchmarks: Harbor Terminal-Bench harness for Buzz agent-team orchestration (harbor-buzz-orchestra)#1504

benchmarks: Harbor Terminal-Bench harness for Buzz agent-team orchestration (harbor-buzz-orchestra)#1504
tlongwell-block wants to merge 31 commits into
mainfrom
wren/harbor-buzz-orchestra

Commits on Jul 3, 2026

feat(benchmarks): scaffold Harbor Buzz orchestra agent

feat(benchmarks): carry NIP-OA agent attestations

feat(benchmarks): add Buzz trial provisioner testbed

feat(benchmarks): add M1 personas, manifest, and benchmark schema

feat(benchmarks): add serialized Harbor terminal broker

feat(benchmarks): wire Buzz ACP subprocess runtime

feat(benchmarks): add M1 local endpoint launch config

benchmark: allow pinned Buzz binary paths

fix(benchmarks): isolate agent hints and bound rounds

feat(benchmarks): M1 personas publish via buzz_exec CLI surface

Add production Buzz CLI agent surface

fix(benchmarks): update M1 manifest persona SHA-256 pins

fix(benchmarks): worker persona must @mention its assigner in reports

bench: prepare offline verifier dependencies

bench: add laptop advisory verifier mode

fix(benchmarks): personas must not invent absolute paths outside the task workdir

Gate Buzz trials on agent readiness

fix(benchmarks): orchestrator must not invent constraints the task omits

bench: align M1 probe and recover silent worker reports

docs(benchmarks): document and validate Harbor harness

ci(benchmarks): use hosted uv cache offline

ci(benchmarks): install Harbor dependencies with pip

bench: generalize verifier preparation to real TB tasks

bench: add Sonnet+Haiku Terminal-Bench team condition

bench: one-command leaderboard runner

bench: production-faithful trial seeding — user identity, composed system prompts

Commits on Jul 4, 2026

bench: just benchmark — one-command stack bring-up, leaderboard defaults, live GUI

bench: run the production Buzz agent stack inside the task container

bench: diagnose cross-clone Postgres credential collisions at stack bring-up

bench: print the benchmark user's nsec at startup for GUI onboarding

bench: isolate the --gui app under its own bundle identifier