Add pg-sync observation support for agents

[KyleAMathews]

Summary

Adds pg-sync observation — a new way for agents to observe Postgres table changes via Electric shape streams and wake on matching row operations (insert/update/delete). This replaces the need for agents to poll or use webhooks for database change detection.

Approach

Architecture

The feature spans three packages in a layered design:

1. agents-runtime — Defines the pgSync() observation source factory. Produces a PgSyncObservationSource with a deterministic sourceRef (content-addressed hash of canonical options) and registers it with the server via registerPgSyncSource().

2. agents-server — Houses PgSyncBridgeManager, which manages the lifecycle of "bridges" — each bridge subscribes to an Electric ShapeStream for a given table/shape and forwards change messages into a Durable Stream. Bridges persist their cursor (shapeHandle + shapeOffset) across restarts so they resume from where they left off. The PgSyncRouter exposes a POST /_electric/pg-sync/register endpoint for runtime registration. Wake evaluation filters changes by operation type and collection.

3. agents — Exposes the observe_pg_sync tool to Horton agents, letting the LLM dynamically subscribe to table changes with optional operation filtering and debounce.

Key design decisions

• Canonical options & sourceRef: Shape options are canonicalized (sorted params, defaulted replica, copied arrays) before hashing. This ensures two registrations for the same logical shape always produce the same sourceRef, preventing duplicate bridges. The canonicalization is exported from agents-runtime and reused by agents-server — single source of truth.

• Initial snapshot skipping: When a bridge starts fresh, it sets skipChangesUntilUpToDate = true to avoid treating the initial Electric snapshot as live changes. Changes before the first up-to-date control message are silently dropped. Resumed bridges (with a stored cursor) process all changes immediately.

• Cursor as atomic unit: shapeHandle and shapeOffset are modeled as a single PgSyncCursor object ({ handle, offset }), making the "both or neither" invariant unrepresentable as an illegal state.

• Subscription error recovery: If the ShapeStream subscription errors or the async message callback throws, the bridge attempts recovery — resuming from its last known cursor if available, or restarting from scratch in bootstrap-skip mode.

Key invariants

• One bridge per sourceRef per tenant (enforced via bridges map + starting deduplication map)
• sourceRef is deterministic: same canonical options → same hash, regardless of field order
• Bridges survive server restarts via cursor persistence in pg_sync_bridges table
• Wake evaluation respects ops filters — an insert-only subscriber won't wake on delete
• BigInt values from Electric are converted to strings before JSON serialization

Non-goals

• No automatic bridge cleanup/garbage collection (future work)
• No retry with exponential backoff on bridge failures (recovery is immediate restart)
• No UI for pg-sync observation management

Verification

# Bridge manager (lifecycle, cursor persistence, recovery, BigInt safety)
pnpm --dir packages/agents-server exec vitest run test/pg-sync-bridge-manager.test.ts --reporter=verbose

# Wake delivery filtering by operation and collection
pnpm --dir packages/agents-server exec vitest run test/pg-sync-wake-delivery.test.ts --reporter=verbose

# Router registration, validation, 503 handling
pnpm --dir packages/agents-server exec vitest run test/pg-sync-router.test.ts --reporter=verbose

# Server routing integration
pnpm --dir packages/agents-server exec vitest run test/oss-server-router.test.ts -t 'pg-sync stream reads' --reporter=verbose

# Runtime sourceRef determinism (key-order independence, column-order, defaults)
pnpm --dir packages/agents-runtime exec vitest run test/pg-sync-source.test.ts --reporter=verbose

# Runtime server client round-trip
pnpm --dir packages/agents-runtime exec vitest run test/runtime-server-client-pg-sync.test.ts --reporter=verbose

# Process wake integration
pnpm --dir packages/agents-runtime exec vitest run test/process-wake.test.ts --reporter=verbose

# observe_pg_sync tool (including debounceMs: 0 regression)
pnpm --dir packages/agents exec vitest run test/observe-pg-sync-tool.test.ts --reporter=verbose

# Shutdown lifecycle
pnpm --dir packages/agents-server exec vitest run test/runtime-shutdown.test.ts --reporter=verbose
pnpm --dir packages/agents-server exec vitest run test/standalone-runtime.test.ts --reporter=verbose

# Type checking
pnpm --dir packages/agents-server typecheck
pnpm --dir packages/agents-runtime typecheck
pnpm --dir packages/agents typecheck

Files changed

agents-runtime

• observation-sources.ts — New pgSync() factory, PgSyncOptions/CanonicalPgSyncConfig types, sourceRefForPgSync(), canonical options export
• runtime-server-client.ts — registerPgSyncSource() HTTP client method
• process-wake.ts — Wire pg-sync source registration into wake processing
• entity-schema.ts — Add value/oldValue to WakeChangeEntryValue
• index.ts — Re-export pg-sync types and helpers

agents-server

• pg-sync-bridge-manager.ts — PgSyncBridge (stream subscription + durable forwarding), PgSyncBridgeManager (lifecycle, deduplication, recovery)
• pg-sync-router.ts — HTTP registration endpoint with schema validation
• entity-registry.ts — pg_sync_bridges CRUD (upsert, cursor update/clear, touch, list)
• db/schema.ts — pgSyncBridges table definition + migration
• wake-registry.ts — pg-sync wake evaluation with operation filtering
• runtime.ts / standalone-runtime.ts — Bridge manager integration into server lifecycle
• routing/ — Mount pg-sync router on global router

agents

• tools/observe-pg-sync.ts — observe_pg_sync agent tool with TypeBox schema
• agents/horton.ts — Register tool in Horton's tool list

Other

• scripts/dev.sh — Improved dev stack isolation and port configurability
• docs/agents-pg-sync-observation-plan.md — Design document

[netlify]

✅ Deploy Preview for electric-next ready!

Name	Link
🔨 Latest commit	4e573c7
🔍 Latest deploy log	https://app.netlify.com/projects/electric-next/deploys/6a16408e3ee7220008558241
😎 Deploy Preview	https://deploy-preview-4421--electric-next.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify project configuration.

[github-actions]

Electric Agents Desktop Builds

Build artifacts for commit 1b0f719.

Platform	Status	Artifact
macOS Apple Silicon	Passed	DMG
macOS Intel	Passed	DMG
Windows x64	Passed	Installer
Linux x64	Passed	AppImage / deb

Workflow run

[codecov]

Codecov Report

❌ Patch coverage is 86.32911% with 54 lines in your changes missing coverage. Please review.
✅ Project coverage is 60.91%. Comparing base (2d1615d) to head (1b0f719).
✅ All tests successful. No failed tests found.

Files with missing lines	Patch %	Lines
...ckages/agents-server/src/pg-sync-bridge-manager.ts	86.34%	27 Missing and 1 partial ⚠️
packages/agents-server/src/entity-registry.ts	18.51%	22 Missing ⚠️
...ckages/agents-server/src/routing/pg-sync-router.ts	80.00%	3 Missing ⚠️
packages/agents-server/src/runtime.ts	80.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #4421      +/-   ##
==========================================
+ Coverage   60.59%   60.91%   +0.31%     
==========================================
  Files         306      309       +3     
  Lines       31677    32062     +385     
  Branches     8611     8733     +122     
==========================================
+ Hits        19195    19530     +335     
- Misses      12464    12513      +49     
- Partials       18       19       +1     

Flag	Coverage Δ
packages/agents	69.04% <100.00%> (+0.56%)	⬆️
packages/agents-mcp	77.54% <ø> (ø)
packages/agents-runtime	81.95% <100.00%> (+0.14%)	⬆️
packages/agents-server	74.94% <80.43%> (+0.27%)	⬆️
packages/agents-server-ui	6.19% <ø> (ø)
packages/electric-ax	43.81% <ø> (ø)
packages/experimental	87.73% <ø> (ø)
packages/react-hooks	86.48% <ø> (ø)
packages/start	82.83% <ø> (ø)
packages/typescript-client	94.39% <ø> (ø)
packages/y-electric	56.05% <ø> (ø)
typescript	60.91% <86.32%> (+0.31%)	⬆️
unit-tests	60.91% <86.32%> (+0.31%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.