Build San José Legistar decision pipeline

[ThatXliner]

Closes #282

Summary

• replace the unused Legistar cache schema with a source-neutral local-government model for jurisdictions, bodies, canonical decisions, meeting occurrences, documents, histories, votes, and ingestion runs
• register a paged, retry-bounded San José scraper with an explicit resident-relevance body policy, conservative topic/geography classification, native PDF extraction, hashes, idempotent upserts, and source-removal tracking
• add durable decision-centric tRPC reads for lists, detail/timeline, included bodies, ingestion health, district relevance, search, official documents, and public-comment links/counts
• make the source transport stateless and remove database cache/mock behavior from it
• document the implemented architecture and decisions

Privacy decision

Public-comment letters remain relevant as official records, but they may expose names, signatures, home/contact details, or sensitive personal circumstances. In v1 they are link-only: the scraper does not download, OCR, index, or summarize their contents, and the API strips attachment names while returning generic official links and an aggregate count.

Migration note

Migration 0012_tough_stone_men.sql intentionally drops the five legacy legistar_* cache tables. They have no ingestion or application consumer today, and issue scope explicitly allows replacing them. The migration was generated with Drizzle, replayed from scratch against PostgreSQL 17, checked with drizzle-kit check, and enables RLS on all nine replacement tables.

Validation

• pnpm typecheck
• pnpm lint
• pnpm format
• pnpm db:check
• pnpm --filter @acme/api test (22 passing)
• pnpm --filter @acme/scraper test (135 passing)
• pnpm --filter @acme/scraper build
• fresh migration replay against disposable PostgreSQL 17
• bounded live San José runs, including native extraction of 379,761 characters across 19 official documents and a final two-meeting ingestion smoke test
• verified all new tables have RLS enabled and public-comment rows contain no extracted text

Deliberately quality-gated follow-ups

The PR does not publish uncited AI summaries. Approved-minutes fallback with page citations, the OCR worker and measured validation set, agenda-specific participation extraction, official GIS address resolution, cited AI explanations, and extraction of San José policy into a jurisdiction profile before enabling a second city remain explicit follow-ups rather than silent or inferred production behavior.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
billion-nextjs	Ready	Preview	Aug 20, 2026 3:58am