Skip to content

feat(db): add PostgreSQL + pgvector storage adapter for semantic search - #182

Merged
kavix merged 1 commit into
kavix:mainfrom
yoshiiita:feat/pgvector-storage-adapter
Aug 26, 2026
Merged

feat(db): add PostgreSQL + pgvector storage adapter for semantic search#182
kavix merged 1 commit into
kavix:mainfrom
yoshiiita:feat/pgvector-storage-adapter

Conversation

@yoshiiita

Copy link
Copy Markdown
Contributor

Summary

Closes #145

Implements internal/db/pgvector.go — a production-grade PostgreSQL 16 + pgvector storage adapter for indexing code ASTs, diffs, and repository memory embeddings with sub-millisecond semantic search.

What's in this PR

  • PGVectorStore — opens a Postgres connection and auto-migrates the schema on startup
  • embeddings table — stores content with a vector(1536) column backed by an HNSW index (vector_cosine_ops) for approximate nearest-neighbour queries at sub-millisecond latency
  • Upsert — idempotent insert-or-update of an embedding (keyed by content-derived ID)
  • Search — top-k cosine similarity query, optionally scoped to a content kind ("ast", "diff", "memory")
  • DeleteBySnapshot — bulk removal for GC / snapshot pruning
  • New deps: github.com/pgvector/pgvector-go, github.com/lib/pq

Schema

CREATE TABLE embeddings (
    id          TEXT PRIMARY KEY,
    snapshot_id TEXT NOT NULL,
    kind        TEXT NOT NULL,   -- "ast" | "diff" | "memory"
    content     TEXT NOT NULL,
    embedding   vector(1536)
);

CREATE INDEX embeddings_hnsw_idx
    ON embeddings USING hnsw (embedding vector_cosine_ops);

Test plan

  • Start a local Postgres 16 instance with pgvector enabled
  • Call NewPGVectorStore(dsn) — verify extension + table + index are created
  • Upsert several embeddings with different kind values
  • Search with kind="" — confirm all kinds are returned in distance order
  • Search with kind="ast" — confirm only AST rows are returned
  • DeleteBySnapshot — confirm rows are removed
  • go build ./... passes

@vercel

vercel Bot commented Aug 24, 2026

Copy link
Copy Markdown

@yoshita-madamala is attempting to deploy a commit to the Kavindu's projects Team on Vercel.

A member of the Team first needs to authorize it.

Implements internal/db/pgvector.go — a production-grade storage adapter
that indexes code ASTs, diffs, and repository memory embeddings in
PostgreSQL 16 with the pgvector extension.

Key additions:
- PGVectorStore: opens a Postgres connection and auto-migrates schema
- embeddings table with vector(1536) column backed by an HNSW index
  (vector_cosine_ops) for sub-millisecond approximate nearest-neighbour
  queries
- Upsert: idempotent insert-or-update of an embedding row
- Search: top-k cosine similarity search, optionally scoped to a
  content kind ("ast", "diff", "memory")
- DeleteBySnapshot: bulk removal for GC / snapshot pruning
- Close: releases the connection pool

Adds dependencies: github.com/pgvector/pgvector-go, github.com/lib/pq

Closes kavix#145
@yoshiiita
yoshiiita force-pushed the feat/pgvector-storage-adapter branch from b8cea8d to 84b3271 Compare August 25, 2026 06:45
@kavix
kavix self-requested a review August 26, 2026 12:27
@kavix kavix self-assigned this Aug 26, 2026
@kavix
kavix merged commit 45c61bf into kavix:main Aug 26, 2026
2 of 3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

arch: Implement PostgreSQL + pgvector Storage Adapter for Sub-Millisecond Semantic Search

2 participants