Skip to content

Commit cc7fa26

Browse files
Merge branch 'main' into dependabot/cargo/dirs-6.0.0
2 parents 73f7958 + 54e3f18 commit cc7fa26

8 files changed

Lines changed: 212 additions & 92 deletions

File tree

‎hyperdb-mcp/CHANGELOG.md‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -131,6 +131,15 @@ and this project adheres to [Semantic Versioning](https://semver.org/).
131131
immediately after `CREATE TABLE IF NOT EXISTS`. `detach_database`
132132
clears the alias's entry so a re-attach to a different file isn't
133133
served stale.
134+
- **Cross-process catalog write safety via optimistic concurrency.**
135+
`upsert_stub_in` now uses UPDATE-then-conditional-INSERT instead
136+
of DELETE+INSERT in a transaction. Each statement is individually
137+
atomic at the `hyperd` level, so multiple MCP server processes
138+
sharing the same persistent database via the daemon can no longer
139+
produce duplicate `_table_catalog` rows from concurrent ingests.
140+
The UPDATE path also eliminates one round-trip (the pre-read of the
141+
existing row to preserve prose fields — UPDATE preserves them
142+
implicitly by only touching mechanical columns).
134143

135144
### Fixed
136145

‎hyperdb-mcp/README.md‎

Lines changed: 17 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -10,17 +10,31 @@ Built on the pure-Rust [`hyperdb-api`](../hyperdb-api/) crate for maximum perfor
1010

1111
LLMs are powerful at reasoning but cannot natively crunch millions of rows. This plugin bridges that gap: another MCP tool produces data, the LLM passes it to `hyperdb-mcp`, Hyper ingests it and makes it SQL-queryable, the LLM runs analytical SQL, and results come back as JSON. Optionally export to CSV, Parquet, Apache Iceberg, Arrow IPC, or `.hyper` (opens directly in **Tableau Desktop**).
1212

13+
### Queryable Memory for AI
14+
15+
Unlike flat-text memory systems that store blobs and retrieve by similarity search, HyperDB gives LLMs **structured, queryable long-term memory**. The persistent database survives across sessions — anything the LLM stores there can be JOINed, filtered, aggregated, and reasoned over with full SQL in any future conversation.
16+
17+
This means an LLM can:
18+
- **Accumulate knowledge over time** — store reference tables, project decisions, user preferences, learned facts
19+
- **Cross-reference across sessions** — JOIN today's analysis against historical data from last week
20+
- **Answer complex recall questions** — "Which projects had budget overruns in Q1?" is a SQL query, not a fuzzy text search
21+
- **Build on prior work** — load yesterday's cleaned dataset and extend it without re-processing from scratch
22+
- **Maintain structured context** — store relationship graphs, timelines, or decision logs as proper tables with typed columns
23+
24+
The ephemeral database is scratch space (think: a whiteboard). The persistent database is long-term memory (think: a filing cabinet you can query). Multiple AI clients sharing the same daemon see the same persistent data — so Claude Code, Cursor, and VS Code Copilot can all read from and contribute to the same knowledge base.
25+
1326
---
1427

1528
## Features
1629

1730
- **Zero setup** — `HyperProcess` auto-starts the Hyper server
1831
- **Shared `hyperd` daemon** — one Hyper process per user, shared across all MCP clients (Claude Code, Cursor, VS Code, etc.) for reduced memory overhead and concurrent access to the same persistent databases
32+
- **Queryable long-term memory** — persistent database survives across sessions; LLMs can store, recall, JOIN, and aggregate structured knowledge over time — not just retrieve text blobs, but reason over them with SQL
1933
- **Any data in** — JSON, CSV, Parquet, Arrow IPC, Apache Iceberg; schema inferred or exact
2034
- **SQL at scale** — thousands to billions of rows
2135
- **Data out** — export to CSV, Parquet, Apache Iceberg, Arrow IPC, or `.hyper` (Tableau Desktop-ready)
2236
- **One-shot queries** — `query_file("/tmp/sales.csv", "SELECT ...")` — single call, zero management
23-
- **Persistent workspace** — load multiple tables, JOIN across them, persist across sessions
37+
- **Cross-session continuity** — load multiple tables, JOIN across them, persist across sessions; pick up exactly where you left off
2438
- **Read-only safe mode** — `--read-only` flag for safe deployment
2539
- **Schema resources** — auto-discover table schemas via `resources/list`
2640
- **Guided prompts** — `analyze-table`, `compare-tables`, `data-quality`, `suggest-queries`
@@ -163,7 +177,7 @@ Any tool that supports the MCP stdio transport can use this server. Point it at
163177

164178
## Operating Modes
165179

166-
Each session has **two databases**: an ephemeral primary (always created fresh per session, deleted on exit) and a persistent attachment (stored at the platform-default location or a path you supply). Unqualified SQL targets the ephemeral primary; the persistent attachment is reachable as the `"persistent"` alias and survives across sessions.
180+
Each session has **two databases**: an ephemeral primary (scratch space — always created fresh per session, deleted on exit) and a persistent database (queryable long-term memory — stored at the platform-default location or a path you supply, survives indefinitely). Unqualified SQL targets the ephemeral primary; the persistent database is reachable as the `"persistent"` alias.
167181

168182
### Hyper engine
169183

@@ -186,7 +200,7 @@ The shared daemon is the bigger win for users running multiple AI clients (Claud
186200

187201
### Working with both databases
188202

189-
Tool calls default to the ephemeral primary — that's the LLM's scratch space for exploratory work. There are two ways to reach the persistent attachment:
203+
Tool calls default to the ephemeral primary — that's the LLM's scratch space for exploratory work that doesn't need to outlive the session. To store data in long-term memory (the persistent database), there are two ways to reach it:
190204

191205
**1. Per-tool `database` parameter** (preferred for ergonomic LLM workflows):
192206

‎hyperdb-mcp/src/readme.rs‎

Lines changed: 38 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -27,23 +27,52 @@ or shell pipelines: it parses files faster, runs SQL natively, and keeps
2727
intermediate state in a workspace database the LLM can re-query without
2828
re-loading.
2929
30-
## Workspace model
30+
## Workspace model — queryable memory
3131
3232
Every session has TWO databases, plus optional user-attached ones:
3333
3434
- **Ephemeral primary** (default destination). Created fresh per
35-
session, deleted on exit. Unqualified SQL routes here. Use for
36-
scratch work and intermediate transformations the user doesn't
37-
need to keep.
38-
- **Persistent attachment** under alias `\"persistent\"`. Survives
39-
across sessions at the platform-default path (or override via
40-
`--persistent-db`). Use when the user wants the data to stick
41-
around. Disabled when the server runs with `--ephemeral-only`.
35+
session, deleted on exit. Unqualified SQL routes here. Use as
36+
scratch space for exploratory work, intermediate transformations,
37+
and one-off analysis the user doesn't need to keep.
38+
- **Persistent database** (alias `\"persistent\"`). Survives across
39+
sessions — this is your **long-term structured memory**. Store
40+
reference tables, accumulated results, user preferences, learned
41+
facts, or any data you want to recall in future conversations.
42+
Unlike flat-text memory, persistent data is **queryable**: you can
43+
JOIN, filter, aggregate, and reason over it with SQL. Disabled
44+
when the server runs with `--ephemeral-only`.
4245
- **User-attached writable databases** via `attach_database` with
4346
`writable: true`. Each lives in its own `.hyper` file under a
4447
user-chosen alias.
4548
46-
Pick a destination in one of two ways:
49+
### Persistent as memory — when and how to use it
50+
51+
Store data in persistent whenever:
52+
- The user says \"remember this\", \"save this\", \"keep this\"
53+
- You produce a useful reference table (lookups, configs, mappings)
54+
- You accumulate results across multiple conversations
55+
- You want to recall context in future sessions
56+
57+
Retrieve from persistent whenever:
58+
- You need context from a prior session
59+
- The user asks \"what do we have?\" or \"show me what's saved\"
60+
- You want to JOIN current scratch work against historical data
61+
62+
```
63+
// Save something for later
64+
load_data({ table: \"project_decisions\", data: \"[...]\", persist: true })
65+
66+
// Recall it next session
67+
query({ sql: \"SELECT * FROM project_decisions\", database: \"persistent\" })
68+
69+
// Cross-reference: join session scratch with persistent memory
70+
query({ sql: \"SELECT s.*, p.decision FROM scratch_analysis s \
71+
JOIN \\\"persistent\\\".\\\"public\\\".\\\"project_decisions\\\" p \
72+
ON s.topic = p.topic\" })
73+
```
74+
75+
### Routing data to a destination
4776
4877
- **`database` parameter** (preferred for tools that build their own
4978
SQL): `query`, `execute`, `load_data`, `load_file`, `load_files`,

‎hyperdb-mcp/src/server.rs‎

Lines changed: 35 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -34,10 +34,10 @@ use rmcp::handler::server::router::tool::ToolRouter;
3434
use rmcp::handler::server::wrapper::Parameters;
3535
use rmcp::model::{
3636
AnnotateAble, CallToolResult, Content, GetPromptRequestParams, GetPromptResult, Implementation,
37-
ListPromptsResult, ListResourceTemplatesResult, ListResourcesResult, PaginatedRequestParams,
38-
PromptMessage, PromptMessageRole, RawResource, RawResourceTemplate, ReadResourceRequestParams,
39-
ReadResourceResult, ResourceContents, ServerCapabilities, ServerInfo, SubscribeRequestParams,
40-
UnsubscribeRequestParams,
37+
InitializeRequestParams, InitializeResult, ListPromptsResult, ListResourceTemplatesResult,
38+
ListResourcesResult, PaginatedRequestParams, PromptMessage, PromptMessageRole, RawResource,
39+
RawResourceTemplate, ReadResourceRequestParams, ReadResourceResult, ResourceContents,
40+
ServerCapabilities, ServerInfo, SubscribeRequestParams, UnsubscribeRequestParams,
4141
};
4242
use rmcp::service::RequestContext;
4343
use rmcp::{
@@ -827,6 +827,10 @@ pub struct HyperMcpServer {
827827
no_daemon: bool,
828828
/// Last time a heartbeat was sent to the daemon (debounced to avoid per-call TCP overhead).
829829
last_heartbeat: std::sync::Mutex<std::time::Instant>,
830+
/// MCP client name from the `initialize` handshake (e.g. "Claude Code",
831+
/// "cursor-mcp-client"). Populated once per session; used for catalog
832+
/// provenance tracking (`created_by` / `last_modified_by`).
833+
client_name: std::sync::Mutex<Option<String>>,
830834
// Under rmcp 1.x the router fields are constructed for downstream
831835
// macro-generated dispatch but not read through a direct field access
832836
// that the compiler can see. Keep them; the `#[tool_router]` /
@@ -899,6 +903,7 @@ impl HyperMcpServer {
899903
read_only,
900904
no_daemon,
901905
last_heartbeat: std::sync::Mutex::new(std::time::Instant::now()),
906+
client_name: std::sync::Mutex::new(None),
902907
tool_router: Self::tool_router(),
903908
prompt_router: Self::prompt_router(),
904909
}
@@ -942,6 +947,12 @@ impl HyperMcpServer {
942947
self.subscriptions.notify_list_changed();
943948
}
944949

950+
/// The MCP client name from the `initialize` handshake, or `None` if
951+
/// the handshake hasn't completed yet. Used for catalog provenance.
952+
fn client_name(&self) -> Option<String> {
953+
self.client_name.lock().ok().and_then(|g| g.clone())
954+
}
955+
945956
/// Return a clone of the engine Arc so background tasks (watchers) can
946957
/// share access to the same lazy-initialized engine instance.
947958
#[must_use]
@@ -1129,10 +1140,6 @@ impl HyperMcpServer {
11291140
/// their own per-DB catalog. Read-only attachments are rejected
11301141
/// upstream by `resolve_db(require_writable=true)` so this helper
11311142
/// never sees them.
1132-
#[expect(
1133-
clippy::unused_self,
1134-
reason = "&self required for method-call dispatch; body uses only engine + params"
1135-
)]
11361143
fn after_ingest_catalog_update(
11371144
&self,
11381145
engine: &Engine,
@@ -1150,6 +1157,7 @@ impl HyperMcpServer {
11501157
row_count,
11511158
true,
11521159
target_db,
1160+
self.client_name().as_deref(),
11531161
) {
11541162
tracing::warn!(
11551163
table = %table_name,
@@ -3839,6 +3847,25 @@ Full SQL reference: https://developer.salesforce.com/docs/data/data-cloud-query-
38393847
info
38403848
}
38413849

3850+
async fn initialize(
3851+
&self,
3852+
request: InitializeRequestParams,
3853+
context: RequestContext<RoleServer>,
3854+
) -> Result<InitializeResult, rmcp::ErrorData> {
3855+
let name = &request.client_info.name;
3856+
let version = &request.client_info.version;
3857+
let label = if version.is_empty() {
3858+
name.clone()
3859+
} else {
3860+
format!("{name} {version}")
3861+
};
3862+
if let Ok(mut guard) = self.client_name.lock() {
3863+
*guard = Some(label);
3864+
}
3865+
context.peer.set_peer_info(request);
3866+
Ok(self.get_info())
3867+
}
3868+
38423869
/// Handle a `resources/subscribe` request by recording the calling
38433870
/// peer in the registry under the requested URI.
38443871
///

0 commit comments

Comments
 (0)