Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 45 additions & 0 deletions .changeset/mcp-server.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
---
"ohlc-resample": minor
---

Add a **zero-dependency MCP server** (`ohlc-resample-mcp` bin) and fix a
streaming CSV→CSV output bug.

**MCP server (bundled, no second package)**

- New `ohlc-resample-mcp` binary ships inside this same package — install
`ohlc-resample` and you get the MCP server for free. No separate publish.
- It is a **thin adapter over the package's own CLI** (`runCli` in-process
with injected streams): it has no resampling/parsing/formatting logic of its
own, so every new CLI feature is inherited with zero maintenance. A tool
call just builds an argv array and runs the CLI, returning stdout or a file
path.
- **No `@modelcontextprotocol/sdk` dependency** — MCP is plain JSON-RPC 2.0
over stdio, so the server hand-rolls the tiny protocol (initialize,
`tools/list`, `tools/call`, `ping`) directly. Runtime deps stay just `mri`
and `hyparquet`.
- Two tools: `resample_ohlcv_file` (takes an `input_path`
(csv/json/jsonl/ndjson/parquet), `base_timeframe`, `new_timeframe`,
`format`, `shape`, optional `map` (e.g. CCXT `timestamp`/`amount`), and
optional `output_path`) and `audit_ohlcv_file` (takes an `input_path` and
optional `map`, returns a trust report). Both are file-path-based, so the
LLM pays tokens for intent, not payload.

**New `audit` capability**

- New library function `auditOhlcv` + `AuditReport`: validates and describes
market-data input in one streaming pass, answering "can I trust the output
of this resampling, and exactly why?". Reports record count, time range,
source timeframe (modal positive interval), ordering (sorted, out-of-order
count, max lateness), duplicate timestamps, OHLC integrity violations, bad
values (NaN/Infinity/negative prices/volume), and missing bars.
- Wired to the CLI as `--audit` (prints a JSON report instead of resampling)
and exposed to MCP as `audit_ohlcv_file`, which inherits the CLI path with
zero extra logic.

**Bug fix**

- Streaming CSV → CSV output previously failed with `shaped.join is not a
function`: `IncrementalWriter` rendered object-shaped candles as CSV rows
without converting to the 6-tuple shape. CSV rows are now always written in
array shape regardless of the requested JSON shape.
58 changes: 57 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
<h1 align="center">ohlc-resample 🕯️</h1>
<p align="center">
Turn trade, tick, or OHLCV data into clean candlestick charts on any time frame
Transform, resample, and stream market data at any scale
</p>
<p align="center">
<a href="https://www.npmjs.com/package/ohlc-resample" target="_blank">
Expand Down Expand Up @@ -298,6 +298,7 @@ Options:
-b, --base-timeframe <number> Base timeframe in seconds (default: "60")
-n, --new-timeframe <number> New timeframe in seconds (default: "300")
--map <mapping> Feed data as-is; map fields to canonical keys (e.g. time=timestamp,volume=amount)
--audit Audit the input instead of resampling (print a JSON trust report)
-h, --help Display help for command
```

Expand Down Expand Up @@ -359,6 +360,61 @@ cat data.json | ohlc
cat data.csv | ohlc --input-format csv
```

## Transform and stream market data, straight from your AI

Tell your AI assistant to reshape your market data and it just does it. Point
it at your feed, name the time frame you want, and it hands back clean output
at any scale. It already speaks the street: CCXT-style feeds, tick data,
Parquet exports, gaps in the series, files too big for memory. You ask, it
delivers.

Set it up in one shot, then it works with whichever assistant you use:

```bash
npm i -g ohlc-resample
```

That installs both the `ohlc` command and the MCP server. Most assistants
auto-detect it; for the ones that need a nudge, add a server entry:

```json
{
"mcpServers": {
"ohlc-resample": {
"command": "ohlc-resample-mcp",
"args": []
}
}
}
```

Your assistant gets two tools.

**`resample_ohlcv_file`** reshapes your data. Give it the file to read and
the time frame you want, and it does the rest:

- 1-minute bars into 5-minute (or any coarser frame)
- raw trades or ticks into clean OHLCV
- CCXT `timestamp` / `amount` data without renaming a thing
- CSV, JSON, JSONL, or Parquet in; JSON, CSV, or JSONL out

Pass an output file and it writes there, or let it return the data directly.
No need to paste anything into the chat.

Example: resample 1-minute bars to 5-minute and save them.

`resample_ohlcv_file(input_path: "data.csv", base_timeframe: 60, new_timeframe: 300, output_path: "out.json")`

**`audit_ohlcv_file`** tells you whether you can trust the source before you
resample it, and exactly why. Point it at the same file and it reports the
record count, the time span, the source time frame, any out-of-order or
duplicate bars, bars whose high/low/open/close don't add up, NaN or negative
values, and bars that are simply missing.

Example: check a feed before committing to a resample.

`audit_ohlcv_file(input_path: "data.csv")`

## Contributors

👤 **Adil Shaikh <hello@adils.me> (https://adils.me)**
Expand Down
202 changes: 202 additions & 0 deletions __tests__/audit.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,202 @@
"use strict";

import { test, describe, expect, beforeAll, afterEach, afterAll } from "vitest";
import * as fs from "fs";
import * as path from "path";
import * as os from "os";
import { Writable } from "stream";
import { auditOhlcv } from "../src";
import { runCli } from "../src/cli";
import { dispatch } from "../src/mcp";
import { withTimeout } from "./utils";
import type { IOHLCV } from "../src/types";

const clean: IOHLCV[] = [
{ time: 1609459200000, open: 100, high: 105, low: 95, close: 102, volume: 1000 },
{ time: 1609459260000, open: 102, high: 107, low: 101, close: 106, volume: 1200 },
{ time: 1609459320000, open: 106, high: 108, low: 104, close: 105, volume: 800 },
{ time: 1609459380000, open: 105, high: 106, low: 103, close: 104, volume: 900 },
{ time: 1609459440000, open: 104, high: 105, low: 102, close: 103, volume: 1100 },
];

async function* iter(data: IOHLCV[]) {
for (const c of data) yield c;
}

describe("auditOhlcv", () => {
test("reports a clean 1-minute series", async () => {
const r = await auditOhlcv(iter(clean));
expect(r.records).toBe(5);
expect(r.timeRange.startMs).toBe(1609459200000);
expect(r.timeRange.endMs).toBe(1609459440000);
expect(r.timeRange.spanMs).toBe(240000);
expect(r.baseTimeframe).toBe(60);
expect(r.ordering).toEqual({ sorted: true, outOfOrder: 0, maxLatenessMs: 0 });
expect(r.duplicates.duplicateTimestamps).toBe(0);
expect(r.ohlc.invalidBars).toBe(0);
expect(r.values).toEqual({ nan: 0, infinity: 0, negativePrices: 0, negativeVolume: 0 });
expect(r.gaps).toEqual({ expectedBars: 5, observed: 5, missing: 0 });
});

test("flags out-of-order, duplicate, invalid-bar, and bad-value records", async () => {
const messy: IOHLCV[] = [
{ time: 1609459200000, open: 1, high: 2, low: 0, close: 1.5, volume: 10 },
{ time: 1609459260000, open: 1, high: 2, low: 3, close: 1, volume: 5 }, // high < low
{ time: 1609459200000, open: 1, high: 2, low: 0, close: 1.5, volume: 10 }, // duplicate
{ time: 1609459380000, open: 2, high: 3, low: 1, close: 2, volume: -1 }, // negative volume
{ time: 1609459320000, open: 1, high: 2, low: 0, close: 1, volume: 8 }, // out-of-order
{ time: 1609459440000, open: NaN, high: 2, low: 0, close: 1, volume: 8 }, // NaN
];
const r = await auditOhlcv(iter(messy));
expect(r.ordering.sorted).toBe(false);
expect(r.ordering.outOfOrder).toBeGreaterThan(0);
expect(r.ordering.maxLatenessMs).toBeGreaterThan(0);
expect(r.duplicates.duplicateTimestamps).toBe(1);
expect(r.ohlc.invalidBars).toBe(1);
expect(r.values.negativeVolume).toBe(1);
expect(r.values.nan).toBeGreaterThan(0);
});

test("computes missing bars from the modal timeframe", async () => {
const gappy: IOHLCV[] = [
{ time: 1609459200000, open: 1, high: 2, low: 0, close: 1, volume: 10 },
{ time: 1609459260000, open: 1, high: 2, low: 0, close: 1, volume: 10 },
{ time: 1609459380000, open: 1, high: 2, low: 0, close: 1, volume: 10 },
{ time: 1609459440000, open: 1, high: 2, low: 0, close: 1, volume: 10 },
];
const r = await auditOhlcv(iter(gappy));
expect(r.baseTimeframe).toBe(60);
expect(r.gaps.expectedBars).toBe(5);
expect(r.gaps.observed).toBe(4);
expect(r.gaps.missing).toBe(1);
});

test("honors an explicit baseTimeframe option", async () => {
const r = await auditOhlcv(iter(clean), { baseTimeframe: 300 });
expect(r.baseTimeframe).toBe(300);
expect(r.gaps.expectedBars).toBe(1);
});

test("applies a field map to object records", async () => {
const mapped = clean.slice(0, 2).map((c) => ({
timestamp: c.time,
amount: c.volume,
open: c.open,
high: c.high,
low: c.low,
close: c.close,
}));
const r = await auditOhlcv(iter(mapped as unknown as IOHLCV[]), {
map: { time: "timestamp", volume: "amount" },
});
expect(r.records).toBe(2);
expect(r.timeRange.startMs).toBe(1609459200000);
expect(r.values.negativeVolume).toBe(0);
});

test("audits a parquet file path directly", async () => {
const p = path.join(import.meta.dirname, "fixtures", "ohlcv.parquet");
const r = await auditOhlcv(p);
expect(r.records).toBeGreaterThan(0);
expect(r.baseTimeframe).toBeGreaterThan(0);
expect(r.ohlc.invalidBars).toBe(0);
});

test("throws on empty input", async () => {
await expect(auditOhlcv(iter([]))).rejects.toThrow("no candles");
});
});

describe("CLI --audit and MCP audit_ohlcv_file", () => {
let tempDir: string;
let csvPath: string;
let messyJsonlPath: string;

function captureWritable() {
let data = "";
const writable = new Writable({
write(chunk, _encoding, callback) {
data += chunk.toString();
callback();
},
});
return { writable, getData: () => data };
}

beforeAll(() => {
tempDir = fs.mkdtempSync(path.join(os.tmpdir(), "audit-test-"));
const csvRows = clean.map((c) => `${c.time},${c.open},${c.high},${c.low},${c.close},${c.volume}`).join("\n");
csvPath = path.join(tempDir, "ohlcv.csv");
fs.writeFileSync(csvPath, "time,open,high,low,close,volume\n" + csvRows);

const messy = [
{ time: 1609459200000, open: 1, high: 2, low: 0, close: 1.5, volume: 10 },
{ time: 1609459260000, open: 1, high: 2, low: 3, close: 1, volume: 5 },
{ time: 1609459200000, open: 1, high: 2, low: 0, close: 1.5, volume: 10 },
{ time: 1609459380000, open: 2, high: 3, low: 1, close: 2, volume: -1 },
{ time: 1609459320000, open: 1, high: 2, low: 0, close: 1, volume: 8 },
];
messyJsonlPath = path.join(tempDir, "messy.jsonl");
fs.writeFileSync(messyJsonlPath, messy.map((c) => JSON.stringify(c)).join("\n"));
});

afterEach(async () => {
process.exitCode = 0;
});

test("prints a JSON trust report for a CSV file", async () => {
await withTimeout(async () => {
const out = captureWritable();
const err = captureWritable();
await runCli(["node", "cli.js", "-i", csvPath, "--audit"], undefined, out.writable, err.writable);
const r = JSON.parse(out.getData());
expect(r.format).toBe("csv");
expect(r.records).toBe(5);
expect(r.baseTimeframe).toBe(60);
expect(r.schema).toEqual(["time", "open", "high", "low", "close", "volume"]);
expect(r.gaps.missing).toBe(0);
}, 1000, "audit csv");
});

test("flags issues in a messy JSONL file", async () => {
await withTimeout(async () => {
const out = captureWritable();
const err = captureWritable();
await runCli(["node", "cli.js", "-i", messyJsonlPath, "--audit"], undefined, out.writable, err.writable);
const r = JSON.parse(out.getData());
expect(r.format).toBe("jsonl");
expect(r.ordering.sorted).toBe(false);
expect(r.ordering.outOfOrder).toBeGreaterThan(0);
expect(r.duplicates.duplicateTimestamps).toBeGreaterThan(0);
expect(r.ohlc.invalidBars).toBeGreaterThan(0);
expect(r.values.negativeVolume).toBeGreaterThan(0);
}, 1000, "audit messy jsonl");
});

test("audits a parquet file", async () => {
await withTimeout(async () => {
const p = path.join(import.meta.dirname, "fixtures", "ohlcv.parquet");
const out = captureWritable();
const err = captureWritable();
await runCli(["node", "cli.js", "-i", p, "--audit"], undefined, out.writable, err.writable);
const r = JSON.parse(out.getData());
expect(r.format).toBe("parquet");
expect(r.records).toBeGreaterThan(0);
}, 1000, "audit parquet");
});

test("MCP audit_ohlcv_file returns an audit report", async () => {
const res = await dispatch({
jsonrpc: "2.0",
id: 1,
method: "tools/call",
params: { name: "audit_ohlcv_file", arguments: { input_path: csvPath } },
}) as any;
const r = JSON.parse(res.content[0].text);
expect(r.records).toBe(5);
expect(r.baseTimeframe).toBe(60);
expect(r.timeRange.startMs).toBe(1609459200000);
});

afterAll(() => fs.rmSync(tempDir, { recursive: true, force: true }));
});
14 changes: 14 additions & 0 deletions __tests__/cli.ts
Original file line number Diff line number Diff line change
Expand Up @@ -333,6 +333,20 @@ describe('CLI', () => {
}, 1000, 'stream CSV to JSONL');
});

test('streams CSV file -> CSV output (object shape to CSV rows)', async () => {
await withTimeout(async () => {
const out = captureWritable();
const err = captureWritable();
await runCli(['node', 'cli.js', '-i', csvPath, '-f', 'csv'], undefined, out.writable, err.writable);
const lines = out.getData().trim().split('\n');
expect(lines[0]).toBe('time,open,high,low,close,volume');
expect(lines[1]).toBe(
`${expectedCandle.time},${expectedCandle.open},${expectedCandle.high},${expectedCandle.low},${expectedCandle.close},${expectedCandle.volume}`
);
expect(process.exitCode).toBe(0);
}, 1000, 'stream CSV to CSV');
});

test('streams JSONL file input', async () => {
await withTimeout(async () => {
const jsonlPath = path.join(tempDir, 'test.jsonl');
Expand Down
Loading
Loading