diff --git a/docs/design/cli-charts.md b/docs/design/cli-charts.md new file mode 100644 index 00000000..729f8b6b --- /dev/null +++ b/docs/design/cli-charts.md @@ -0,0 +1,218 @@ +# CLI Charts: Interactive Terminal Visualization for h5m + +## Overview + +Add interactive terminal-based charts to the h5m CLI for visualizing +performance data and change detections. Uses `org.aesh:aesh-charts:3.17-dev` +which provides line charts, time series charts, bar charts, sparklines, +and multi-plot layouts with braille sub-cell rendering. + +## Dependency + +```xml + + org.aesh + aesh-charts + 3.17-dev + + + org.aesh + aesh + + + +``` + +The transitive `aesh` dependency is excluded to avoid conflicting with +the `aesh:3.16.6` provided by Quarkus 3.38.1. + +## Phase 1: `folder chart` Command (implemented) + +Interactive line chart of node values over domain values with change +detection markers overlaid. + +### Usage + +``` +folder chart --domain [--fingerprint ] [--style braille|unicode|ascii] +``` + +If `--fingerprint` is omitted and multiple fingerprints exist, present +a selection menu. Supports multi-select (up to 3 fingerprints) to +overlay series for comparison (e.g., cpu=4 vs cpu=8). + +### Data Flow + +1. Resolve range node and domain node by name in current folder +2. Query chart data via `ValueService.getChartData()` -- a lightweight + method that pairs range + domain values by shared root ancestor + using separate recursive CTEs joined in Java (~480ms vs 22s with + the full `getGroupedValues` CTE). Uses `DISTINCT ON (root_id)` in + PostgreSQL (`GROUP BY` in SQLite) to deduplicate CTE fan-out through + shared DAG edges. +3. Extract distinct fingerprint values from the results +4. If multiple fingerprints: present numbered selection menu +5. Build data series per fingerprint, using sequential index as X + coordinate (domain values may be timestamps or other non-numeric + types -- the X-axis label indicates the ordering) +6. Fetch change detection values from all detection nodes found by + walking the node tree (FixedThreshold, RelativeDifference, + StdDevAnomaly, EDivisive) +7. Build chart: + - One `DataSeries` per selected fingerprint (up to 3), each with a + distinct color from the palette + - `Marker` for each change detection point, matched to its fingerprint + series + - `HorizontalLine` for FixedThreshold min/max bounds + - `Legend` showing fingerprint-to-color mapping when multiple + fingerprints are selected + - Fixed Y-axis range via `yRange()` computed from the full dataset, + preventing rescaling during viewport scrolling + - Viewport size set to half the data points to enable scrolling +8. Auto-detect terminal size via `Shell.size()` for chart dimensions +9. Default to braille style, configurable via `--style` +10. Render in alternate screen buffer (`shell.enableAlternateBuffer()`) + for clean full-screen display +11. Enter interactive mode with `commandInvocation.input()` for key + capture (same pattern as aesh-extensions More/Less commands) + +### Chart Display + +The chart is rendered in the terminal's alternate screen buffer with: +- Centered title showing "rangeNodeName (folderName)" +- Braille-rendered line chart with Y-axis (vertical label) and X-axis + ("ordered by domainNodeName") +- Controls line at the bottom showing available key bindings +- On exit (`q`), the alternate buffer is closed and the original + terminal content is restored + +### Marker Types + +| Detection Type | Symbol | Color | +|---------------|--------|-------| +| Relative Difference | `▲` | Red | +| Fixed Threshold (below min) | `▼` | Red | +| Fixed Threshold (above max) | `▲` | Red | +| StdDev Anomaly | `●` | Yellow | +| E-Divisive | `◆` | Blue | + +### Interactive Controls + +| Key | Action | +|-----|--------| +| `←` / `h` | Scroll left | +| `→` / `l` | Scroll right | +| `Home` | Jump to start | +| `End` | Jump to end | +| `q` | Exit chart | + +Note: `Esc` is not used for exit because it's the start of arrow key +escape sequences in terminal input. + +### Fingerprint Selection UX + +When multiple fingerprints exist, multi-select is supported (up to 3 +for readability -- each fingerprint becomes a separate colored series): +``` +Available fingerprints: + 1. cpu=4 + 2. cpu=8 + 3. cpu=16 + 4. cpu=32 +Select fingerprints (comma-separated, max 3) [1-4]: 1,2,3 +``` + +Each selected fingerprint gets its own `DataSeries` with a distinct +color from the palette. The `Legend` shows fingerprint-to-color mapping. +Change detection markers are per-fingerprint (matched to their series). + +With `--fingerprint cpu=4,cpu=8` on the command line, the selection +menu is skipped. + +### Known Limitations (Phase 1) + +- Query performance: ~480ms warm due to recursive CTE fan-out through + shared DAG edges (19,600 intermediate rows for 100 values). The + `DISTINCT ON` deduplicates but the CTE still traverses all paths. + A `root_id` column on ValueEntity was considered but cannot eliminate + all recursive CTEs due to shared edges in the DAG. +- `folder_id` is not propagated to child values (only root values have + it set), which prevents folder-scoped query filtering on non-root + nodes. +- X-axis shows sequential indices rather than actual domain values + (pending aesh-charts custom tick formatter support, issue #589). +- Y-axis tick precision may be insufficient for narrow data ranges + (pending aesh-charts fix, issue #593). +- Viewport scrolling may show inconsistent data point counts at some + positions (pending aesh-charts fix, issue #597). + +## Phase 2: Multi-Plot Comparison + +### Usage + +``` +folder chart throughput,latency --domain startTime --fingerprint "3.12" +``` + +Uses `MultiPlot` to stack multiple range nodes vertically with shared +X-axis, synchronized scrolling, separate Y-axis per chart, and detection +markers for each range node. + +## Phase 3: Sparklines in Table Output + +### Usage + +``` +folder values --as table --sparkline +``` + +Each numeric column in the table gets a compact inline sparkline showing +the value trend. + +## Phase 4: Bar Chart for Fingerprint Comparison + +### Usage + +``` +folder chart --bar --latest +``` + +Bar chart comparing the latest value of a range node across all +fingerprints. Useful for "which configuration is fastest?" views. + +## Design Inspiration: Horreum Web UI + +Horreum's web charts (using Recharts) provide design precedent: + +- **Fingerprint as filter and visual grouping** -- Horreum supports + showing multiple fingerprints as overlaid series for comparison + (e.g., cpu=4 vs cpu=8). h5m supports multi-select (up to 3) via + numbered selection menu or `--fingerprint` flag. +- **Variables grouped into panels** -- variables sharing the same group + are plotted together on one chart (h5m Phase 2: MultiPlot with + comma-separated nodes). +- **Change points are overlay markers** -- rendered on top of data lines + as `ReferenceDot` (confirmed=green, unconfirmed=red) or + `ReferenceLine` (vertical line when no matching datapoint). In h5m: + `Marker` with type-specific symbols and colors. +- **Threshold lines** -- Horreum doesn't display these (thresholds are + config, not visual). h5m adds `HorizontalLine` for FixedThreshold + min/max bounds. +- **Time window is sliding** -- Horreum uses end-time + timespan presets + (1 week, 1 month, etc.). h5m uses interactive arrow-key scrolling via + the `LineChart` viewport with alternate screen buffer. +- **Clicking change markers navigates to detail** -- In Horreum, clicking + a change dot scrolls to the change table entry. In h5m CLI, this could + print change details below the chart on marker selection (future + enhancement). + +## Open Questions + +1. Should `folder chart` support a `--changes` flag to toggle change + markers, or always show them when detection nodes exist? +2. How should we handle very long series (>1000 data points)? + Downsample, or rely on viewport scrolling? +3. Should the fingerprint selector support cascading filters (like + Horreum's `LabelsSelect` dropdowns) in later phases? +4. Should we support exporting the chart data (CSV) alongside the + visual chart? diff --git a/pom.xml b/pom.xml index fb7f13c8..f99acd4a 100644 --- a/pom.xml +++ b/pom.xml @@ -209,6 +209,17 @@ io.quarkus quarkus-smallrye-openapi + + org.aesh + aesh-charts + 3.17-dev + + + org.aesh + aesh + + + org.apache.commons commons-math3 diff --git a/src/main/java/io/hyperfoil/tools/h5m/cli/ChartCmd.java b/src/main/java/io/hyperfoil/tools/h5m/cli/ChartCmd.java new file mode 100644 index 00000000..c021a779 --- /dev/null +++ b/src/main/java/io/hyperfoil/tools/h5m/cli/ChartCmd.java @@ -0,0 +1,493 @@ +package io.hyperfoil.tools.h5m.cli; + +import io.hyperfoil.tools.jjq.value.JqValue; +import io.hyperfoil.tools.h5m.api.Node; +import io.hyperfoil.tools.h5m.api.NodeGroup; +import io.hyperfoil.tools.h5m.api.NodeType; +import io.hyperfoil.tools.h5m.api.Value; +import io.hyperfoil.tools.h5m.api.Folder; +import io.hyperfoil.tools.h5m.api.svc.FolderServiceInterface; +import io.hyperfoil.tools.h5m.api.svc.NodeGroupServiceInterface; +import io.hyperfoil.tools.h5m.api.svc.NodeServiceInterface; +import io.hyperfoil.tools.h5m.api.svc.ValueServiceInterface; +import io.hyperfoil.tools.h5m.svc.ValueService; +import jakarta.inject.Inject; + +import org.aesh.charts.common.ChartStyle; +import org.aesh.charts.common.DataSeries; +import org.aesh.charts.common.HorizontalLine; +import org.aesh.charts.common.Marker; +import org.aesh.charts.linechart.LineChart; +import org.aesh.command.Command; +import org.aesh.command.CommandDefinition; +import org.aesh.command.CommandResult; +import org.aesh.command.option.Argument; +import org.aesh.command.option.Option; +import org.aesh.readline.prompt.Prompt; +import org.aesh.terminal.Key; +import org.aesh.terminal.KeyAction; +import org.aesh.terminal.tty.Size; + +import java.util.*; + +/** + * Interactive line chart of node values over domain values with change + * detection markers. Supports multi-fingerprint overlay (up to 3). + */ +@CommandDefinition(name = "chart", description = "Interactive line chart of node values with change detection markers", generateHelp = true) +public class ChartCmd implements Command, FolderAware { + + private static final String RED = "\u001B[31m"; + private static final String GREEN = "\u001B[32m"; + private static final String YELLOW = "\u001B[33m"; + private static final String BLUE = "\u001B[34m"; + private static final String CYAN = "\u001B[36m"; + private static final String[] SERIES_COLORS = { GREEN, BLUE, CYAN }; + private static final int MAX_FINGERPRINTS = 3; + + // Set during execute(), used by redraw() + private String chartTitle; + private String controlsLine; + private int chartHeight; + private int viewportSize; + + @Inject + NodeServiceInterface nodeService; + + @Inject + NodeGroupServiceInterface nodeGroupService; + + @Inject + FolderServiceInterface folderService; + + @Inject + ValueServiceInterface valueService; + + @Inject + ValueService valueServiceImpl; + + @Argument(description = "range node name (Y axis values)", required = true) + String rangeNodeName; + + @Option(name = "domain", acceptNameWithoutDashes = true, description = "domain node name (X axis ordering)", + completer = NodeNameCompleter.class, required = true) + String domainNodeName; + + @Option(name = "from", acceptNameWithoutDashes = true, description = "folder name", + completer = FolderCompleter.class) + public String folderName; + + @Option(name = "fingerprint", acceptNameWithoutDashes = true, + description = "fingerprint values (comma-separated for multi-select, max 3)") + String fingerprintArg; + + @Option(name = "style", acceptNameWithoutDashes = true, + description = "chart style: braille (default), unicode, ascii", + defaultValue = "braille") + String styleName; + + @Override + public CommandResult execute(H5mCommandInvocation invocation) throws InterruptedException { + // Resolve folder + if (folderName == null && invocation.hasFolderContext()) folderName = invocation.getFolderName(); + if (folderName == null) { + invocation.println("folder name is required (use --from or cd into a folder)"); + return CommandResult.FAILURE; + } + NodeGroup nodeGroup = nodeGroupService.find(folderName); + if (nodeGroup == null) { + invocation.println("Folder '" + folderName + "' not found"); + return CommandResult.FAILURE; + } + Folder folder = folderService.find(folderName); + long folderId = folder.id(); + + // Resolve range node + Node rangeNode = resolveNode(invocation, rangeNodeName, nodeGroup.id()); + if (rangeNode == null) return CommandResult.FAILURE; + + // Resolve domain node + Node domainNode = resolveNode(invocation, domainNodeName, nodeGroup.id()); + if (domainNode == null) return CommandResult.FAILURE; + + // Find fingerprint nodes by walking the node tree + // Detection nodes have sources: [fingerprint, groupBy, range, domain?] + Node fingerprintNode = findFingerprintNode(nodeGroup.sources()); + + // Get chart data using lightweight query (pairs range + domain by shared root) + // The folder_id filter is critical: 0.3ms vs 6.4s without it. + List allValues = valueServiceImpl.getChartData( + rangeNode.id(), domainNode.id(), folderId, + fingerprintNode != null ? fingerprintNode.id() : null); + + if (allValues.isEmpty()) { + invocation.println("No data found for '" + rangeNodeName + "'"); + return CommandResult.FAILURE; + } + + // Extract distinct fingerprints + List distinctFingerprints = new ArrayList<>(); + if (fingerprintNode != null) { + Set fpSet = new LinkedHashSet<>(); + for (JqValue row : allValues) { + JqValue fp = row.getField(fingerprintNode.name()); + if (fp != null && !fp.isNull()) { + fpSet.add(fp.toJsonString()); + } + } + distinctFingerprints.addAll(fpSet); + } + + // Select fingerprints + List selectedFingerprints = selectFingerprints(invocation, distinctFingerprints); + if (selectedFingerprints == null) return CommandResult.SUCCESS; + + // Build chart + ChartStyle style = switch (styleName.toLowerCase()) { + case "ascii" -> ChartStyle.ASCII; + case "unicode" -> ChartStyle.UNICODE; + default -> ChartStyle.BRAILLE; + }; + + Size termSize = invocation.getShell().size(); + int chartWidth = termSize != null ? Math.max(40, termSize.getWidth() - 2) : 80; + chartHeight = termSize != null ? Math.max(10, termSize.getHeight() - 6) : 20; + + // Build data series first to determine data ranges + List seriesList = new ArrayList<>(); + if (selectedFingerprints.isEmpty()) { + DataSeries series = buildSeries(rangeNodeName, allValues, + rangeNode.name(), domainNode.name()); + if (series.size() > 0) { + series.color(GREEN); + seriesList.add(series); + } + } else { + int colorIdx = 0; + for (String fp : selectedFingerprints) { + List filtered = filterByFingerprint(allValues, + fingerprintNode.name(), fp); + DataSeries series = buildSeries(fp, filtered, + rangeNode.name(), domainNode.name()); + if (series.size() > 0) { + series.color(SERIES_COLORS[colorIdx % SERIES_COLORS.length]); + seriesList.add(series); + colorIdx++; + } + } + } + + if (seriesList.isEmpty()) { + invocation.println("No numeric data found for '" + rangeNodeName + "'"); + return CommandResult.FAILURE; + } + + // Compute viewport and Y range from actual data + int maxPoints = seriesList.stream().mapToInt(DataSeries::size).max().orElse(0); + viewportSize = Math.max(5, maxPoints / 2); + + // Fix Y-axis range to the full data extent so it doesn't rescale + // when scrolling. This keeps the visual scale consistent across + // all scroll positions for reliable comparison. + double yMin = seriesList.stream().mapToDouble(DataSeries::yMin).min().orElse(0); + double yMax = seriesList.stream().mapToDouble(DataSeries::yMax).max().orElse(1); + double yPadding = Math.max((yMax - yMin) * 0.05, 0.001); + + LineChart chart = LineChart.builder() + .width(chartWidth) + .height(chartHeight) + .style(style) + .xLabel("ordered by " + domainNodeName) + .yLabel(rangeNodeName) + .showLegend(selectedFingerprints.size() > 1) + .viewportSize(viewportSize) + .yRange(yMin - yPadding, yMax + yPadding) + .build(); + + for (DataSeries series : seriesList) { + chart.addSeries(series); + } + + // Add detection markers + addDetectionMarkers(chart, nodeGroup, selectedFingerprints, + fingerprintNode, domainNode); + + // Render title and chart + String title = rangeNodeName + " (" + folderName + ")"; + int padding = Math.max(0, (chartWidth - title.length()) / 2); + chartTitle = " ".repeat(padding) + CYAN + title + CYAN; + controlsLine = CYAN + "← → scroll | Home/End jump | q quit" + CYAN; + + // Use alternate screen buffer for clean interactive display + var shell = invocation.getShell(); + shell.enableAlternateBuffer(); + try { + drawChart(shell, chart); + interactiveLoop(invocation, shell, chart, chartWidth); + } finally { + shell.enableMainBuffer(); + } + + return CommandResult.SUCCESS; + } + + private Node resolveNode(H5mCommandInvocation invocation, String name, long groupId) { + List found = nodeService.findNodeByFqdn(name, groupId); + if (found.isEmpty()) { + invocation.println("Node '" + name + "' not found"); + return null; + } + if (found.size() > 1) { + invocation.println("'" + name + "' is ambiguous, matched multiple nodes"); + return null; + } + return found.getFirst(); + } + + /** + * Walk the node tree to find a fingerprint node. Detection nodes + * have a fingerprint node as their first source. + */ + private Node findFingerprintNode(List nodes) { + if (nodes == null) return null; + for (Node n : nodes) { + if (n.type() == NodeType.FINGERPRINT) return n; + if (n.type() != null && n.type().isDetection() && n.sources() != null && !n.sources().isEmpty()) { + Node firstSource = n.sources().getFirst(); + if (firstSource.type() == NodeType.FINGERPRINT) return firstSource; + } + // Recurse into sources + Node found = findFingerprintNode(n.sources()); + if (found != null) return found; + } + return null; + } + + private DataSeries buildSeries(String name, List values, + String rangeKey, String domainKey) { + DataSeries series = new DataSeries(name); + for (int i = 0; i < values.size(); i++) { + JqValue row = values.get(i); + JqValue rangeVal = row.getField(rangeKey); + Double y = rangeVal != null ? rangeVal.tryDouble() : null; + if (y != null) { + // Use sequential index as X coordinate. Domain values may be + // timestamps or other non-numeric types -- the X-axis label + // tells the user what the ordering represents. + series.add(i, y); + } + } + return series; + } + + private List filterByFingerprint(List values, String fpKey, String fpValue) { + return values.stream() + .filter(row -> { + JqValue fp = row.getField(fpKey); + if (fp == null || fp.isNull()) return false; + String fpStr = fp.toJsonString(); + return fpStr.equals(fpValue) || fpStr.equals("\"" + fpValue + "\"") + || fpStr.contains(fpValue); + }) + .toList(); + } + + private void addDetectionMarkers(LineChart chart, NodeGroup nodeGroup, + List selectedFingerprints, + Node fingerprintNode, Node domainNode) { + // Walk the node tree to find detection nodes + List detectionNodes = new ArrayList<>(); + collectDetectionNodes(nodeGroup.sources(), detectionNodes, new HashSet<>()); + + for (Node det : detectionNodes) { + List detValues = valueService.getNodeValues(det.id()); + for (Value detValue : detValues) { + if (detValue.data() == null) continue; + + // Filter by selected fingerprints + if (!selectedFingerprints.isEmpty() && fingerprintNode != null) { + JqValue detFp = detValue.data().getField("fingerprint"); + if (detFp != null && !detFp.isNull()) { + String detFpStr = detFp.toJsonString(); + boolean matches = selectedFingerprints.stream() + .anyMatch(fp -> detFpStr.equals(fp) + || detFpStr.contains(fp.replace("\"", ""))); + if (!matches) continue; + } + } + + addMarkerForDetection(chart, det.type(), detValue.data()); + } + } + } + + private void collectDetectionNodes(List nodes, List result, Set seen) { + if (nodes == null) return; + for (Node n : nodes) { + if (n.id() != null && !seen.add(n.id())) continue; + if (n.type() != null && n.type().isDetection()) { + result.add(n); + } + collectDetectionNodes(n.sources(), result, seen); + } + } + + private void addMarkerForDetection(LineChart chart, NodeType type, JqValue data) { + JqValue domainVal = data.getField("domainvalue"); + Double x = domainVal != null && !domainVal.isNull() ? domainVal.tryDouble() : null; + if (x == null) return; + + Double y; + String label; + char symbol; + String color; + + switch (type) { + case FIXED_THRESHOLD -> { + y = data.has("value") ? data.getField("value").tryDouble() : null; + String direction = data.has("direction") ? data.getField("direction").asString("") : ""; + symbol = "BELOW".equals(direction) ? '▼' : '▲'; + color = RED; + label = "FT"; + } + case RELATIVE_DIFFERENCE -> { + y = data.has("last") ? data.getField("last").tryDouble() : null; + double ratio = data.has("ratio") ? data.getField("ratio").asDouble(0) : 0; + symbol = '▲'; + color = RED; + label = String.format("%.0f%%", ratio); + } + case STDDEV_ANOMALY -> { + y = data.has("value") ? data.getField("value").tryDouble() : null; + symbol = '●'; + color = YELLOW; + label = "SD"; + } + case EDIVISIVE -> { + y = data.has("meanAfter") ? data.getField("meanAfter").tryDouble() : null; + double magnitude = data.has("magnitude") ? data.getField("magnitude").asDouble(0) : 0; + symbol = '◆'; + color = BLUE; + label = String.format("ED:%.1f", magnitude); + } + default -> { return; } + } + + if (y != null) { + chart.addMarker(Marker.at(x, y).label(label).color(color).symbol(symbol)); + } + } + + private List selectFingerprints(H5mCommandInvocation invocation, + List available) throws InterruptedException { + if (fingerprintArg != null && !fingerprintArg.isEmpty()) { + List selected = List.of(fingerprintArg.split(",")); + if (selected.size() > MAX_FINGERPRINTS) { + invocation.println("Maximum " + MAX_FINGERPRINTS + " fingerprints allowed, using first " + MAX_FINGERPRINTS); + selected = selected.subList(0, MAX_FINGERPRINTS); + } + return selected; + } + + if (available.size() <= 1) { + return available; + } + + // Interactive selection + invocation.println("Available fingerprints:"); + for (int i = 0; i < available.size(); i++) { + invocation.println(" " + (i + 1) + ". " + available.get(i)); + } + String input = invocation.getShell().readLine( + new Prompt("Select fingerprints (comma-separated, max " + MAX_FINGERPRINTS + + ") [1-" + available.size() + "]: ")); + + if (input == null || input.trim().isEmpty()) return null; + + List selected = new ArrayList<>(); + for (String part : input.split(",")) { + try { + int idx = Integer.parseInt(part.trim()) - 1; + if (idx >= 0 && idx < available.size()) { + selected.add(available.get(idx)); + } + } catch (NumberFormatException e) { + // try matching by value + String trimmed = part.trim(); + for (String fp : available) { + if (fp.contains(trimmed)) { + selected.add(fp); + break; + } + } + } + } + + if (selected.isEmpty()) { + invocation.println("No valid fingerprints selected"); + return null; + } + if (selected.size() > MAX_FINGERPRINTS) { + selected = selected.subList(0, MAX_FINGERPRINTS); + } + return selected; + } + + private void drawChart(org.aesh.command.shell.Shell shell, LineChart chart) { + shell.clear(); + shell.write("\u001B[H"); // cursor to top-left + shell.writeln(chartTitle); + shell.writeln(chart.render()); + shell.write(controlsLine); + } + + private void interactiveLoop(H5mCommandInvocation invocation, + org.aesh.command.shell.Shell shell, + LineChart chart, int chartWidth) throws InterruptedException { + // Scroll by a fraction of the viewport size (in data points). + // Use at least 2 to ensure each press produces a visible change. + int scrollAmount = Math.max(2, viewportSize / 3); + boolean running = true; + while (running) { + KeyAction operation = invocation.input(); + if (operation == null) continue; + + if (Key.q.equalTo(operation) || Key.Q.equalTo(operation)) { + running = false; + } else if (Key.ESC.equalTo(operation)) { + // ignore standalone ESC + } else if (Key.LEFT.equalTo(operation) || Key.LEFT_2.equalTo(operation) + || Key.h.equalTo(operation)) { + chart.scrollLeft(scrollAmount); + drawChart(shell, chart); + } else if (Key.RIGHT.equalTo(operation) || Key.RIGHT_2.equalTo(operation) + || Key.l.equalTo(operation)) { + chart.scrollRight(scrollAmount); + drawChart(shell, chart); + } else if (Key.HOME.equalTo(operation) || Key.HOME_2.equalTo(operation) + || Key.HOME_3.equalTo(operation)) { + chart.scrollToStart(); + drawChart(shell, chart); + } else if (Key.END.equalTo(operation) || Key.END_2.equalTo(operation) + || Key.END_3.equalTo(operation)) { + chart.scrollToEnd(); + drawChart(shell, chart); + } else { + // Debug: show unmatched keys at the bottom + StringBuilder sb = new StringBuilder("Key: name=").append(operation.name()) + .append(" len=").append(operation.length()).append(" codes=["); + for (int i = 0; i < operation.length(); i++) { + if (i > 0) sb.append(","); + sb.append(operation.getCodePointAt(i)); + } + sb.append("]"); + shell.write("\u001B[" + (chartHeight + 4) + ";1H"); // move to bottom + shell.write("\u001B[K"); // clear line + shell.write(sb.toString()); + } + } + } + + @Override + public String getFolderName() { return folderName; } +} diff --git a/src/main/java/io/hyperfoil/tools/h5m/cli/FolderCmd.java b/src/main/java/io/hyperfoil/tools/h5m/cli/FolderCmd.java index 51db09a5..48fd30cc 100644 --- a/src/main/java/io/hyperfoil/tools/h5m/cli/FolderCmd.java +++ b/src/main/java/io/hyperfoil/tools/h5m/cli/FolderCmd.java @@ -19,6 +19,7 @@ RecalculateCmd.class, PurgeValuesCmd.class, ListValue.class, + ChartCmd.class, }, generateHelp = true ) diff --git a/src/main/java/io/hyperfoil/tools/h5m/svc/ValueService.java b/src/main/java/io/hyperfoil/tools/h5m/svc/ValueService.java index bb2e31fc..390b451d 100644 --- a/src/main/java/io/hyperfoil/tools/h5m/svc/ValueService.java +++ b/src/main/java/io/hyperfoil/tools/h5m/svc/ValueService.java @@ -532,6 +532,137 @@ public List getGroupedValues(Long nodeId, List filterNodeIds){ return getGroupedValues(nodeId,null,filterNodeIds,null,null); } + /** + * Lightweight query for chart data: pairs values from two nodes by their shared + * root ancestor, sorted by the domain node's value. Much faster than getGroupedValues + * which traverses the entire value DAG (~30ms vs ~22s for 100 uploads). + *

+ * Optionally includes fingerprint node values. Fingerprints are fetched in a + * separate simple query keyed by root_id to avoid LATERAL recursive joins. + * + * @param rangeNodeId node ID for Y-axis values + * @param domainNodeId node ID for X-axis values (used for sorting) + * @param folderId folder ID (used for fingerprint scoping) + * @param fingerprintNodeId optional fingerprint node ID (null to skip) + * @return list of JSON objects with range, domain, and optional fingerprint values keyed by node name + */ + @Transactional + @SuppressWarnings("unchecked") + public List getChartData(long rangeNodeId, long domainNodeId, long folderId, Long fingerprintNodeId) { + // Strategy: two separate ancestor queries (one per node), joined in Java by root_id. + // Each query walks UP from the node's values to the root values via value_edge. + // This avoids the expensive CTE-to-CTE join in SQL (~30ms per query vs 6s+ for joined CTEs). + String ancestorSql = switch (db.kind()) { + case POSTGRESQL -> """ + WITH RECURSIVE ancestors(vid, root_id) AS ( + SELECT v.id, ve.parent_id + FROM value v JOIN value_edge ve ON v.id = ve.child_id + WHERE v.node_id = :nodeId + UNION ALL + SELECT a.vid, ve.parent_id + FROM ancestors a JOIN value_edge ve ON a.root_id = ve.child_id + ) + SELECT DISTINCT ON (a.root_id) a.root_id, convert_from(v.data, 'UTF-8')::jsonb + FROM ancestors a + JOIN value v ON v.id = a.vid + JOIN value rv ON rv.id = a.root_id + JOIN node n ON n.id = rv.node_id + WHERE n.type = 'root' + """; + case SQLITE -> """ + WITH RECURSIVE ancestors(vid, root_id) AS ( + SELECT v.id, ve.parent_id + FROM value v JOIN value_edge ve ON v.id = ve.child_id + WHERE v.node_id = :nodeId + UNION ALL + SELECT a.vid, ve.parent_id + FROM ancestors a JOIN value_edge ve ON a.root_id = ve.child_id + ) + SELECT a.root_id, CAST(v.data AS TEXT) + FROM ancestors a + JOIN value v ON v.id = a.vid + JOIN value rv ON rv.id = a.root_id + JOIN node n ON n.id = rv.node_id + WHERE n.type = 'root' + GROUP BY a.root_id + """; + }; + + NodeEntity rangeNode = NodeEntity.findById(rangeNodeId); + NodeEntity domainNode = NodeEntity.findById(domainNodeId); + + // Query 1: range values with their root IDs (~30ms) + // Use putIfAbsent to deduplicate -- the recursive CTE can find multiple + // paths to the same root through shared edges (fan-out in the DAG). + List rangeRows = em.createNativeQuery(ancestorSql, Object[].class) + .setParameter("nodeId", rangeNodeId) + .getResultList(); + Map rangeByRoot = new LinkedHashMap<>(); + for (Object[] row : rangeRows) { + long rootId = ((Number) row[0]).longValue(); + if (row[1] != null && !rangeByRoot.containsKey(rootId)) { + try { rangeByRoot.put(rootId, JqValues.parse(row[1].toString())); } + catch (Exception e) { /* skip unparseable */ } + } + } + + // Query 2: domain values with their root IDs (~30ms) + List domainRows = em.createNativeQuery(ancestorSql, Object[].class) + .setParameter("nodeId", domainNodeId) + .getResultList(); + // Build domain map and sort keys by domain value + // Deduplicate by root_id (same reason as range) + record DomainEntry(long rootId, JqValue value, String sortKey) {} + Map domainByRoot = new LinkedHashMap<>(); + for (Object[] row : domainRows) { + long rootId = ((Number) row[0]).longValue(); + if (row[1] != null && !domainByRoot.containsKey(rootId)) { + try { + JqValue val = JqValues.parse(row[1].toString()); + domainByRoot.put(rootId, new DomainEntry(rootId, val, row[1].toString())); + } catch (Exception e) { /* skip unparseable */ } + } + } + List domainEntries = new ArrayList<>(domainByRoot.values()); + // Sort by domain value (text sort -- handles both numeric and timestamp strings) + domainEntries.sort((a, b) -> a.sortKey().compareTo(b.sortKey())); + + // Step 2: Fingerprint values (optional, ~30ms) + Map fpByRoot = Collections.emptyMap(); + NodeEntity fpNode = null; + if (fingerprintNodeId != null && !rangeByRoot.isEmpty()) { + fpNode = NodeEntity.findById(fingerprintNodeId); + List fpRows = em.createNativeQuery(ancestorSql, Object[].class) + .setParameter("nodeId", fingerprintNodeId) + .getResultList(); + fpByRoot = new HashMap<>(); + for (Object[] row : fpRows) { + long rootId = ((Number) row[0]).longValue(); + if (row[1] != null && !fpByRoot.containsKey(rootId)) { + try { fpByRoot.put(rootId, JqValues.parse(row[1].toString())); } + catch (Exception e) { /* skip */ } + } + } + } + + // Step 3: Join in Java by root_id, ordered by domain + List result = new ArrayList<>(); + for (DomainEntry de : domainEntries) { + JqValue rangeVal = rangeByRoot.get(de.rootId()); + if (rangeVal == null) continue; // no range value for this root + + io.hyperfoil.tools.jjq.value.JqObject.Builder builder = io.hyperfoil.tools.jjq.value.JqObject.builder(); + builder.put(rangeNode.name, rangeVal); + builder.put(domainNode.name, de.value()); + if (fpNode != null) { + JqValue fpVal = fpByRoot.get(de.rootId()); + if (fpVal != null) builder.put(fpNode.name, fpVal); + } + result.add(builder.build()); + } + return result; + } + @Override @Transactional public List getGroupedValues(Long nodeId){