Skip to content
Open
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions workspaces/scorecard/.changeset/breezy-numbers-hide.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
'@red-hat-developer-hub/backstage-plugin-scorecard-backend': minor
'@red-hat-developer-hub/backstage-plugin-scorecard-common': minor
---

Entity time-series API (`GET /metrics/catalog/:kind/:namespace/:name/time-series`) now returns entity-resolved `thresholds` and per-point `thresholdEvaluation` (classified at read time against those current thresholds) so clients can render sparkline legends and chart colors without a separate snapshot call.
23 changes: 20 additions & 3 deletions workspaces/scorecard/plugins/scorecard-backend/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -309,6 +309,8 @@ curl -X GET "{{url}}/api/scorecard/metrics/catalog/component/default/my-service?

Returns daily time-series points for one metric on a catalog entity. Each point is the latest sample (`MAX(id)` among success or calculation-error rows) for that UTC calendar day. On a mixed day the later sample wins, so a later error is returned as `{ "value": null, "error": "..." }` (and clients can gap a sparkline). Days with no rows (or only null without `error_message`) are omitted. Returns `200` with `points: []` when the entity and metric are authorized but no data exists in the range.

The response also includes entity-resolved `thresholds` (provider defaults, then app-config, then entity annotation overrides) for sparkline legend rendering. Successful points include `thresholdEvaluation`: the matched threshold rule key from **read-time** evaluation of the point's `value` against those current `thresholds` (e.g. `success`, `warning`, `error`). This keeps legend keys and point classifications consistent when config changes. Calculation-error points omit `thresholdEvaluation`. When no rule matches, `thresholdEvaluation` is `null`.

#### Path Parameters

| Parameter | Type | Required | Description |
Expand Down Expand Up @@ -350,14 +352,29 @@ curl -X GET "{{url}}/api/scorecard/metrics/catalog/component/default/my-service/
"defaultVisualization": "donut"
},
"points": [
{ "value": 8, "timestamp": "2026-04-27T23:10:00.000Z" },
{
"value": 8,
"timestamp": "2026-04-27T23:10:00.000Z",
"thresholdEvaluation": "success"
},
{
"value": null,
"timestamp": "2026-04-28T16:00:00.000Z",
"error": "GitHub API 500"
},
{ "value": 7, "timestamp": "2026-04-29T22:55:00.000Z" }
]
{
"value": 25,
"timestamp": "2026-04-29T22:55:00.000Z",
"thresholdEvaluation": "warning"
}
],
"thresholds": {
"rules": [
{ "key": "success", "expression": "<10" },
{ "key": "warning", "expression": "10-50" },
{ "key": "error", "expression": ">50" }
]
}
}
```

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -532,13 +532,20 @@ describe('CatalogMetricService', () => {
defaultVisualization: provider.getMetrics()[0].defaultVisualization,
collectorIds: provider.getMetrics()[0].collectorIds,
},
thresholds: { rules: mockThresholdRules },
});
expect(
mockedDatabase.readLatestEntityMetricValuesPerUtcDay,
).toHaveBeenCalledWith(entityRef, metricId, from, to);
expect(
mockedThresholdResolver.resolveEntityThresholds,
).toHaveBeenCalledWith(
mockEntity,
expect.objectContaining({ id: metricId }),
);
});

it('should map each daily DB row to a time-series point', async () => {
it('should map each daily DB row to a time-series point with read-time thresholdEvaluation', async () => {
mockedDatabase.readLatestEntityMetricValuesPerUtcDay.mockResolvedValue([
{
id: 3,
Expand All @@ -553,9 +560,10 @@ describe('CatalogMetricService', () => {
id: 2,
catalogEntityRef: entityRef,
metricId: metricId,
value: 7,
value: 25,
timestamp: new Date('2024-01-02T12:00:00.000Z'),
errorMessage: null,
// Stale write-time status must be ignored in favor of read-time evaluation
status: 'success',
},
] as DbMetricValue[]);
Expand All @@ -568,12 +576,21 @@ describe('CatalogMetricService', () => {
);

expect(result.points).toEqual([
{ value: 9, timestamp: '2024-01-01T20:00:00.000Z' },
{ value: 7, timestamp: '2024-01-02T12:00:00.000Z' },
{
value: 9,
timestamp: '2024-01-01T20:00:00.000Z',
thresholdEvaluation: 'success',
},
{
value: 25,
timestamp: '2024-01-02T12:00:00.000Z',
thresholdEvaluation: 'warning',
},
]);
expect(result.thresholds).toEqual({ rules: mockThresholdRules });
});

it('should map calculation-error rows to null value with error', async () => {
it('should map calculation-error rows to null value with error and omit thresholdEvaluation', async () => {
mockedDatabase.readLatestEntityMetricValuesPerUtcDay.mockResolvedValue([
{
id: 1,
Expand Down Expand Up @@ -612,16 +629,80 @@ describe('CatalogMetricService', () => {
);

expect(result.points).toEqual([
{ value: 8, timestamp: '2024-01-01T10:00:00.000Z' },
{
value: 8,
timestamp: '2024-01-01T10:00:00.000Z',
thresholdEvaluation: 'success',
},
{
value: null,
timestamp: '2024-01-02T16:00:00.000Z',
error: 'GitHub API 500',
},
{ value: 7, timestamp: '2024-01-03T10:00:00.000Z' },
{
value: 7,
timestamp: '2024-01-03T10:00:00.000Z',
thresholdEvaluation: 'success',
},
]);
});

it('should evaluate thresholdEvaluation from current thresholds even when DB status is null', async () => {
mockedDatabase.readLatestEntityMetricValuesPerUtcDay.mockResolvedValue([
{
id: 1,
catalogEntityRef: entityRef,
metricId: metricId,
value: 5,
timestamp: new Date('2024-01-01T10:00:00.000Z'),
errorMessage: null,
status: null,
},
] as DbMetricValue[]);

const result = await service.getEntityMetricTimeSeries(
entityRef,
metricId,
from,
to,
);

expect(result.points).toEqual([
{
value: 5,
timestamp: '2024-01-01T10:00:00.000Z',
thresholdEvaluation: 'success',
},
]);
});

it('should fall back to resolveMetricThresholds when resolveEntityThresholds throws', async () => {
const metricThresholds = {
rules: [
{ key: 'success', expression: '<5' },
{ key: 'error', expression: '>=5' },
],
};
mockedThresholdResolver.resolveEntityThresholds.mockImplementation(() => {
throw new Error('Merge thresholds failed');
});
mockedThresholdResolver.resolveMetricThresholds.mockReturnValue(
metricThresholds,
);

const result = await service.getEntityMetricTimeSeries(
entityRef,
metricId,
from,
to,
);

expect(
mockedThresholdResolver.resolveMetricThresholds,
).toHaveBeenCalledWith(expect.objectContaining({ id: metricId }));
expect(result.thresholds).toEqual(metricThresholds);
});

it('should pass permission filter to filterAuthorizedMetrics', async () => {
await service.getEntityMetricTimeSeries(
entityRef,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@ import { isMetricCalculationError } from '../utils/metricCalculationError';
import { AggregatedMetricMapper } from './mappers';
import { DbMetricValue } from '../database/types';
import { ThresholdResolver } from '../threshold/ThresholdResolver';
import { ThresholdEvaluator } from '../threshold/ThresholdEvaluator';

type CatalogMetricServiceOptions = {
catalog: CatalogService;
Expand Down Expand Up @@ -82,6 +83,7 @@ export class CatalogMetricService {
private readonly registry: MetricProvidersRegistry;
private readonly database: DatabaseMetricValues;
private readonly thresholdResolver: ThresholdResolver;
private readonly thresholdEvaluator = new ThresholdEvaluator();

private static readonly MAX_FETCHABLE_ROWS = 10_000;
private static readonly BATCH_SIZE = 100;
Expand Down Expand Up @@ -233,6 +235,17 @@ export class CatalogMetricService {
to,
);

let thresholds: ThresholdConfig;
try {
thresholds = this.thresholdResolver.resolveEntityThresholds(

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know if we want to silently switch what thresholds are used if there is error with them?
For snapshot, frontend shows there was error with evaluating entity thresholds because thresholds are malformed.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. Updated this in the latest version to be returned as a response thresholdsError

entity,
metric,
);
} catch {
// Keep app-config / provider thresholds when entity annotation merge fails
thresholds = this.thresholdResolver.resolveMetricThresholds(metric);
}

const points: MetricTimeSeriesPoint[] = rows.map(row => {
if (isMetricCalculationError(row)) {
return {
Expand All @@ -241,9 +254,29 @@ export class CatalogMetricService {
error: row.errorMessage!,
};
}

let thresholdEvaluation: string | null = null;
if (row.value !== null) {
try {
thresholdEvaluation =

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we need to also let frontend know there were evaluation problems.

We will need to fix error handling in backend for next release I think.
Right now everywhere we use:
error?: string
We might deprecate it and use something like:
errors?: [{code: string, message: string}]

This way with codes, frontend can also do translation and it is secure.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Almost everywhere, aggregated time-series returns: errors?: [{message: string, count: num}]) so we can follow this pattern here as well, without count, so errors?: [{message: string}])

this.thresholdEvaluator.getFirstMatchingThreshold(
row.value,
metric.type,
thresholds,
) ?? null;
} catch (error) {
this.logger.warn(
`Failed to evaluate thresholds for metric '${
metric.id
}' on entity '${entityRef}': ${stringifyError(error)}`,
);
}
}

return {
value: row.value,
timestamp: row.timestamp.toISOString(),
thresholdEvaluation,
};
});

Expand All @@ -260,6 +293,7 @@ export class CatalogMetricService {
defaultVisualization: metric.defaultVisualization,
collectorIds: metric.collectorIds,
},
thresholds,
};
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -654,9 +654,24 @@ describe('createRouter', () => {
defaultVisualization: 'donut',
},
points: [
{ value: 8, timestamp: '2024-01-01T20:00:00.000Z' },
{ value: 7, timestamp: '2024-01-02T12:00:00.000Z' },
{
value: 8,
timestamp: '2024-01-01T20:00:00.000Z',
thresholdEvaluation: 'success',
},
{
value: 7,
timestamp: '2024-01-02T12:00:00.000Z',
thresholdEvaluation: 'success',
},
],
thresholds: {
rules: [
{ key: 'error', expression: '>40' },
{ key: 'warning', expression: '>20' },
{ key: 'success', expression: '<=20' },
],
},
};

const timeSeriesPath =
Expand Down
2 changes: 2 additions & 0 deletions workspaces/scorecard/plugins/scorecard-common/report.api.md
Original file line number Diff line number Diff line change
Expand Up @@ -181,6 +181,7 @@ export type MetricTimeSeriesPoint = {
value: MetricValue | null;
timestamp: string;
error?: string;
thresholdEvaluation?: string | null;
};

// @public
Expand All @@ -197,6 +198,7 @@ export type MetricTimeSeriesResponse = {
defaultVisualization?: ScorecardVisualizationType;
collectorIds?: string[];
};
thresholds: ThresholdConfig;
};

// @public (undocumented)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -138,6 +138,12 @@ export type MetricTimeSeriesPoint = {
timestamp: string;
/** Present when this point is a calculation failure */
error?: string;
/**
* Matched threshold rule key from read-time evaluation against the response
* `thresholds` (e.g., "elite", "success", "warning").
* `null` when the value could not be classified. Absent on calculation-error points.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[low] naming-convention

The new field thresholdEvaluation introduces a third name for the same domain concept. The codebase uses evaluation in ThresholdResult and status in EntityMetricDetail/DbMetricValue.

*/
thresholdEvaluation?: string | null;
};

/**
Expand All @@ -157,4 +163,9 @@ export type MetricTimeSeriesResponse = {
defaultVisualization?: ScorecardVisualizationType;
collectorIds?: string[];
};
/**
Comment thread
djanickova marked this conversation as resolved.
* Entity-resolved threshold rules (provider defaults, then app-config, then entity annotation overrides).
* Used for sparkline legend rendering and mapping `thresholdEvaluation` keys to colors.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[low] api-shape-consistency

thresholds is required on MetricTimeSeriesResponse but ThresholdConfig | undefined on MetricResult.result.thresholdResult.definition. Different contracts for the same underlying data, by design due to different error-handling strategies.

*/
thresholds: ThresholdConfig;
};
Loading