Skip to content

Fix classic report ignoring category rules with messageRegex/traceRegex - #842

Open
meganemura wants to merge 1 commit into
allure-framework:mainfrom
meganemura:fix-classic-category-regex-matching
Open

Fix classic report ignoring category rules with messageRegex/traceRegex#842
meganemura wants to merge 1 commit into
allure-framework:mainfrom
meganemura:fix-classic-category-regex-matching

Conversation

@meganemura

@meganemura meganemura commented Aug 2, 2026

Copy link
Copy Markdown

Context

categories.json rules with messageRegex or traceRegex never match any
test result in the classic report — only status-only rules (including the
built-in "Product defects" / "Test defects" fallbacks) do. For example:

{ "name": "Timeout", "matchedStatuses": ["failed"], "messageRegex": ".*timed out.*" }

A failing test whose message contains "timed out" still lands under
"Product defects" instead of "Timeout" — with no warning that the rule was
never even evaluated correctly.

Root cause: generators.ts builds the match input as {message, trace, status, flaky}, but categoryMatch in categories.ts destructures
statusMessage/statusTrace from it. The names never matched, so those
fields were always undefined, and match() treats undefined as a
non-match whenever a regex is set.

message/trace is the correct naming for this plugin: ClassicTestResult
is built on @allurereport/core-api's TestResult, whose error info is a
TestError ({message, trace, actual, expected}), and generators.ts
passes error?.message/error?.trace straight through under those same
keys.

statusMessage/statusTrace isn't a typo — it's the sibling plugin
plugin-allure2's field name. plugin-allure2/src/categories.ts has
identical matching logic, but that plugin's own data model
(Allure2TestResult) defines statusMessage/statusTrace directly, and
its caller passes those same keys — so that plugin isn't affected by this
bug. The two categories.ts files are close enough to be evidently related
implementations; this one ended up carrying the other model's field names.

Fix: destructure message/trace in categoryMatch, matching both
the caller and the underlying TestError type. Added a regression test
using the exact call shape generators.ts uses; also fixed 4 existing
tests that were unintentionally exercising a call shape the real caller
never uses.

Checklist

@CLAassistant

CLAassistant commented Aug 2, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

categoryMatch() destructured statusMessage/statusTrace, but the caller
in generators.ts builds the result object with message/trace. The
mismatch left statusMessage/statusTrace always undefined, so any
category rule with messageRegex or traceRegex could never match — only
status-only rules (including the built-in Product/Test defects
fallbacks) worked, which silently masked the bug in reports.

message/trace is the correct naming here: ClassicTestResult (built on
core-api's TestResult) carries error info as TestError, whose fields
are {message, trace, actual, expected}, and generators.ts passes
error?.message/error?.trace straight through under those same keys.

statusMessage/statusTrace isn't a typo either — it's the field name
used by the sibling plugin-allure2, whose categories.ts has identical
matching logic but a different data model (Allure2TestResult defines
statusMessage/statusTrace directly). plugin-allure2's caller passes
those same keys, so it isn't affected by this bug.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@meganemura
meganemura force-pushed the fix-classic-category-regex-matching branch from 57f6c99 to a12b0f1 Compare August 4, 2026 11:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants