Skip to content

Replace history id by retry hash - #903

Open
epszaw wants to merge 1 commit into
mainfrom
new-identities-support
Open

Replace history id by retry hash#903
epszaw wants to merge 1 commit into
mainfrom
new-identities-support

Conversation

@epszaw

@epszaw epszaw commented Aug 31, 2026

Copy link
Copy Markdown
Member

Context

Checklist

@github-actions

github-actions Bot commented Aug 31, 2026

Copy link
Copy Markdown

Allure Report Summary

Name Duration Stats New Flaky Retry Report
My Dashboard 38m 17s Passed tests 10208   Skipped tests 15   Unknown tests 15 24 0 35 View
Allure 3 Report 38m 17s Passed tests 10208   Skipped tests 15   Unknown tests 15 24 0 35 View
Allure 3 Report (plugin-classic) 38m 17s Passed tests 10208   Skipped tests 15   Unknown tests 15 24 0 35 View
Allure 3 Report (plugin-allure2) 38m 17s Passed tests 10208   Skipped tests 15   Unknown tests 15 24 0 35 View
Allure Report 0s 0 0 0

@github-actions

github-actions Bot commented Aug 31, 2026

Copy link
Copy Markdown

Allure perf metrics

Generated at: 2026-09-03T14:59:39.202Z

Phase Count Total Avg Min Max
restoreState.total 1 2941.5 ms 2941.5 ms 2941.5 ms 2941.5 ms
restoreState.dump 3 2940.9 ms 980.3 ms 969.8 ms 993.7 ms
restoreState.attachments 3 2256.4 ms 752.1 ms 722.0 ms 782.5 ms
restoreState.storeRestore 3 196.1 ms 65.4 ms 57.7 ms 72.1 ms
generate.total 1 18223.0 ms 18223.0 ms 18223.0 ms 18223.0 ms
generate.plugins.done 1 17589.6 ms 17589.6 ms 17589.6 ms 17589.6 ms
publish.upload.total 1 464444.1 ms 464444.1 ms 464444.1 ms 464444.1 ms
summary.generate 2 21.7 ms 10.9 ms 10.5 ms 11.3 ms
generate.plugin.done.agent 1 0.1 ms 0.1 ms 0.1 ms 0.1 ms
generate.plugin.done.allure2 1 4083.0 ms 4083.0 ms 4083.0 ms 4083.0 ms
generate.plugin.done.awesome 1 6752.9 ms 6752.9 ms 6752.9 ms 6752.9 ms
generate.plugin.done.classic 1 5607.1 ms 5607.1 ms 5607.1 ms 5607.1 ms
generate.plugin.done.dashboard 1 1141.5 ms 1141.5 ms 1141.5 ms 1141.5 ms
generate.plugin.done.log 1 3.8 ms 3.8 ms 3.8 ms 3.8 ms
generate.plugin.done.testops 1 0.3 ms 0.3 ms 0.3 ms 0.3 ms
publish.upload.plugin.allure2 1 140783.1 ms 140783.1 ms 140783.1 ms 140783.1 ms
publish.upload.plugin.awesome 1 158055.6 ms 158055.6 ms 158055.6 ms 158055.6 ms
publish.upload.plugin.classic 1 142464.7 ms 142464.7 ms 142464.7 ms 142464.7 ms
publish.upload.plugin.dashboard 1 3930.3 ms 3930.3 ms 3930.3 ms 3930.3 ms

Artifacts: allure-perf-metrics

@epszaw
epszaw force-pushed the new-identities-support branch from 0140551 to 82cfc14 Compare August 31, 2026 16:32
const testCase: TestCase = {
id,
allureId,
id: testCaseHash,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The spec describes _fallbackTestCaseId as a migration path when an adapter changes its test id: current testCaseHash may be new, but fallback hash should let us find the old test case/history. In this PR TestCase.id is set to the current testCaseHash, so when the adapter id changes, the product-level test case id changes too. Where do we use _fallbackTestCaseId to attach the new hash back to the old test case/history, or is that intentionally deferred?

}

return selectHistoryTestResults(this.#historyPoints, historyIdCandidates);
return selectHistoryTestResults(this.#historyPoints, [tr.retryHash]);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we clarify how _fallbackTestCaseId is supposed to affect history lookup? Right now history is selected only by the current retryHash, so if an adapter changes test id from A to B and sends _fallbackTestCaseId=A, the old history under A still won’t be found. Is fallback-based history migration intentionally deferred to a later catalog/index change?

const name = raw.name || "Unknown test";
const testCase = processTestCase(stateData, raw);
const parameters = convertParameters(raw.parameters);
const testCaseHash = calculateTestCaseHash(raw.testId, raw.fullName);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Current allure-js adapters often emit an already-hashed testCaseId. With this PR we always compute testCaseHash = md5(raw.testId), so old allure-js output becomes double-hashed, while future canonical output likely won’t. Can we add old-vs-new allure-js fixtures or document the accepted mixed-version behavior?

const indexTestResultsByHistoryId = await requiredEntryData(
AllureStoreDumpFiles.IndexTestResultsByHistoryId,
);
const indexTestResultsByRetryHash =

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we clarify what compatibility this legacy history-id read gives us? Since restore rebuilds indexes from test results with the new retryHash rules, I’m not sure old history-id keys are actually migrated here?


const { name } = parameter;

if (typeof name !== "string" || name.length === 0 || parameter.excluded === true) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We calculate parametersHash from raw params on first ingest, but after restore we calculate it from converted TestResult.parameters. Are those guaranteed to produce the same hash for null, hidden/masked, excluded, malformed, and duplicate params? A dump/restore regression test for these cases would help.

@epszaw
epszaw force-pushed the new-identities-support branch from 82cfc14 to f739a5d Compare September 3, 2026 14:18
@andrcuns

andrcuns commented Sep 4, 2026

Copy link
Copy Markdown

I went over the code but with a very limited experience on working with this codebase can't really say I could thoroughly reviewed it, but once concern that seems to arise if I followed the change correctly, is history continuity after this change.

In utils/history.ts we basically reassign old historyId as retryHash which will be used for lookups if I followed the code correctly. Old historyId did not take environment in to account which new retryHash does, so it might end up creating a mismatch?

I believe similar issue could be with ALLURE_ID? It was used for old identity but not for new one?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants