RHIDP-14421: Implement new E2E test scenarios for Scorecard in plugins by imykhno · Pull Request #4631 · redhat-developer/rhdh-plugins

imykhno · 2026-09-08T19:53:52Z

Hey, I just made a Pull Request!

In this PR was added new e2e tests for scorecard plugin to increase test coverage exists for supported plugin functionality.

This PR for: RHIDP-14421

✔️ Checklist

A changeset describing the change and affected packages. (more info)
Added or Updated documentation
Tests for new functionality and regression tests for bug fixes
Screenshots attached (for UI changes)

…verage Signed-off-by: Ihor Mykhno imykhno@redhat.com Assisted-By: Cursor <cursoragent@cursor.com>

rhdh-gh-app · 2026-09-08T19:54:18Z

Missing Changesets

The following package(s) are changed by this PR but do not have a changeset:

@red-hat-developer-hub/backstage-plugin-scorecard

See CONTRIBUTING.md for more information about how to add changesets.

Changed Packages

Package Name	Package Path	Changeset Bump	Current Version
app-legacy	workspaces/scorecard/packages/app-legacy	none	`v0.0.0`
@red-hat-developer-hub/backstage-plugin-scorecard	workspaces/scorecard/plugins/scorecard	none	`v4.2.0`

rhdh-qodo-merge · 2026-09-08T19:54:47Z

PR Summary by Qodo

Expand Scorecard E2E coverage for license KPI and drill-downs

🧪 Tests 🕐 20-40 Minutes

AI Description

• Covers the LICENSE-file KPI card, thresholds, and drill-down navigation.
• Verifies weighted KPI entity tables and catalog entity navigation.
• Validates calculation-error warning icons, tooltips, and no-data mocks.

Diagram

sequenceDiagram
  actor Test as Playwright Tests
  participant Helper as Widget Helpers
  participant Mock as API Mocks
  participant UI as Scorecard UI
  participant Drill as Drill-down POM
  participant Catalog as Catalog Route
  Test->>Mock: Register KPI fixtures
  Test->>Helper: Add license widget
  Helper->>UI: Configure homepage
  UI->>Mock: Request aggregation
  Mock-->>UI: Return KPI response
  Test->>Drill: Validate drill-down
  Drill->>UI: Inspect entities and warnings
  Test->>Drill: Click entity link
  Drill->>Catalog: Navigate to entity

High-Level Assessment

The existing mock-driven Playwright suite and page-object abstractions are the appropriate approach because they provide deterministic coverage of complete user flows. A fully data-driven KPI suite could reduce repetition but would obscure scenario-specific assertions, while live backend integration would increase flakiness.

Files changed (7) +313 / -15

Tests (7) +313 / -15

aggregations.tsRegister the LICENSE-file aggregation KPI +7/-0

Register the LICENSE-file aggregation KPI
• Adds the LICENSE-file KPI identifier, widget title, and metric metadata used when configuring and locating its homepage card.
workspaces/scorecard/packages/app-legacy/e2e-tests/constants/aggregations.ts

routes.tsAdd the LICENSE KPI aggregation route +2/-0

Add the LICENSE KPI aggregation route
• Defines the mocked API route for requests targeting the licenseFileExistsKpi aggregation.
workspaces/scorecard/packages/app-legacy/e2e-tests/constants/routes.ts

ScorecardDrillDownPage.tsExtend drill-down assertions and entity navigation +61/-11

Extend drill-down assertions and entity navigation
• Supports the filecheck license metric and introduces reusable warning-icon and tooltip assertions. It also locates health links within specific cards and verifies catalog navigation from entity table links.
workspaces/scorecard/packages/app-legacy/e2e-tests/pages/ScorecardDrillDownPage.ts

scorecard.test.tsCover license KPI and expanded drill-down scenarios +114/-0

Cover license KPI and expanded drill-down scenarios
• Adds homepage coverage for the LICENSE-file KPI's content, boolean threshold buckets, and drill-down link. It also tests weighted KPI entities, catalog navigation, and calculation-error warning behavior.
workspaces/scorecard/packages/app-legacy/e2e-tests/scorecard.test.ts

homepageWidgetUtils.tsAllow selective aggregated widget setup +10/-3

Allow selective aggregated widget setup
• Parameterizes aggregated widget creation so callers can provide a selected widget map while preserving all configured widgets as the default.
workspaces/scorecard/packages/app-legacy/e2e-tests/utils/homepageWidgetUtils.ts

mockHomepageAggregations.tsMock LICENSE KPI metadata and empty results +15/-1

Mock LICENSE KPI metadata and empty results
• Returns license KPI metadata and no-data aggregation responses from the homepage mock router. Metadata matching is narrowed to the explicit /metadata path to avoid accidental matches.
workspaces/scorecard/packages/app-legacy/e2e-tests/utils/mockHomepageAggregations.ts

scorecardResponseUtils.tsAdd license and drill-down response fixtures +104/-0

Add license and drill-down response fixtures
• Introduces LICENSE-file KPI metadata, populated and empty boolean aggregation responses, a partial calculation-error response, and weighted KPI entity data. These fixtures support deterministic homepage and drill-down scenarios.
workspaces/scorecard/packages/app-legacy/e2e-tests/utils/scorecardResponseUtils.ts

fullsend-ai-review · 2026-09-08T19:55:18Z

🤖 Finished Review · ✅ Success · Started 7:55 PM UTC · Completed 8:12 PM UTC

Commit: 188e3b2 · View workflow run →

Runtime: claude · Model: opus → claude-opus-4-6 · Cost: $6.91

rhdh-qodo-merge · 2026-09-08T19:57:21Z

Code Review by Qodo

🐞 Bugs (1) 📘 Rule violations (3) 🔗 Cross-repo conflicts (0) 📜 Skill insights (0)

1. Accessibility regressions go undetected 📘 Rule violation ☑ Accessibility

Description

The new filecheck scorecard scenarios render and interact with the card without accepting testInfo
or invoking the existing runAccessibilityTests helper. If the boolean buckets or drill-down
control introduce an accessibility violation, no tagged scan runs and no diagnostic result is
attached to the test report.

Code

workspaces/scorecard/packages/app-legacy/e2e-tests/scorecard.test.ts[492]
+      test('Verify title and description', async () => {

Relevance

●●● Strong
The repository requires attached axe scans for E2E accessibility coverage, and these new scenarios
omit the shared helper entirely.
ⓘ Recommendations generated based on similar findings in past PRs

Evidence
Compliance rule 2654 requires axe-core checks with WCAG tags and attached results. The complete
added filecheck test group contains no accessibility-helper invocation, although the suite's shared
helper implements the required scan and attachment behavior.
Rule 2654: E2E accessibility checks must use axe-core with WCAG tags and attach results to TestInfo
workspaces/scorecard/packages/app-legacy/e2e-tests/scorecard.test.ts[477-518]
workspaces/scorecard/packages/app-legacy/e2e-tests/utils/accessibility.ts[20-53]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The new filecheck KPI scenarios do not perform the required axe-core accessibility scan.

## Issue Context
The existing `runAccessibilityTests` helper already configures WCAG tags, filters the documented false positive, attaches results to `TestInfo`, and fails on remaining violations.

## Fix Focus Areas
- workspaces/scorecard/packages/app-legacy/e2e-tests/scorecard.test.ts[492-518]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

2. Localized runs still rely on English 📘 Rule violation ⚙ Maintainability

Description

existLabel and missingLabel fall back to literal English strings, while the new widget title
used by addCard is also stored as English text. In a non-English project, the threshold assertions
can accept untranslated labels and the widget-picker selector can fail to locate the localized
button.

Code

workspaces/scorecard/packages/app-legacy/e2e-tests/scorecard.test.ts[R501-502]
+        const existLabel = translations.thresholds.exist ?? 'Exist';
+        const missingLabel = translations.thresholds.missing ?? 'Missing';

Relevance

●●● Strong
Recent scorecard reviews accepted translation-based assertions and rejected hardcoded or
non-localized test conventions.
PR-#4196
PR-#3245

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
Compliance rule 2596 prohibits literal user-facing strings in E2E selectors and assertions. The
added threshold test supplies Exist and Missing as English fallbacks, and the added widget title
flows through the homepage helper into a button-name selector.
Rule 2596: E2E tests must use translation keys instead of hardcoded UI strings
workspaces/scorecard/packages/app-legacy/e2e-tests/scorecard.test.ts[500-506]
workspaces/scorecard/packages/app-legacy/e2e-tests/constants/aggregations.ts[30-33]
workspaces/scorecard/packages/app-legacy/e2e-tests/utils/homepageWidgetUtils.ts[49-54]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The new filecheck scenarios rely on hardcoded English strings for threshold assertions and widget selection.

## Issue Context
Load these labels through the existing scorecard translation utilities so localized projects exercise their actual UI text without English fallbacks.

## Fix Focus Areas
- workspaces/scorecard/packages/app-legacy/e2e-tests/scorecard.test.ts[500-506]
- workspaces/scorecard/packages/app-legacy/e2e-tests/constants/aggregations.ts[30-33]
- workspaces/scorecard/packages/app-legacy/e2e-tests/constants/aggregations.ts[62-65]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

3. License widget errors escape the test 🐞 Bug ≡ Correctness

Description

AGGREGATED_CARDS_METRIC_IDS adds the license widget to every default-widget loop, but
mockAllDefaultHomepageAggregationsSuccess does not register its aggregation route. When the
management test adds all defaults, that request reaches the live backend or an error response, and
the visibility-only assertion still passes because error panels retain the card test ID.

Code

workspaces/scorecard/packages/app-legacy/e2e-tests/constants/aggregations.ts[23]
+  licenseFileExistsKpi: 'licenseFileExistsKpi',

Relevance

●●● Strong
Accepted precedent fixes incomplete aggregation mocks in the same scorecard E2E helpers; this
omission can produce false-positive visibility tests.
PR-#3245

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The default widget helper iterates all IDs, so the newly added license ID is automatically included.
The success-mock helper covers the other five routes but omits the dedicated license route;
meanwhile, the management test checks only card visibility, and the production component assigns the
same card test ID to its error panel.
workspaces/scorecard/packages/app-legacy/e2e-tests/constants/aggregations.ts[17-24]
workspaces/scorecard/packages/app-legacy/e2e-tests/utils/homepageWidgetUtils.ts[41-57]
workspaces/scorecard/packages/app-legacy/e2e-tests/utils/mockHomepageAggregations.ts[159-192]
workspaces/scorecard/packages/app-legacy/e2e-tests/scorecard.test.ts[457-474]
workspaces/scorecard/packages/app-legacy/e2e-tests/pages/HomePage.ts[83-93]
workspaces/scorecard/plugins/scorecard/src/components/ScorecardHomepageSection/ScorecardHomepageCard.tsx[65-79]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The license KPI is now included in all default homepage widgets, but the helper that mocks successful responses for those widgets does not mock the license aggregation endpoint. Add the license success response to that helper so the management test cannot pass with an error card or depend on a live backend response.

## Issue Context
Use `ScorecardRoutes.LICENSE_FILE_EXISTS_KPI_AGGREGATION_ROUTE` with `licenseFileExistsAggregatedResponse`, following the existing mock registrations. Ensure the management test verifies the license card received successful content rather than only checking its container visibility.

## Fix Focus Areas
- workspaces/scorecard/packages/app-legacy/e2e-tests/utils/mockHomepageAggregations.ts[163-192]
- workspaces/scorecard/packages/app-legacy/e2e-tests/scorecard.test.ts[457-474]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

4. Markup changes break drill-down tests 📘 Rule violation ⚙ Maintainability

Description

getEntitiesTableCalculationWarningIcon selects any nested svg, and clickEntityLink chains
tbody with a CSS attribute selector instead of querying accessible roles. A harmless change to the
heading icon or table structure can make the tests hover the wrong element or stop finding the
entity link even when users can still navigate.

Code

workspaces/scorecard/packages/app-legacy/e2e-tests/pages/ScorecardDrillDownPage.ts[R258-260]
+    const link = entitiesTable
+      .locator('tbody')
+      .locator(`a[href*="/catalog/default/component/${slug}"]`)

Relevance

● Weak
Recent scorecard precedent rejected similar raw-locator and selector-hardening requests as
unnecessary maintainability changes.
PR-#4196
PR-#3245

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
Compliance rule 2616 requires role-based or translated-text queries instead of CSS and UI-structure
selectors. The added helpers identify the warning through svg and the entity link through tbody
plus a[href*=...].
Rule 2616: E2E selectors must use role-based or translated-text queries instead of class names
workspaces/scorecard/packages/app-legacy/e2e-tests/pages/ScorecardDrillDownPage.ts[178-179]
workspaces/scorecard/packages/app-legacy/e2e-tests/pages/ScorecardDrillDownPage.ts[255-270]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The new drill-down helpers depend on SVG, table-body, and CSS attribute structure rather than accessible roles or translated names.

## Issue Context
Use role-based queries for the warning control and entity link, adding stable accessible names in the application if necessary.

## Fix Focus Areas
- workspaces/scorecard/packages/app-legacy/e2e-tests/pages/ScorecardDrillDownPage.ts[178-179]
- workspaces/scorecard/packages/app-legacy/e2e-tests/pages/ScorecardDrillDownPage.ts[255-270]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Context sources

⚠️ Tickets: not configured — ticket URL found in PR but could not be fetched — check ticket provider credentials

✅ Compliance rules (platform): 11 rules

✅ Cross-repo context — repo relationships

Review mode: ⚖️ Balanced: This adds and modifies substantial E2E test logic across multiple helpers and page-object paths, with enough independent behavior to warrant a complete single-pass review but not the density or production-risk required for extended.

Tip of the day

💡 Did you know, you can ask Qodo to dismiss a finding you disagree with, with your reason on record

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

rhdh-qodo-merge · 2026-09-08T19:57:34Z

Important

The /generate_labels command by Qodo is sunsetting on the 1st of October 2026 and will no longer be available. We recommend switching to the latest Qodo review capabilities. Learn more

codecov · 2026-09-08T19:59:34Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 62.39%. Comparing base (0e772a4) to head (b50688c).
⚠️ Report is 64 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #4631   +/-   ##
=======================================
  Coverage   62.39%   62.39%           
=======================================
  Files        2607     2608    +1     
  Lines      104798   104828   +30     
  Branches    29477    29500   +23     
=======================================
+ Hits        65386    65407   +21     
- Misses      37551    37561   +10     
+ Partials     1861     1860    -1

Flag	Coverage Δ		*Carryforward flag
adoption-insights	`84.77% <ø> (ø)`		Carriedforward from 188e3b2
ai-integrations	`78.80% <ø> (ø)`		Carriedforward from 188e3b2
app-defaults	`56.22% <ø> (ø)`		Carriedforward from 188e3b2
augment	`46.67% <ø> (ø)`		Carriedforward from 188e3b2
boost	`80.51% <ø> (ø)`		Carriedforward from 188e3b2
bulk-import	`73.12% <ø> (ø)`		Carriedforward from 188e3b2
cost-management	`13.35% <ø> (ø)`		Carriedforward from 188e3b2
dcm	`73.47% <ø> (ø)`		Carriedforward from 188e3b2
e2e-adoption-insights	`60.00% <ø> (ø)`		Carriedforward from 188e3b2
e2e-extensions	`62.31% <ø> (ø)`		Carriedforward from 188e3b2
e2e-global-header	`50.35% <ø> (ø)`		Carriedforward from 188e3b2
e2e-homepage	`61.11% <ø> (ø)`		Carriedforward from 188e3b2
e2e-intelligent-assistant	`47.04% <ø> (ø)`		Carriedforward from 188e3b2
e2e-orchestrator	`49.52% <ø> (ø)`		Carriedforward from 188e3b2
e2e-orchestrator-plugin	`49.51% <ø> (ø)`		Carriedforward from 188e3b2
e2e-quickstart	`55.21% <ø> (ø)`		Carriedforward from 188e3b2
e2e-scorecard	`50.10% <ø> (ø)`		Carriedforward from 188e3b2
e2e-theme	`16.36% <ø> (ø)`		Carriedforward from 188e3b2
extensions	`57.37% <ø> (ø)`		Carriedforward from 188e3b2
global-floating-action-button	`71.18% <ø> (ø)`		Carriedforward from 188e3b2
global-header	`68.09% <ø> (ø)`		Carriedforward from 188e3b2
homepage	`48.39% <ø> (ø)`		Carriedforward from 188e3b2
install-dynamic-plugins	`71.31% <ø> (ø)`		Carriedforward from 188e3b2
intelligent-assistant	`76.45% <ø> (ø)`		Carriedforward from 188e3b2
konflux	`91.98% <ø> (ø)`		Carriedforward from 188e3b2
lightspeed	`69.02% <ø> (ø)`		Carriedforward from 188e3b2
mcp-integrations	`84.46% <ø> (ø)`		Carriedforward from 188e3b2
orchestrator	`71.13% <ø> (ø)`		Carriedforward from 188e3b2
quickstart	`63.74% <ø> (ø)`		Carriedforward from 188e3b2
sandbox	`79.56% <ø> (ø)`		Carriedforward from 188e3b2
scorecard	`87.98% <ø> (+0.02%)`	⬆️
theme	`87.91% <ø> (ø)`		Carriedforward from 188e3b2
translations	`5.12% <ø> (ø)`		Carriedforward from 188e3b2
x2a	`77.18% <ø> (ø)`		Carriedforward from 188e3b2

*This pull request uses carry forward flags. Click here to find out more.

Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0e772a4...b50688c. Read the comment docs.

🚀 New features to boost your workflow:

📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

fullsend-ai-review · 2026-09-08T20:12:18Z

Review

Findings

Medium

[logic-error] workspaces/scorecard/packages/app-legacy/e2e-tests/utils/mockHomepageAggregations.ts:149 — mockAllDefaultHomepageAggregationsSuccess is not updated to mock the new LICENSE_FILE_EXISTS_KPI_AGGREGATION_ROUTE. After this PR, AGGREGATED_CARDS_METRIC_IDS includes licenseFileExistsKpi (6 entries), and addAggregatedScorecardWidgets called without explicit widgetIds adds all 6 widgets. mockAggregationNoDataFound was updated with a handler for licenseFileExistsKpi, but mockAllDefaultHomepageAggregationsSuccess was not, creating an inconsistency. The function's JSDoc ("Mocks all default homepage scorecard aggregation KPI endpoints used by addAggregatedScorecardWidgets") is no longer accurate.
Remediation: Add a mockApiResponse call for the license file exists KPI aggregation route with licenseFileExistsAggregatedResponse inside mockAllDefaultHomepageAggregationsSuccess, matching the pattern of the other 5 routes.

Low

[edge-case] workspaces/scorecard/packages/app-legacy/e2e-tests/pages/ScorecardDrillDownPage.ts:29 — The MetricId type is expanded to include 'filecheck.license', but scorecardMessages.metric has a 'filecheck' key (with {{name}} template), not a 'filecheck.license' key. Methods like expectPageTitle, expectDrillDownCardSnapshot, and others access this.translations.metric[metricId].title without guarding against undefined, so passing 'filecheck.license' without the optional customTitle parameter would throw TypeError. Current callers supply customTitle, but the type signature does not enforce this constraint, making it a latent runtime error for future callers.

Previous run

Review

Findings

Medium

[test adequacy] workspaces/scorecard/packages/app-legacy/e2e-tests/utils/mockHomepageAggregations.ts:149 — mockAllDefaultHomepageAggregationsSuccess is not updated to mock the new LICENSE_FILE_EXISTS_KPI_AGGREGATION_ROUTE. After this PR, AGGREGATED_CARDS_METRIC_IDS includes licenseFileExistsKpi, and the "Manage scorecards on Home page" test (scorecard.test.ts ~line 454) calls addAggregatedScorecardWidgets(homePage) without a widgetIds argument, so the default parameter resolves to the full AGGREGATED_CARDS_METRIC_IDS (now 6 entries). The licenseFileExistsKpi card will issue an unmocked API call to /api/scorecard/aggregations/licenseFileExistsKpi, which could cause test flakiness or timing issues in CI since the dev-server response is unpredictable.
Remediation: Add a mockApiResponse call for ScorecardRoutes.LICENSE_FILE_EXISTS_KPI_AGGREGATION_ROUTE with licenseFileExistsAggregatedResponse inside mockAllDefaultHomepageAggregationsSuccess, consistent with how the other five aggregation routes are mocked.

Low

[unused import] workspaces/scorecard/packages/app-legacy/e2e-tests/pages/ScorecardDrillDownPage.ts:26 — getHomepageEntityCalculationHealthText is imported but never referenced in this file. This may trigger lint/build failures if the project enforces no-unused-imports rules.
Remediation: Remove the unused import.
[type safety] workspaces/scorecard/packages/app-legacy/e2e-tests/pages/ScorecardDrillDownPage.ts:29 — Adding 'filecheck.license' to the MetricId union type permits unsafe future usage. The translations object has a 'filecheck' key (with {{name}} template interpolation) but no 'filecheck.license' key. Calling methods like expectPageTitle with 'filecheck.license' without providing the optional customTitle override would cause a runtime TypeError. Current PR tests are safe (they always pass overrides), but the type contract allows mistakes.

Previous run (2)

Review — comment

PR #4631 adds E2E test scenarios for the Scorecard plugin's license file exists KPI, calculation error warning icons/tooltips, weighted KPI drill-down entities table, and entity link navigation. The PR also refactors ScorecardDrillDownPage page object helpers and fixes an incidental bug in mock route matching.

This is a well-structured, test-only PR that follows existing patterns. One medium-severity finding and several low-severity suggestions are noted below.

Findings

Medium

1. Missing mock route in mockAllDefaultHomepageAggregationsSuccess — mockHomepageAggregations.ts

Adding licenseFileExistsKpi to AGGREGATED_CARDS_METRIC_IDS means addAggregatedScorecardWidgets(homePage) now adds 6 widgets (up from 5). However, mockAllDefaultHomepageAggregationsSuccess was not updated to mock the new LICENSE_FILE_EXISTS_KPI_AGGREGATION_ROUTE. The existing "Manage scorecards on Home page" test (line 454 of scorecard.test.ts) calls this function, then iterates over all AGGREGATED_CARDS_METRIC_IDS values asserting card visibility. The license KPI card's API request will bypass mocking, making this test non-deterministic or causing it to fail.

Remediation: Add await mockApiResponse(page, ScorecardRoutes.LICENSE_FILE_EXISTS_KPI_AGGREGATION_ROUTE, licenseFileExistsAggregatedResponse) to mockAllDefaultHomepageAggregationsSuccess and import the response fixture.

Low

2. Broadened SVG selector reduces test precision — ScorecardDrillDownPage.ts:185

getEntitiesTableCalculationWarningIcon() uses .locator('svg') which matches any SVG in the heading. The previous implementation used 'svg.MuiSvgIcon-colorWarning' which verified both element type and warning styling. The broadened selector means expectDrillDownCalculationErrorWarningIcon() would pass even if the SVG is not a warning icon.

3. Unsafe type assertion in addAggregatedScorecardWidgets — homepageWidgetUtils.ts:47

The instanceId as keyof typeof AGGREGATED_CARDS_WIDGET_TITLES cast silences TypeScript but is unsafe: if a future caller passes widgetIds with keys not in AGGREGATED_CARDS_WIDGET_TITLES, the lookup yields undefined. Consider constraining the parameter type or adding a runtime guard.

4. clickEntityLink embeds assertion — ScorecardDrillDownPage.ts:253

This action method includes a toHaveURL assertion, breaking the established click*/expect* separation pattern in the page object (cf. clickNextPage, clickPreviousPage, clickDrillDownLink which are pure actions).

5. verifySomeEntitiesNotReportingTooltip refactored but unused — ScorecardDrillDownPage.ts:193

The method was refactored with a new signature (metricId, options) and scenario-specific defaults (healthy='8', total='10'), but no test in this PR or the existing suite calls it. Consider deferring this change until a test exercises it, or using neutral defaults consistent with other methods.

6. Incidental bug fix undocumented — mockHomepageAggregations.ts:88

The url.includes('metadata') → url.includes('/metadata') change is a correct and valuable bug fix, but it's incidental to the stated purpose and not mentioned in the PR description. Noting it aids reviewer traceability.

fullsend-ai-review · 2026-09-09T12:31:07Z

🤖 Review · ⚠️ Cancelled · Started 12:31 PM UTC · Ended 12:35 PM UTC

Commit: e9563ba · View workflow run →

fullsend-ai-review · 2026-09-09T12:36:13Z

🤖 Review · ❌ Terminated · Started 12:36 PM UTC · Ended 12:56 PM UTC

Commit: a3b937e · View workflow run →

fullsend-ai-review · 2026-09-09T12:56:23Z

🤖 Finished Review · ✅ Success · Started 12:36 PM UTC · Completed 12:56 PM UTC

Commit: a3b937e · View workflow run →

Runtime: claude · Model: opus → claude-opus-4-6 · Cost: $4.50

… ids, and page object refactor Signed-off-by: Ihor Mykhno imykhno@redhat.com Assisted-By: Cursor <cursoragent@cursor.com>

fullsend-ai-review · 2026-09-09T14:45:17Z

🤖 Review · ❌ Terminated · Started 2:45 PM UTC · Ended 2:47 PM UTC

Commit: b50688c · View workflow run →

sonarqubecloud · 2026-09-09T14:46:06Z

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

fullsend-ai-review · 2026-09-09T14:47:17Z

🤖 Finished Review · ❌ Failure (validation failed after 2 iteration(s)) · Started 2:45 PM UTC · Completed 2:47 PM UTC

Commit: b50688c · View workflow run →

Runtime: claude · Model: opus → claude-opus-5

fullsend-ai-review · 2026-09-10T08:16:49Z

🤖 Finished Review · ❌ Failure (validation failed after 2 iteration(s)) · Started 8:16 AM UTC · Completed 8:18 AM UTC

Commit: b50688c · View workflow run →

Runtime: claude · Model: opus → claude-opus-5

fullsend-ai-review · 2026-09-10T15:41:45Z

🤖 Finished Review · ✅ Success · Started 3:41 PM UTC · Completed 4:02 PM UTC

Commit: b50688c · View workflow run →

Runtime: claude · Model: opus → claude-opus-4-6 · Cost: $6.13

fullsend-ai-review

Note: The following review comments could not be posted on the diff (GitHub returned 422) and are included here instead:

workspaces/scorecard/packages/app-legacy/e2e-tests/utils/mockHomepageAggregations.ts (file-level): Line 149 · [medium] logic-error

mockAllDefaultHomepageAggregationsSuccess is not updated to mock the new LICENSE_FILE_EXISTS_KPI_AGGREGATION_ROUTE. After this PR, AGGREGATED_CARDS_METRIC_IDS includes licenseFileExistsKpi (6 entries), and addAggregatedScorecardWidgets called without explicit widgetIds adds all 6 widgets. mockAggregationNoDataFound was updated with a handler for licenseFileExistsKpi, but mockAllDefaultHomepageAggregationsSuccess was not, creating an inconsistency.

Suggested fix: Add a mockApiResponse call for the license file exists KPI aggregation route with licenseFileExistsAggregatedResponse inside mockAllDefaultHomepageAggregationsSuccess, matching the pattern of the other 5 routes.

fullsend-ai-review

See the review comment for full details.

dzemanov

Thank you!

fullsend-ai-retro · 2026-09-14T08:10:03Z

🤖 Finished Retro · ✅ Success · Started 8:10 AM UTC · Completed 8:24 AM UTC

Commit: b50688c · View workflow run →

Runtime: claude · Model: opus → claude-opus-4-6 · Effort: high · Cost: $4.72

fullsend-ai-retro · 2026-09-14T08:24:11Z

Retro: PR #4631 — E2E test scenarios for Scorecard

PR: #4631 by imykhno (human-authored with Cursor AI assistance)
Workflow: Human PR → 5 review agent runs (3 successful, 2 failed) → human approval → merge
Agents repo: fullsend-ai/agents @ 4bbe4f5

Timeline

Sep 8 — PR opened with commit 188e3b2 adding E2E tests for the Scorecard LICENSE_FILE_EXISTS KPI.
Sep 8, 20:12 — First review completed ($6.91). Found 1 medium finding (missing mock in mockAllDefaultHomepageAggregationsSuccess for LICENSE_FILE_EXISTS_KPI_AGGREGATION_ROUTE) and 5 low findings. Qodo independently flagged the same mock gap.
Sep 9, 11:33 — Author replied "Fixed" to 2 low findings (SVG selector, type safety) but did not address the medium finding.
Sep 9, 12:56 — Second review on commit a3b937e ($4.50). Same medium finding persisted, 2 new low findings.
Sep 9, 14:47 — Review on commit b50688c failed: claude-opus-5 returned 403 on Vertex AI. CLI v2.1.260's model-availability probe passed but the model lacked IAM permissions. Both validation iterations exhausted at $0 cost.
Sep 10, 08:19 — Re-run failed identically (same CLI version, same opus-5 403).
Sep 10, 16:02 — Re-run succeeded on CLI v2.1.263 which detected opus-5 unavailability and fell back to opus-4-6 ($6.13). Same medium finding raised a 3rd time. Sonnet sub-agents (style-conventions, intent-coherence) also failed due to claude-sonnet-4-5@20250929 unavailability on Vertex.
Sep 14, 06:32 — dzemanov approved. The medium finding was never addressed.
Sep 14, 08:08 — PR self-merged by author.

Review quality

The review agent performed well. Its medium-severity finding about the missing mock is confirmed valid by code analysis: mockAllDefaultHomepageAggregationsSuccess mocks 5 of 6 KPI routes but omits the newly added LICENSE_FILE_EXISTS_KPI_AGGREGATION_ROUTE. The other two analogous mock functions (mockHomepageAggregationsPermissionDenied, mockAggregationNoDataFound) were correctly updated for the new KPI — only the success mock was missed. The "Manage scorecards on Home page" test calls this function and then iterates over all 6 metric IDs, meaning the license card renders without a success mock. Qodo independently found the same bug, providing strong cross-validation.

Evidence for existing issues (not filing new proposals)

Model fallback failures (opus-5 403 → no fallback in v2.1.260): Already fixed in CLI v2.1.263. Tracked by #6967, #7154, #1771. This retro provides additional evidence: same CLI v2.1.260 succeeded with fallback at 12:34 but failed at 14:46 on Sep 9, suggesting the probe passed when the endpoint was registered but IAM permissions were absent.
Persistent finding repetition (same medium finding raised 3 times without deduplication): Tracked by #2816. Prior review provenance was unverifiable-wrong-app, preventing the agent from seeing its own prior reviews.
Sonnet sub-agent unavailability (claude-sonnet-4-5@20250929 not available on Vertex): Tracked by #6922. The successful review ran with only the correctness sub-agent; style-conventions and intent-coherence were degraded to info-level notes.
Review cost from failed runs: Tracked by #6964. Two failed attempts consumed CI time but $0 in API cost. Total successful review cost: ~$17.54 across 3 passes.

Proposals filed

mockAllDefaultHomepageAggregationsSuccess missing LICENSE_FILE_EXISTS mock (in redhat-developer/rhdh-plugins)

test(scorecard): add license file exists KPI and drill-down entity co…

188e3b2

…verage Signed-off-by: Ihor Mykhno imykhno@redhat.com Assisted-By: Cursor <cursoragent@cursor.com>

imykhno requested review from a team, Eswaraiahsapram, christoph-jerolimov and its-mitesh-kumar as code owners September 8, 2026 19:53

github-actions Bot added the workspace/scorecard label Sep 8, 2026

rhdh-qodo-merge Bot added the Tests label Sep 8, 2026

This comment was marked as outdated.

Sign in to view

fullsend-ai-review Bot added the requires-manual-review Review requires human judgment label Sep 8, 2026

imykhno force-pushed the feat/add-e2e-tests-for-scorecard branch from e9563ba to a3b937e Compare September 9, 2026 12:34

This comment was marked as outdated.

Sign in to view

fullsend-ai-review Bot added requires-manual-review Review requires human judgment and removed requires-manual-review Review requires human judgment labels Sep 9, 2026

test(scorecard): improve e2e coverage with accessibility checks, test…

b50688c

… ids, and page object refactor Signed-off-by: Ihor Mykhno imykhno@redhat.com Assisted-By: Cursor <cursoragent@cursor.com>

imykhno force-pushed the feat/add-e2e-tests-for-scorecard branch from a3b937e to b50688c Compare September 9, 2026 14:43

fullsend-ai-review Bot reviewed Sep 10, 2026

View reviewed changes

Comment thread workspaces/scorecard/packages/app-legacy/e2e-tests/pages/ScorecardDrillDownPage.ts

fullsend-ai-review Bot added requires-manual-review Review requires human judgment and removed requires-manual-review Review requires human judgment labels Sep 10, 2026

dzemanov approved these changes Sep 14, 2026

View reviewed changes

imykhno merged commit e788230 into redhat-developer:main Sep 14, 2026
94 of 96 checks passed

imykhno deleted the feat/add-e2e-tests-for-scorecard branch September 14, 2026 08:08

This was referenced Sep 14, 2026

mockAllDefaultHomepageAggregationsSuccess missing LICENSE_FILE_EXISTS mock #4717

Closed

fix(#4717): add LICENSE_FILE_EXISTS mock to homepage aggregations #4718

Merged

Uh oh!

Conversation

imykhno commented Sep 8, 2026

Hey, I just made a Pull Request!

This PR for: RHIDP-14421

✔️ Checklist

Uh oh!

rhdh-gh-app Bot commented Sep 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Missing Changesets

Changed Packages

Uh oh!

rhdh-qodo-merge Bot commented Sep 8, 2026

PR Summary by Qodo

Uh oh!

fullsend-ai-review Bot commented Sep 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rhdh-qodo-merge Bot commented Sep 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code Review by Qodo

Uh oh!

rhdh-qodo-merge Bot commented Sep 8, 2026

Uh oh!

codecov Bot commented Sep 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

fullsend-ai-review Bot commented Sep 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review

Findings

Medium

Low

Review

Findings

Medium

Low

Review — comment

Findings

Medium

Low

Uh oh!

This comment was marked as outdated.

Uh oh!

This comment was marked as outdated.

Uh oh!

fullsend-ai-review Bot commented Sep 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

fullsend-ai-review Bot commented Sep 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment was marked as outdated.

Uh oh!

This comment was marked as outdated.

Uh oh!

fullsend-ai-review Bot commented Sep 9, 2026

Uh oh!

fullsend-ai-review Bot commented Sep 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sonarqubecloud Bot commented Sep 9, 2026

Quality Gate passed

Uh oh!

fullsend-ai-review Bot commented Sep 9, 2026

Uh oh!

fullsend-ai-review Bot commented Sep 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

fullsend-ai-review Bot commented Sep 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

fullsend-ai-review Bot left a comment

Choose a reason for hiding this comment

Uh oh!

fullsend-ai-review Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

rhdh-gh-app Bot commented Sep 8, 2026 •

edited

Loading

fullsend-ai-review Bot commented Sep 8, 2026 •

edited

Loading

rhdh-qodo-merge Bot commented Sep 8, 2026 •

edited

Loading

codecov Bot commented Sep 8, 2026 •

edited

Loading

fullsend-ai-review Bot commented Sep 8, 2026 •

edited

Loading

fullsend-ai-review Bot commented Sep 9, 2026 •

edited

Loading

fullsend-ai-review Bot commented Sep 9, 2026 •

edited

Loading

fullsend-ai-review Bot commented Sep 9, 2026 •

edited

Loading

fullsend-ai-review Bot commented Sep 10, 2026 •

edited

Loading

fullsend-ai-review Bot commented Sep 10, 2026 •

edited

Loading

fullsend-ai-retro Bot commented Sep 14, 2026 •

edited

Loading