TRT-2733: Add CR BigQuery/PostgreSQL parity tooling

[mstaeble]

Summary

Adds two scripts for measuring and diagnosing Component Readiness (CR) parity between the BigQuery and PostgreSQL providers (selected per request via the dataSource query parameter). These support the ongoing work to bring the PostgreSQL CR implementation to parity with BigQuery under epic TRT-2733.

• scripts/cr-parity-check.py: samples CR API configurations against both providers and reports factual differences without attributing causes: grid status-pair mismatches, regressed_tests differences per cell, and per-provider response times. Supports --json for downstream analysis.
• scripts/cr-parity-analyze.py: consumes the --json output and drills into test_details for both providers to attribute each mismatch. Resolves grid-level cells to a representative test whose per-test status actually flips (drilling both providers), and reports every contributing factor per cell:
  • job_scope (TRT-2861)
  • ingestion_gap_bq / ingestion_gap_pg (e.g. ROSA HCP release-variant assignment, TRT-2912)
  • run_count_divergence with net BQ-PG direction (dedup, TRT-2804)
  • outcome_divergence (same runs, different pass/fail/flake)
  • pg_capability_empty (PG component drill-down returns no capability/test rows, TRT-2911)

Documentation

• scripts/README.md documents purpose, both scripts with options and examples, the typical workflow, and links to the tracked parity gaps.
• scripts/requirements.txt adds pyyaml (used by the analyzer to parse config/openshift.yaml; falls back to a built-in parser if absent).

Notes

These are read-only diagnostic tools that talk to a running Sippy API (staging or prod); they do not require direct database access. Force refresh is kept on by default when measuring parity, since cached CR results can be up to about 4 hours stale and produce spurious mismatches.

🤖 Generated with Claude Code

[openshift-merge-bot]

Pipeline controller notification
This repo is configured to use the pipeline controller. Second-stage tests will be triggered either automatically or after lgtm label is added, depending on the repository configuration. The pipeline controller will automatically detect which contexts are required and will utilize /test Prow commands to trigger the second stage.

For optional jobs, comment /test ? to see a list of all defined jobs. To trigger manually all jobs from second stage use /pipeline required command.

This repository is configured in: automatic mode

[openshift-ci-robot]

@mstaeble: This pull request references TRT-2733 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the epic to target the "5.1.0" version, but no target version was set.

Details

In response to this:

Summary

Adds two scripts for measuring and diagnosing Component Readiness (CR) parity between the BigQuery and PostgreSQL providers (selected per request via the dataSource query parameter). These support the ongoing work to bring the PostgreSQL CR implementation to parity with BigQuery under epic TRT-2733.

• scripts/cr-parity-check.py: samples CR API configurations against both providers and reports factual differences without attributing causes: grid status-pair mismatches, regressed_tests differences per cell, and per-provider response times. Supports --json for downstream analysis.
• scripts/cr-parity-analyze.py: consumes the --json output and drills into test_details for both providers to attribute each mismatch. Resolves grid-level cells to a representative test whose per-test status actually flips (drilling both providers), and reports every contributing factor per cell:
• job_scope (TRT-2861)
• ingestion_gap_bq / ingestion_gap_pg (e.g. ROSA HCP release-variant assignment, TRT-2912)
• run_count_divergence with net BQ-PG direction (dedup, TRT-2804)
• outcome_divergence (same runs, different pass/fail/flake)
• pg_capability_empty (PG component drill-down returns no capability/test rows, TRT-2911)

Documentation

• scripts/README.md documents purpose, both scripts with options and examples, the typical workflow, and links to the tracked parity gaps.
• scripts/requirements.txt adds pyyaml (used by the analyzer to parse config/openshift.yaml; falls back to a built-in parser if absent).

Notes

These are read-only diagnostic tools that talk to a running Sippy API (staging or prod); they do not require direct database access. Force refresh is kept on by default when measuring parity, since cached CR results can be up to about 4 hours stale and produce spurious mismatches.

🤖 Generated with Claude Code

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[coderabbitai]

Warning

Review limit reached

@mstaeble, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 3 minutes

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

Wait for the limit to reset, then comment @coderabbitai review or push new commits to the PR.

An organization admin can change what happens after included review limits in Billing.

How do review limits work?

CodeRabbit enforces per-developer PR review limits within each organization.

For paid Pro and Pro+ reviews, CodeRabbit uses a developer's included PR review attempts over the past 7 days to set the current hourly allowance. At typical activity levels, the full plan allowance applies. Higher sustained activity can lower the allowance until earlier attempts leave the 7-day window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Pro Plus

Run ID: dd3e0950-9e68-4f4b-9c2b-06c6270f79aa

📥 Commits

Reviewing files that changed from the base of the PR and between a22c6fb and cc73a9a.

📒 Files selected for processing (4)

• scripts/README.md
• scripts/cr-parity-analyze.py
• scripts/cr-parity-check.py
• scripts/requirements.txt

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: mstaeble

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• OWNERS [mstaeble]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[openshift-ci]

@mstaeble: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/security	cc73a9a	link	true	/test security

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.