Skip to content

Replace hardcoded Lanczos eigenvalue goldens with property-based checks#3086

Open
says1117 wants to merge 2 commits into
NVIDIA:mainfrom
says1117:fix-2519-lanczos-hardcoded-tests
Open

Replace hardcoded Lanczos eigenvalue goldens with property-based checks#3086
says1117 wants to merge 2 commits into
NVIDIA:mainfrom
says1117:fix-2519-lanczos-hardcoded-tests

Conversation

@says1117

Copy link
Copy Markdown

Fixes #2519.

The problem

The Lanczos gtests checked computed eigenvalues against hardcoded arrays (devArrMatch + CompareApprox(1e-5)). That's fragile: floating-point addition isn't associative, so cuSPARSE/cuBLAS can sum things in a different order depending on CUDA version and GPU architecture, and the last few digits of an eigenvalue shift. #3021 hit this on A100/ARM64 — 0.01367824 vs 0.01369178, a difference just outside the tolerance, even though the solve was correct.

The fix

Instead of comparing against a fixed answer, check that what came back is actually a valid eigenpair:

  • Residual: ||A*v - λv|| should be tiny. This is the same thing the solver's own tolerance config is defined to converge on, so it's not an arbitrary check - it's the solver's own success criterion.
  • Rayleigh quotient: λ ≈ (vᵀAv)/(vᵀv). Added this because the residual alone gets loose on matrices with a large norm, and can miss a small eigenvalue being slightly wrong. The Rayleigh quotient pins it down directly.
  • Unit norm: eigenvectors should have length 1.
  • Orthogonality: distinct eigenvectors shouldn't overlap. This one isn't in the old test at all - it catches a specific Lanczos failure mode where the solver returns the same eigenpair twice ("ghost duplicates"), which the residual check alone can't see.
  • Ordering: eigenvalues come back sorted ascending, which I confirmed against all 11 of the old golden arrays before deleting them.

rmat_lanczos_tests already had the matrix-vector-multiply helper needed for the residual check sitting there unused; I just started using it. lanczos_tests needed it added. Eigenvectors were already being computed by every test, just never checked - no solver changes needed.

Left the reproducibility check alone (it runs the solve twice with the same seed and diffs the results exactly - that's a determinism check, not a golden-value comparison, so it isn't affected by the cross-platform issue).

Picking the tolerance

I didn't want to just guess a number, so I instrumented the checks, ran all 11 test fixtures, and looked at what the residuals actually came out to:

fixture residual ÷ (‖A‖·ε) orthogonality ÷ ε |λ − Rayleigh| ÷ (‖A‖·ε)
double, n=100, ‖A‖=8.2 2.7 2.7 1.7
float, n=100, ‖A‖=8.2 0.7 2.5 0.9
double, n=100, ‖A‖=10.5 14.2 2.1 18.8
float, n=100, ‖A‖=10.5 8.6 0.4 4.9
float, n=4096, ‖A‖=298 4.8 18.9 2.0

Two things stood out. First, the residual scales with ‖A‖ · machine epsilon, not with the matrix size n - the small fixtures (n=100) and the big RMAT one (n=4096) land in the same range. That mattered a lot (see below). Second, orthogonality and unit-norm error are both just small multiples of epsilon.

So the tolerances ended up being:

```
matrix_tol = 500 × max(‖A‖, 1) × ε — residual, Rayleigh, ordering
unit_tol = 1000 × ε — unit norm, orthogonality
```

Every measured value sits at 0.04%–3.8% of its tolerance - comfortable margin for cross-platform noise, without being so loose the check stops meaning anything.

Making sure I didn't just weaken the test

I proved these checks can actually fail, by corrupting a solver's output on purpose and confirming the test catches it:

  • Scale every eigenvalue by 1% → caught by the residual and Rayleigh checks, on all 11 tests.
  • Zero out an eigenvector → caught immediately by the unit-norm check.
  • Copy eigenpair 0 into eigenpair 1's slot (still a perfectly valid, unit-norm, low-residual eigenpair — just a duplicate) → only the orthogonality check catches this. Everything else passes. This is exactly why that check earns its place.

Worth being honest about: my first attempt at this got it wrong. I'd included an n factor in the residual tolerance, and it turned out RmatLanczosTestF still passed even with every eigenvalue scaled by 1% - a real regression from the old test, which would've caught that instantly. Digging into the data above is what showed the residual doesn't actually scale with n, and once I fixed that (and added the Rayleigh check), the same corruption test failed cleanly across the board, including on the hardest case - the smallest eigenvalue in the RMAT set, where a 1% error is only 0.08 in absolute terms.

Verification

Built and ran on an RTX 2060 SUPER (sm_75), CUDA 13.3, WSL2:

```
$ cmake --build cpp/build -j2 --target SOLVERS_TEST
[2/2] Linking CXX executable gtests/SOLVERS_TEST

$ ./cpp/build/gtests/SOLVERS_TEST --gtest_filter='Lanczos'
[==========] 11 tests from 11 test suites ran. (162255 ms total)
[ PASSED ] 11 tests.
```

pre-commit passes clean.

For reviewers

Refs #2519, #3021.

Golden-value comparisons via devArrMatch/CompareApprox were flaky across
CUDA versions/architectures because FP reduction order isn't deterministic.
Replace with residual, orthogonality, unit-norm, and ordering checks that
verify the defining properties of an eigenpair instead of exact values.

Fixes NVIDIA#2519
@copy-pr-bot

copy-pr-bot Bot commented Jul 20, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@coderabbitai

coderabbitai Bot commented Jul 20, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 99d34650-d210-4c06-a13e-9c97ddda7836

📥 Commits

Reviewing files that changed from the base of the PR and between 3547c6a and 38d9c69.

📒 Files selected for processing (1)
  • cpp/tests/sparse/solver/lanczos.cu
🚧 Files skipped from review as they are similar to previous changes (1)
  • cpp/tests/sparse/solver/lanczos.cu

📝 Walkthrough

Summary by CodeRabbit

  • Tests
    • Strengthened Lanczos eigensolver validation by checking intrinsic eigenpair properties, including residual accuracy, eigenvector normalization and orthogonality, Rayleigh-quotient consistency, and ascending eigenvalue order.
    • Applied the same validation to both CSR and COO matrix formulations.
    • Removed reliance on hardcoded “golden” eigenvalue expectations, improving robustness and maintainability.

Walkthrough

Changes

Lanczos validation

Layer / File(s) Summary
Eigenpair validation helpers
cpp/tests/sparse/solver/lanczos.cu
Adds Frobenius-norm computation and intrinsic checks for ordering, normalization, residuals, Rayleigh quotients, and orthogonality.
RMAT solver validation
cpp/tests/sparse/solver/lanczos.cu
Removes expected-eigenvalue state and validates CSR and COO outputs.
Fixed-matrix solver validation
cpp/tests/sparse/solver/lanczos.cu
Removes expected-eigenvalue state, validates CSR and COO solver results, and updates test setup.
Test input updates
cpp/tests/sparse/solver/lanczos.cu
Removes hardcoded expected eigenvalue arrays from RMAT and fixed-matrix parameter lists.

Estimated code review effort: 4 (Complex) | ~45 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly states the main change: replacing Lanczos eigenvalue goldens with property-based checks.
Description check ✅ Passed The description matches the code changes and explains the property-based Lanczos test update.
Linked Issues check ✅ Passed The PR removes hardcoded Lanczos eigenvalues and preserves coverage with residual, Rayleigh, norm, orthogonality, and ordering checks, matching #2519.
Out of Scope Changes check ✅ Passed No clearly unrelated changes are present; the sync and Doxygen edits appear incidental to the Lanczos test update.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@cpp/tests/sparse/solver/lanczos.cu`:
- Around line 79-87: Synchronize the RAFT stream before every host read of
results produced by host-scalar `raft::linalg::dot` in `compute_frobenius_norm`
and the affected Lanczos calculations involving `rayleigh`, `residual_sq`,
`v_norm_sq`, and `dot_ij`. Use the RAFT stream synchronization helper, and
replace the existing raw `cudaStreamSynchronize` with that helper.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 74ae28a5-be53-450a-8676-c9ef4fdfbe94

📥 Commits

Reviewing files that changed from the base of the PR and between d2bff5d and 3547c6a.

📒 Files selected for processing (1)
  • cpp/tests/sparse/solver/lanczos.cu

Comment thread cpp/tests/sparse/solver/lanczos.cu
@says1117

Copy link
Copy Markdown
Author

Test files aren't in Doxygen scope per cpp/REVIEW_GUIDELINES.md

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] Make new lanczos gtests more robust to varying cuda runtime versions

1 participant