Skip to content

[CI] Debug claude-review-fvm failures: full output + artifact upload - #8630

Open
janezpodhostnik wants to merge 1 commit into
masterfrom
janez/claude-review-debug
Open

[CI] Debug claude-review-fvm failures: full output + artifact upload#8630
janezpodhostnik wants to merge 1 commit into
masterfrom
janez/claude-review-debug

Conversation

@janezpodhostnik

@janezpodhostnik janezpodhostnik commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Problem

claude-review-fvm has failed on every run since it started triggering (5/5, across multiple FVM PRs). Every failure has the identical signature:

result: subtype "success", is_error: true, num_turns: 1, total_cost_usd: 0 (~3 min)
##[error] Claude result reported subtype success with is_error:true

The actual error message is invisible: the action hides Claude's output ("full output hidden for security"), and the raw output written to /home/runner/work/_temp/claude-execution-output.json is lost when the runner exits. Re-running with Actions debug logging does not help — it only adds runner-internal ##[debug] lines.

Changes

Two temporary debugging additions to .github/workflows/claude-review-fvm.yml:

  1. show_full_output: true on the claude-code-action step — prints Claude's messages to the log so the real error is visible (the action warns this echoes all messages incl. tool results; acceptable here for a public repo, to be removed after diagnosis).
  2. An actions/upload-artifact@v4 step (if: always()) that uploads claude-execution-output.json as the claude-execution-output artifact, so the raw error survives the runner even on failure.

The model is intentionally not pinned — first gather the actual error, then decide.

Both additions are marked TEMPORARY DEBUG and should be reverted once the root cause is identified.


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.

Summary by CodeRabbit

  • Chores
    • Improved automated review diagnostics by exposing complete execution details in workflow logs.
    • Added artifact uploads for review output to support troubleshooting.
    • Added warnings when expected diagnostic output is unavailable.

@janezpodhostnik
janezpodhostnik requested a review from a team as a code owner July 29, 2026 16:28
@coderabbitai

coderabbitai Bot commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

The workflow now exposes complete Claude action output in GitHub Actions logs and always attempts to upload the Claude execution JSON artifact, warning when the artifact is missing.

Changes

Claude review diagnostics

Layer / File(s) Summary
Workflow output and artifact diagnostics
.github/workflows/claude-review-fvm.yml
Enables full Claude output logging and adds an always-run artifact upload step with a missing-file warning.

Estimated code review effort: 1 (Trivial) | ~5 minutes

Possibly related PRs

Suggested reviewers: zhangchiqing, kay-zee

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly describes the workflow debug changes for claude-review-fvm, including full output and artifact upload.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch janez/claude-review-debug

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

Copy link
Copy Markdown
Contributor

Dependency Review

The following issues were found:
  • ✅ 0 vulnerable package(s)
  • ✅ 0 package(s) with incompatible licenses
  • ✅ 0 package(s) with invalid SPDX license definitions
  • ⚠️ 1 package(s) with unknown licenses.
See the Details below.

License Issues

.github/workflows/claude-review-fvm.yml

PackageVersionLicenseIssue Type
actions/upload-artifact4.*.*NullUnknown License

OpenSSF Scorecard

PackageVersionScoreDetails
actions/actions/upload-artifact 4.*.* 🟢 5.2
Details
CheckScoreReason
Code-Review🟢 10all changesets reviewed
Binary-Artifacts🟢 10no binaries found in the repo
Maintained⚠️ 00 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0
Dangerous-Workflow🟢 10no dangerous workflow patterns detected
Packaging⚠️ -1packaging workflow not detected
CII-Best-Practices⚠️ 0no effort to earn an OpenSSF best practices badge detected
Token-Permissions⚠️ 0detected GitHub workflow tokens with excessive permissions
Pinned-Dependencies⚠️ 1dependency not pinned by hash detected -- score normalized to 1
Fuzzing⚠️ 0project is not fuzzed
License🟢 10license file detected
Signed-Releases⚠️ -1no releases found
Security-Policy🟢 9security policy file detected
SAST🟢 10SAST tool is run on all commits
Branch-Protection⚠️ 0branch protection not enabled on development/release branches

Scanned Files

  • .github/workflows/claude-review-fvm.yml

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
.github/workflows/claude-review-fvm.yml (1)

96-101: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win

Use the action’s execution_file output instead of a hardcoded runner path.

The Claude action exposes the generated execution-log path as execution_file; use a step id and reference that output, optionally retaining ${{ runner.temp }} as a fallback. This avoids silently missing the artifact if the runner layout changes. (github.com)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/claude-review-fvm.yml around lines 96 - 101, Update the
Claude action step to expose an id, then change the “Upload Claude execution
output” step to use that step’s execution_file output as the artifact path,
optionally falling back to runner.temp. Remove the hardcoded
/home/runner/work/_temp path while preserving the existing artifact name and
always condition.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/claude-review-fvm.yml:
- Around line 60-66: Protect both raw Claude diagnostic outputs from public
exposure: gate or sanitize show_full_output in the workflow debug configuration
at .github/workflows/claude-review-fvm.yml lines 60-66, and apply equivalent
protection to the uploaded JSON artifact at lines 91-102, including shortened
retention or controlled access where applicable.

---

Nitpick comments:
In @.github/workflows/claude-review-fvm.yml:
- Around line 96-101: Update the Claude action step to expose an id, then change
the “Upload Claude execution output” step to use that step’s execution_file
output as the artifact path, optionally falling back to runner.temp. Remove the
hardcoded /home/runner/work/_temp path while preserving the existing artifact
name and always condition.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 73c6daab-7059-497f-af54-7dce91ec28ab

📥 Commits

Reviewing files that changed from the base of the PR and between 73f2c26 and 84cad91.

📒 Files selected for processing (1)
  • .github/workflows/claude-review-fvm.yml

Comment on lines +60 to +66
# TEMPORARY DEBUG: surface Claude's full output to diagnose the
# recurring `is_error:true` / num_turns:1 / $0 failures. The
# action's own warning applies: this logs ALL Claude messages
# (including tool results), publicly visible in the Actions log.
# Remove once the root cause is identified.
show_full_output: true

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟠 Major | ⚡ Quick win

Guard all raw Claude diagnostics against sensitive-data exposure.

Both the public log and uploaded artifact expose unsanitized Claude messages and tool results. Restrict this debugging mode to controlled access, shorten artifact retention, or sanitize the output before publishing.

  • .github/workflows/claude-review-fvm.yml#L60-L66: gate or redact show_full_output.
  • .github/workflows/claude-review-fvm.yml#L91-L102: apply the same protection to the uploaded JSON artifact.
📍 Affects 1 file
  • .github/workflows/claude-review-fvm.yml#L60-L66 (this comment)
  • .github/workflows/claude-review-fvm.yml#L91-L102
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/claude-review-fvm.yml around lines 60 - 66, Protect both
raw Claude diagnostic outputs from public exposure: gate or sanitize
show_full_output in the workflow debug configuration at
.github/workflows/claude-review-fvm.yml lines 60-66, and apply equivalent
protection to the uploaded JSON artifact at lines 91-102, including shortened
retention or controlled access where applicable.

@codecov-commenter

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@janezpodhostnik
janezpodhostnik requested a review from Kay-Zee July 29, 2026 16:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants