Skip to content

fix(artifacts): keep saved findings consistent through recovery - #1310

Open
mldangelo-oai wants to merge 78 commits into
mainfrom
mdangelo/codex/scan-draft-consistency
Open

mldangelo-oai wants to merge 78 commits into
mainfrom
mdangelo/codex/scan-draft-consistency

Conversation

@mldangelo-oai

@mldangelo-oai mldangelo-oai commented Oct 5, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Use consistent finding identities and source ordering across publication, interruption, and resume, while preserving finalized artifacts.

Changes

  • Reconcile revisions without collapsing independent reports at the same location.
  • Retain unfinished work, legacy identities, annotations, and immutable saved sources.
  • Preserve artifact aliases and report export failures through the existing diagnostics.

Testing

Runtime validation on ded8582f8d62 passed both full SDK orders, 884 Python tests with 69 skips and 100 subtests, and 900 selected MCP cases on rerun. The final 49e0c1cde6cd follow-up only explains the existing optional previous-head fallback and moves fixture mutations outside assertions; its production AST is identical. All 96 focused identity tests, portable checks, types, and formatting passed again. Three fresh native reviews and independent source verification found no blocking issues. Hosted CI supplies native macOS and Windows coverage.

Risk and rollout

The main risk is inconsistent matching between TypeScript publication and Python recovery for ambiguous saved observations. The regressions check finalized findings, revision history, frozen-source replay, and unchanged source bytes. Existing artifact formats and CLI behavior are retained. Local checks ran on Linux; native macOS and Windows results remain subject to hosted CI. Portable filesystem tests adapt to the actual volume's alias behavior.

Public disclosure review

  • No customer, partner, prospect, or user identities, data, or identifying details are included.
  • No credentials, personal data, private source, scan findings, or nonpublic links or tickets are included.
  • I reviewed the branch name, title, description, commits, changes, comments, logs, screenshots, attachments, and links for public disclosure.

codex added 24 commits October 5, 2026 08:32
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 5, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-12T03:29:55.812384Z 49e0c1c Manual request
🔒 Security Review ✅ Completed 2026-10-12T03:27:47.463939Z 49e0c1c New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 91434efae5

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread plugins/codex-security/scripts/finalize_scan_contract.py Outdated
Comment thread plugins/codex-security/scripts/finalize_scan_contract.py
Comment thread plugins/codex-security/mcp-app/src/artifact-scan-draft.ts
@github-actions github-actions Bot added the bug Something isn't working label Oct 5, 2026

@zcrab-oai zcrab-oai left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the diff and relevant callers at 7dce657, with an independent second pass. No actionable findings.

Focused validation: All production changes and current historical comments reviewed. Focused Python identity/finalization/progress suites: 263 tests and 70 subtests passed, zero failures, exit 0. A duplicate run also passed; an initially loaded ddtrace plugin emitted unrelated sandbox host-inspection warnings at shutdown.

Validation limits: No native Windows or full SDK/MCP suite. The reported hash-suffix identity collision allocation predates this diff; the retained hash branch is unchanged behavior.

CI still has failed, canceled, or unfinished checks. This source-review approval does not clear those checks.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

parsed = new URL(remote);

P2 Badge Reject control characters before parsing remotes

When an imported sealed manifest contains a remote such as https://exa\nmple.com/repo (similarly with a tab or carriage return), the WHATWG URL constructor silently strips that character, so every check here passes even though the original, unmodified remote is returned and can later be rendered in publication output. The Python contract reader explicitly rejects these controls via REMOTE_CONTROL_RE, so the SDK accepts artifacts the producer rejects and admits terminal controls; validate the original string against the same control-character class before parsing it.

AGENTS.md reference: AGENTS.md:L58-L60

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread plugins/codex-security/tests/test_saved_deferred_identity.py Fixed
Comment thread plugins/codex-security/tests/test_saved_deferred_identity.py Fixed
Comment thread plugins/codex-security/scripts/workbench_saved_sources.py Fixed
@mldangelo-oai
mldangelo-oai requested a balanced review from Copilot October 12, 2026 02:24
@mldangelo-oai

Copy link
Copy Markdown
Collaborator Author

@codex review

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The TypeScript contract reader no longer rejects duplicate finding IDs, diverging from Python validation and downstream identity assumptions.

1 open finding
What changed in this PR

Improves scan artifact consistency across publication, interruption, recovery, and finalization.

Changes:

  • Reconciles finding identities, worker ownership, history, and pending work.
  • Preserves checkpoint ancestry and sealed artifact aliases.
  • Adds extensive TypeScript, Python, and MCP recovery regressions.
File Description
sdk/​typescript/​tests-ts/​stopped-scan-results.test.ts Tests checkpoint ordering.
sdk/​typescript/​tests-ts/​scan-resume.test.ts Tests resumable unfinished manifests.
sdk/​typescript/​tests-ts/​scan-recovery.test.ts Updates malformed identity recovery tests.
sdk/​typescript/​tests-ts/​runtime.test.ts Loads identity helpers in runtime tests.
sdk/​typescript/​tests-ts/​contract.test.ts Expands cross-reader contract tests.
sdk/​typescript/​tests-ts/​compact-diff-scan.test.ts Updates collision identities.
sdk/​typescript/​src/​contract.ts Extends canonical contract validation.
plugins/​codex-security/​tests/​test_workbench_standard_deep_results.py Strengthens replay assertions.
plugins/​codex-security/​tests/​test_workbench_saved_source_order.py Targets extracted source helpers.
plugins/​codex-security/​tests/​test_workbench_progress.py Tests malformed progress values.
plugins/​codex-security/​tests/​test_workbench_db_exports.py Tests interrupted Deep publication.
plugins/​codex-security/​tests/​test_workbench_completion_binding.py Preserves malformed-identity coverage.
plugins/​codex-security/​tests/​test_workbench_checkpoint_heads.py Tests accepted-head precedence.
plugins/​codex-security/​tests/​test_saved_deferred_identity.py Adds identity and recovery regressions.
plugins/​codex-security/​tests/​test_finalize_scan_contract.py Tests aliases and optional SARIF failures.
plugins/​codex-security/​scripts/​workbench_validation.py Handles unhashable usage values safely.
plugins/​codex-security/​scripts/​workbench_scan_history.py Uses shared seal detection.
plugins/​codex-security/​scripts/​workbench_saved_sources.py Adds saved-source freezing and ancestry logic.
plugins/​codex-security/​scripts/​workbench_saved_results.py Reworks recovery reconciliation and identities.
plugins/​codex-security/​scripts/​workbench_progress.py Handles unhashable progress values safely.
plugins/​codex-security/​scripts/​workbench_db.py Propagates mode and shared seal detection.
plugins/​codex-security/​scripts/​finalize_scan_contract.py Preserves sealed projections and optional exports.
plugins/​codex-security/​plugin-files.json Packages the new source helper.
plugins/​codex-security/​mcp-app/​tests/​test_artifact_scan_publication_recovery.ts Adds publication recovery scenarios.
plugins/​codex-security/​mcp-app/​tests/​test_artifact_scan_draft.ts Updates pending-work and collision expectations.
plugins/​codex-security/​mcp-app/​tests/​test_artifact_scan_draft_recovery.ts Expands draft recovery coverage.
plugins/​codex-security/​mcp-app/​tests/​scan-identity-recovery-fixture.ts Adds recovery comparison utilities.
plugins/​codex-security/​mcp-app/​tests/​scan-draft-recovery-fixture.ts Models admitted checkpoint ancestry.
plugins/​codex-security/​mcp-app/​src/​artifact-scan-draft.ts Reconciles identities, history, and reopened work.

🧠 Review effort: Balanced


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread sdk/typescript/src/contract.ts
@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. You're on a roll.

Reviewed commit: ded8582f8d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@mldangelo-oai mldangelo-oai changed the title fix(artifacts): preserve findings and finalized artifacts during scan recovery fix(artifacts): keep saved findings consistent through recovery Oct 12, 2026
@mldangelo-oai
mldangelo-oai requested a balanced review from Copilot October 12, 2026 03:24
@mldangelo-oai

Copy link
Copy Markdown
Collaborator Author

@codex review

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The large, stateful reconciliation phase needs decomposition or focused human validation before approval.

0 open findings

1 resolved since last review
Previously missed (1)

In code that hasn't changed since last review

Low severity Extract reconciliation logic into focused helpers and types

plugins/​codex-security/​scripts/​workbench_saved_results.py:1683

This reconciliation phase adds more than 500 lines of nested, dictionary-shaped state to an already large merge_saved_results function, combining causal source selection, identity matching, revision ranking, and publication ordering in one scope. That makes the recovery invariants difficult to review or change safely. Please extract this into focused reconciliation helpers/types (for example, a structured row-group model plus separate matching and identity-assignment phases) while keeping merge_saved_results as orchestration.

🧠 Review effort: Balanced

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 49e0c1cde6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".



def manifest_is_sealed(scan: dict[str, Any]) -> bool:
return scan.get("sealedAt") is not None or scan.get("artifacts") not in (None, [])

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Apply the draft predicate before custom validation

When an unsealed custom-validation draft contains artifacts: [], this new predicate now deliberately treats it as unfinished, but runCustomValidation still rejects any defined manifest.scan.artifacts at sdk/typescript/src/custom-validation.ts:216-220. Such a draft can therefore pass the revised completion/resume checks yet fail before custom validation runs; update that gate to use the same empty-array semantics.

AGENTS.md reference: AGENTS.md:L43-L45

Useful? React with 👍 / 👎.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants