fix(cose): bound payload nesting explicitly, not via RecursionError

[imran-siddique]

main has been red since #274 merged. Two deeply-nested-payload tests fail on ubuntu 3.12 and 3.13 while passing on 3.11 and on Windows.

It is not a flaky test. The DOS-006 guard did not exist on Linux.

What was happening

_parse_payload relied on json.loads raising RecursionError:

except RecursionError as exc:
    # A manifest is untrusted input, so nesting must produce a verdict
    # rather than unwind the caller's stack (DOS-006).
    raise CoseStructureError(...)

CPython on Linux parses thousands of levels without raising. So the 5000-level payload simply parsed, _check_version then raised on the resulting {"a": {"a": ...}}:

E   CoseVersionError: unsupported manifest version None; this verifier implements '0.2'

Windows tripped its own recursion limit and therefore looked protected. A control whose behaviour depends on which platform it runs on is not a control — on Linux, arbitrarily deep untrusted input was accepted.

The fix

An explicit bound, _MAX_PAYLOAD_NESTING = 64, checked by a string-aware scan of the decoded text before json.loads:

• Before, not after — refusing the work after paying for it defeats the purpose of a DOS guard.
• String-aware, so a brace inside a member value cannot inflate the count and make a legitimate manifest look like an attack. Backslash escapes are handled so "\\" does not swallow the closing quote.
• The RecursionError catch stays as a second line.

64 is generous: the deepest path this spec defines is roughly artifacts.tool_manifest.tools[].approved_scope, about six levels.

The test proves the bound, not the accident

The new test refuses a 65-level payload. That is far below any interpreter's recursion limit, so it cannot be passing for the old reason — if the bound were removed, json.loads would parse it happily on every platform and the test would fail. Plus a string-awareness test and an at-the-limit case that must still parse.

How it was found

Cutting python-v0.11.0 (#289): that PR's CI failed, and the failures predated it. Worth stating plainly — this is the cost of #274 landing without its workflows ever running. Its logic was sound, which my review said and still holds; what was missing was the matrix.

Tests

846 passed, 6 skipped. ruff, mypy clean.

Merge this before #289 so the release does not ship from a red main.

🤖 Generated with Claude Code