Skip to content

fix(bn254): drop subgroup check from V0 G2 addition - #1764

Open
valentynkit wants to merge 2 commits into
Syndica:mainfrom
valentynkit:fix/bn254-g2-add-subgroup
Open

fix(bn254): drop subgroup check from V0 G2 addition#1764
valentynkit wants to merge 2 commits into
Syndica:mainfrom
valentynkit:fix/bn254-g2-add-subgroup

Conversation

@valentynkit

Copy link
Copy Markdown
Contributor

Intent

Fixes #1713.

ALT_BN128_G2_ADD must accept any on-curve G2 point; the prime-order subgroup check is not part of V0 addition. Sig decodes both add operands through the subgroup-checking G2.fromBytes, so an on-curve point outside the order-r subgroup returns r0 = 1 (error) on Sig while Agave and Firedancer return r0 = 0 with the point. A program can branch on r0 or store the output, so the same syscall forks execution and committed account state. enable_alt_bn128_g2_syscalls (SIMD-0302) is active on all clusters, so this is reachable on mainnet today.

Implementation

  • Split the G2 decode checks: isWellFormed becomes isOnCurve (curve equation) and isInSubgroup (order-r membership), composed in fromBytes behind a Validate flag. addSyscall passes .curve_only; mulSyscall and pairingSyscall pass .curve_and_subgroup and keep their current behavior.
  • Matches both oracles, which gate G2 add the same way: Agave into_affine_unchecked (curve-only) vs the subgroup-checking TryFrom<PodG2>, and Firedancer g2_frombytes_check_eq_only vs _check_subgroup.
  • G1 is untouched (cofactor 1, no subgroup check). G2/G1 subtraction is unimplemented, so addition is the only affected path. The add path has routed through the subgroup-checking G2.fromBytes since vm: implement LE & G2 alt_bn128  #1226.

Ramifications

  • V0 G2 addition now adds and returns on-curve non-subgroup operands instead of rejecting them, matching Agave and Firedancer.
  • Multiplication and pairing still run the subgroup check, so nothing that depends on subgroup membership is weakened.

Tests

  • Added test "G2 addition accepts on-curve non-subgroup point": addSyscall(P, infinity) returns P, while mulSyscall(P, 1) still errors NotWellFormed. P is the on-curve non-subgroup point that test "edge cases" already rejects via pairingSyscall.
  • A/B verified: the test fails at the subgroup check without this change and passes with it. add/mul/pairing/edge cases stay green.

V0 G2 addition accepts any on-curve point; the subgroup check belongs to
multiplication and pairing only. Sig decoded both add operands through the
subgroup-checking G2.fromBytes, so it rejected on-curve non-subgroup points
that Agave and Firedancer accept. That is a consensus divergence in the
ALT_BN128_G2_ADD syscall. Thread a Validate flag through the G2 decode:
curve-only for add, curve and subgroup for mul and pairing.
isWellFormed took a validate flag, which made the name mean two things at
once. Split it into isOnCurve and isInSubgroup composed in fromBytes, so
each check stays honest. Add commit-pinned Agave and Firedancer permalinks
for the curve-only add rule, and note the regression point is the same one
test "edge cases" already rejects.
@github-project-automation github-project-automation Bot moved this to 🏗 In progress in Sig Jul 25, 2026
@dnut dnut added the external label Jul 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

Status: 🏗 In progress

Development

Successfully merging this pull request may close these issues.

G2 addition incorrectly rejects on-curve points outside the subgroup

2 participants