Skip to content

fix(linalg): use a lane-wise intrinsic for i32x8 multiplication on x86 - #8579

Open
LuciferYang wants to merge 2 commits into
lance-format:mainfrom
LuciferYang:fix/r9-1-i32x8-mul
Open

fix(linalg): use a lane-wise intrinsic for i32x8 multiplication on x86#8579
LuciferYang wants to merge 2 commits into
lance-format:mainfrom
LuciferYang:fix/r9-1-i32x8-mul

Conversation

@LuciferYang

@LuciferYang LuciferYang commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

What was wrong

impl Mul for i32x8's x86_64 arm called _mm256_mul_epi32 (vpmuldq) where _mm256_mullo_epi32 (vpmulld) was meant. vpmuldq multiplies only the even 32-bit lanes and writes four 64-bit results, so four of the eight lanes were destroyed. Measured on an AVX2 host, [1, 2, ..., 8] squared:

result
_mm256_mul_epi32 [1, 0, 9, 0, 25, 0, 49, 0]
_mm256_mullo_epi32 [1, 4, 9, 16, 25, 36, 49, 64]

Both intrinsics are AVX2, so the file's CPU requirement is unchanged. The aarch64 (vmulq_s32) and loongarch64 (lasx_xvmul_w) arms were already lane-wise, and this brings x86 in line with them.

The two intrinsics share a signature, so picking the wrong one is a silent wrong answer rather than a compile error, and nothing in the file wrote the contract down. fn mul now states it: lane-wise, low 32 bits, wrapping. The wrapping half is scoped to mul deliberately, since reduce_sum panics on overflow in a debug build on x86_64 and loongarch64 while aarch64 wraps.

Scope: latent, not a live wrong answer

Nothing in the repo multiplies i32x8. The only use is as a gather index vector in f32x8::gather, which does no multiplication; the impl exists because SIMD's supertrait bound requires Mul. So no shipped result is wrong today. What made it worth fixing is that the file had an empty mod tests {}, so the next caller would have inherited the bug silently.

Why the test is portable rather than x86-gated

The assertion is that multiplication is lane-wise, which is what all three arms promise, so gating the test on cfg(target_arch = "x86_64") would cost more than it buys: on aarch64 the test would not exist, linux-arm and mac-build would compile it out, and a copy-paste slip in the NEON arm (both halves reading self.0.0) would ship green. Only the feature check is arch-gated, following f32.rs and f64.rs.

That runtime is_x86_feature_detected!("avx2") early return is load-bearing. qemu-pre-haswell runs cargo test --release -p lance-linalg --lib under qemu-x86_64 -cpu Nehalem, which has no AVX2, and these methods carry no #[target_feature] gate, so without the guard that job would SIGILL. Worth stating plainly, because it reads backwards: that job is the only one scoped to lance-linalg, and it is the one where this test asserts nothing. The assertions run on linux-coverage-test and windows-build.

Three cases, because [1..8] squared is narrow enough that a saturating or widening implementation would also pass it. Each was checked against both intrinsics on an AVX2 host, and mutation testing kills exactly one case per mutant, so none of them is padding:

case catches mutant that proves it
squares wrong lane pairing swap the two aarch64 halves
mixed_signs lost sign vabsq_s32 on both operands
wraps_to_low_32_bits saturating impl vmull_s32 + vqmovn_s64

An all-zero case is deliberately absent: vpmuldq returns zeros too, so it discriminates nothing.

One pre-existing precondition, now written down

The x86_64 struct i32x8 gained a note that its intrinsics carry no #[target_feature] gate of their own, so callers must already be inside an AVX2-checked context. This is not a behavior change: Add, Sub, min, splat and load are all in the same position, and it is currently unreachable because the one caller sits inside gather_avx2, which is #[target_feature(enable = "avx2")] behind a runtime check.

Test plan

  • cargo test -p lance-linalg --lib on aarch64-apple-darwin: 152 passed
  • cargo test -p lance-linalg --lib --target x86_64-apple-darwin: 210 passed
  • cargo clippy -p lance-linalg --all-targets -- -D warnings on both targets: clean
  • cargo fmt --all -- --check and cargo doc -p lance-linalg --no-deps: clean
  • Both intrinsics executed on real AVX2 via qemu-x86_64 -cpu max to produce the table above; wrapping (65536² = 0) and sign (-3 × 7 = -21) confirmed there too

Out of scope

impl Mul for u8x16 has the same family of defect with a different failure mode: x86 saturates (_mm_packus_epi16 clamps to 255) while aarch64 and the portable arm wrap, so u8(15) * u8(31) is 255 on one and 209 on the other, and its test pins both behaviors behind #[cfg] instead of resolving them. Choosing one contract changes some platform's current output, so it belongs in its own change. Filed as #8578.

The guard test added with the `_mm256_mullo_epi32` fix was gated on
`cfg(target_arch = "x86_64")`, so the aarch64 and loongarch64 arms of the
same `Mul` impl had no coverage, and on a non-AVX2 x86 host the runtime
feature check returned before any assertion ran. Un-gate the body, keep
only the feature check arch-gated as `f32.rs` and `f64.rs` do, and add a
negative and a wrapping case so the low-32-bits contract is pinned rather
than just the all-positive squares.

Also document what `Mul` promises (lane-wise, low 32 bits, wrapping) and
record that the x86 arm's intrinsics carry no `#[target_feature]` gate.
@github-actions github-actions Bot added A-index Vector index, linalg, tokenizer bug Something isn't working labels Aug 17, 2026

@lance-gatekeeper lance-gatekeeper Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gate recommendation: approve.

The intrinsic replacement fixes the x86 lane-shape bug at its source and matches the established lane-wise, low-32-bit behavior on the other architectures. The portable regression cases cover odd-lane loss, signed products, and wrapping overflow.

@lance-gatekeeper lance-gatekeeper Bot added the K-approved Latest Gatekeeper recommendation permits acceptance. label Aug 17, 2026
@codecov

codecov Bot commented Aug 17, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-index Vector index, linalg, tokenizer bug Something isn't working K-approved Latest Gatekeeper recommendation permits acceptance.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant