WHL: enable Limited API-compliant (abi3) wheels

[neutrinoceros]

I initially just wanted to experiment with this and see how far from a working state we were... turns out the existing code is pretty close already, so only a couple tweaks were needed.

PyTuple_GET_* functions are normally faster, as they skip certain checks that equivalent Limited API functions will run, so there could be a case to be made that we should still use them by default, but I'm not convinced that the impact is even measurable (I'll run the benchmarks to double check before I undraft though).

[neutrinoceros]

This appears to work as expected in CI.
I do seem to get slightly worse performance on some benchmarks though. It still seems worth it to me but maybe I can make the default build use version-specific APIs, and only enable abi3 as an opt in (enabled in ci)

[rgommers]

make the default build use version-specific APIs, and only enable abi3 as an opt in (enabled in ci)

That's the way I'm setting it up for PyWavelets too, and is generally the right strategy I believe. For distros for example, they always only compile for a single Python version, so there are no upsides to using the Stable ABI. Same for end users building from source. Wheel distribution is kinda the odd exception.

[rgommers]

Can you post the benchmark results?

[neutrinoceros]

Ah, there are already lost to my terminal history unfortunately, but making abi3 an opt-in was trivial enough

[neutrinoceros]

opening for review now. The last commit is temporary and intended to be trimmed before merge, but it helps seeing relevant CI in action

[neutrinoceros]

I reran the benchmarks (this time on linux + x86_64)

master:

Bottleneck performance benchmark
    Bottleneck 1.6.0.post0.dev38; Numpy 2.4.2
    Speed is NumPy time divided by Bottleneck time
    NaN means approx one-fifth NaNs; float64 used

              no NaN     no NaN      NaN       no NaN      NaN    
               (100,)  (1000,1000)(1000,1000)(1000,1000)(1000,1000)
               axis=0     axis=0     axis=0     axis=1     axis=1  
nansum         21.3        0.8        2.1        1.7        2.2
nanmean        93.8        1.2        2.1        1.7        2.4
nanstd         91.6        1.6        2.3        2.0        2.6
nanvar         89.4        1.6        2.1        2.0        2.3
nanmin         20.7        0.4        0.5        0.6        0.7
nanmax         21.0        0.4        0.5        0.6        0.7
median         74.7        1.3        6.8        1.1        6.6
nanmedian      74.1        3.1        3.6        3.0        3.5
ss              7.0        0.8        0.8        1.0        1.0
nanargmin      43.3        1.8        3.8        1.4        4.8
nanargmax      43.6        2.9        6.5        2.2        5.0
anynan          8.0        0.4       35.7        0.7       27.5
allnan         11.4      154.3      100.4      139.8       86.5
rankdata       20.5        2.5        1.7        3.1        1.9
nanrankdata    23.3        2.9        1.4        2.9        1.4
partition       2.3        0.4        0.7        0.2        0.5
argpartition    4.2        0.6        1.4        0.5        1.6
replace         7.4        1.8        1.8        1.8        1.9
push         1513.1        9.9        9.6       21.4       14.0
move_sum     1546.1       48.0      179.2      249.5      238.7
move_mean    3753.6      133.1      187.0      330.0      264.8
move_std     5822.4      117.6      256.7      250.0      362.0
move_var     6180.6      146.7      297.3      299.5      412.7
move_min      878.3        5.7        5.6       12.6       16.5
move_max      857.8        5.4        5.3       12.1       15.9
move_argmin  1815.1       37.8       75.3       43.6       87.8
move_argmax  1843.1       39.1       76.6       43.3       90.6
move_median  1199.6       90.7      102.3       95.5      103.8
move_rank     584.6        1.0        1.2        1.5        1.7

this branch:

Bottleneck performance benchmark
    Bottleneck 1.6.0.post0.dev38; Numpy 2.4.2
    Speed is NumPy time divided by Bottleneck time
    NaN means approx one-fifth NaNs; float64 used

              no NaN     no NaN      NaN       no NaN      NaN    
               (100,)  (1000,1000)(1000,1000)(1000,1000)(1000,1000)
               axis=0     axis=0     axis=0     axis=1     axis=1  
nansum         22.8        0.9        1.9        1.2        2.1
nanmean        67.7        1.4        2.2        1.6        2.5
nanstd         84.1        2.0        2.3        2.0        2.6
nanvar        115.1        1.4        2.3        2.0        2.6
nanmin         18.8        0.3        0.4        0.4        0.5
nanmax         19.2        0.3        0.4        0.4        0.5
median         70.8        1.2        5.6        1.1        6.5
nanmedian      69.7        3.3        4.3        3.5        3.8
ss              7.1        0.8        0.8        1.0        1.0
nanargmin      54.3        2.4        5.1        1.3        4.5
nanargmax      53.5        3.0        6.3        1.7        5.7
anynan          6.5        0.4       37.8        0.5       35.8
allnan         10.1      153.3      105.8      138.2       98.3
rankdata       21.3        2.5        1.9        3.1        2.0
nanrankdata    24.4        2.9        1.5        3.6        1.6
partition       2.4        0.4        0.7        0.2        0.5
argpartition    4.0        0.6        1.3        0.5        1.5
replace         7.1        1.8        1.8        1.8        1.8
push         1565.4        9.6        9.5       20.1       13.8
move_sum     1637.4       70.6      171.5      248.3      240.7
move_mean    3939.6      123.2      193.0      339.6      262.7
move_std     6360.4      120.0      251.0      255.3      361.8
move_var     6540.3      143.3      302.4      302.0      428.5
move_min      931.7        6.7        6.7       13.7       16.2
move_max      998.1        5.5        5.6       12.6       a16.5
move_argmin  1923.1       39.6       76.7       43.7       88.3
move_argmax  2089.7       39.5       76.3       44.0       91.0
move_median  1275.1       95.6      102.6       91.9      103.3
move_rank     596.3        1.1        1.3        1.6        1.8

[neutrinoceros]

not sure any of these diffs are significant, but I think it's still worth making abi3 an opt-in so we can grow this space without worrying about maximising performance. I expect this will come in handy later this year when/if some version of abi3 makes it into Python 3.15t (see PEPs 803 and 809)

[rgommers]

not sure any of these diffs are significant,

It's pretty hard to tell, using asv compare is usually more informative. It looks like some of the faster-running benchmarks are slightly slower, but not much (as expected).

I'll do a quick pass on review now.

[rgommers]

The opt-in approach here looks good to me. The Stable ABI is typically irrelevant for distros as well as end users building from source, so opt-in seems better as the default in general.

A couple of initial comments.

[rgommers]

This doesn't look quite right, I think we should always target the lowest-supported Python version, rather than the version that's running the build. Or do you plan to enforce (via code comment?) to handle this in the CI workflow file?

[neutrinoceros]

The problem is that when targeting a version that's older than the running interpreter, there are no guarantees that the resulting binaries will actually be backward compatible. Luckily, because cibuildwheel already runs targets in increasing Python version order, this approach works perfectly: the first (oldest) Python builds a wheel and runs tests, then all the other skip the build step and only check for forward compat by running tests.

[neutrinoceros]

see https://docs.python.org/3/c-api/stable.html#limited-api-caveats

[neutrinoceros]

@rgommers, any follow up on this ?

[neutrinoceros]

@rgommers Can we move forward with this one ? I'm currently using it as a base to #516, because I assume we'll want to ship abi3 wheels. I could also decouple them completely but then this one would be moot the second #516 is done.

[neutrinoceros]

friendly ping @rgommers

[rgommers]

Yes, let's move forward! I'll note that with the latest releases of meson and meson-python we have full abi3t support now, so it should no longer be necessary to set ABI3_TARGET_VERSION.

[rgommers]

I'll try to review by tomorrow.

[neutrinoceros]

Enabling wheel testing (#592) revealed some problem with this PR. Testing a cp310-abi3 wheel on cp311 immediately results in a ModuleNotFoundError. I'll try to fix this today.

[rgommers]

Enabling wheel testing (#592) revealed some problem with this PR. Testing a cp310-abi3 wheel on cp311 immediately results in a ModuleNotFoundError. I'll try to fix this today.

Try dropping 3.10, it's probably this Cython bug: cython/cython#7914

[neutrinoceros]

you know I thought about it... but there's no Cython in bottleneck 😅

[neutrinoceros]

though, I'll try it nonetheless just to make sure I'm not spending time fixing and debugging a 3.10-specific problem

[neutrinoceros]

Same exact issue with 3.11 -> 3.12, removed the temp commit that skipped cp310

[neutrinoceros]

Interesting. It's going much better on the meson branch. I'll just self-merge whatever small adjusments these branch have in common and rebase both of them independently.

[rdbisme]

I'm not very familiar with meson, I'm definitely better with cMake, but I think merging support for meson before will allow to control flags and things better than the setup.py. Let's get that merged, then try again to enable the Limited API?

[neutrinoceros]

right now I'm doing both in #556, though I could easily split out the enabling of abi3 if you'd like

[rdbisme]

Ok, no worries. Let's do both and see how it goes.

[rgommers]

Ah, forgot that we didn't land meson-python support yet. Yeah I wouldn't trust setuptools much, and it's almost unmaintained right now (beyond trivial stuff) - a PR for abi3t has been open for half a year, with not even a single comment from a maintainer despite repeated pings.

you know I thought about it... but there's no Cython in bottleneck 😅

🙈 too much context switching