BLD: migrate build backend to meson-python and enable abi3 wheels

[neutrinoceros]

This is based off #511
close #516
very much a work in progress.

When attempting a build with

uvx meson setup build
uvx meson compile -C build

I get 2 errors about malloc and free being undefined. Here's the first one:

bottleneck/src/reduce_template.c:819:5: error: call to undeclared library function 'malloc' with type 'void *(unsigned long)'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
  819 |     BUFFER_NEW(float64, LENGTH)
      |     ^
bottleneck/src/reduce_template.c:802:31: note: expanded from macro 'BUFFER_NEW'
  802 |         npy_##dtype *buffer = malloc(length * sizeof(npy_##dtype));
      |                               ^
bottleneck/src/reduce_template.c:819:5: note: include the header <stdlib.h> or explicitly provide a declaration for 'malloc'

I don't understand how this happens, #include <stdlib.h> is already present in bottleneck/src/reduce_template.c.

If I run uv build instead, I get

../meson.build:25:3: ERROR: File bottleneck/src/reduce.c does not exist.

so clearly I must be doing something wrong, but I'm not too surprised because this is the first time I attempt to hook python scripts into a meson build.

[neutrinoceros]

I reached a somewhat functional state where at least some jobs are passing.
What's broken:

• docs: many references to setup.py (critical)
• mnt: Makefile and bn_setup.sh should be either removed or updated (tbd)
• warnings-as-errors in tests (very important)
• dynamic config from setuptools (possibly not needed, but I need to understand what I'm giving up)
• dynamic version number (nice to have, but I'm not personally motivated to preserve this)

[neutrinoceros]

looks like this is what's failing windows-arm64 jobs: astral-sh/uv#12906

[neutrinoceros]

got a bit further using the workaround described in the issue I just linked. Now meson correctly detects a native Python interpreter, but then hit

C:\a\_temp\uv-python-dir\cpython-3.11.15-windows-aarch64-none\python311.dll:
      file not recognized: file format not recognized

which I don't understand yet

[rgommers]

I had a quick looks; this looks like a good start to me.

[rgommers]

If you define a generator, this can be a one-liner per file, same as in for example:

https://github.com/scipy/scipy/blob/20c4b5e75727ae1d3dc19a2bb8c65c43f1366592/scipy/sparse/meson.build#L7-L8

[neutrinoceros]

a generator is how I initially tried to write this but I struggled to define a template for the output filename, because it's obtained by subtraction rather than addition.

[rgommers]

Could rename the source files? They're not valid C code, so .c.in or .c.template seems more accurate anyway

[neutrinoceros]

Yes, I was thinking doing just that as a cleanup step after I got CI stable again. Though I prefer extensions like .template.c, as it allows syntax highlighting to sort-of work instead of not at all.

[neutrinoceros]

actually went for it tonight because I had limited bandwidth and this didn't require much. I ended up using your suggestion (.c.template) because I couldn't work out how to implement .template.c using available substitutions .

[neutrinoceros]

I'm hesitant to just drop windows-arm64 as a normal CI target. It works in the context of cibw but I'm struggling to set it up using actions/setup-python and/or astral-sh/setup-uv.

[neutrinoceros]

I got lost as I tried to get everything working at the same time. I've disabled limited-api builds for now because I found it hard to setup correctly with 3.14t also being a target. I'm also disabling failing targets as I go right now, with the goal to reveal the subset that does work already, before I tackle failures one by one again.

[rgommers]

No, meson is a dependency of meson-python.

The issue in build_wheel is specific to 3.14t, where numpy headers aren't found. Not sure what's up with that so quickly since numpy clearly is present:

[0](https://github.com/pydata/bottleneck/actions/runs/34701317755/job/103573926066?pr=556#step:4:971)
    Program python found: YES (C:\Users\runneradmin\AppData\Local\uv\cache\builds-v0\.tmpKIfFM3\Scripts\python.exe)
    Dependency numpy for host machine found: YES 2.4.6 (cached)
....
[73](https://github.com/pydata/bottleneck/actions/runs/34701317755/job/103573926066?pr=556#step:4:1004)
    [7/14] Compiling C object reduce.cp314t-win_arm64.pyd.p/meson-generated_reduce.c.obj
    FAILED: [code=2] reduce.cp314t-win_arm64.pyd.p/meson-generated_reduce.c.obj 
    "cl" "-Ireduce.cp314t-win_arm64.pyd.p" "-I." "-I.." "-I..\bottleneck\src" "-IC:\Users\runneradmin\AppData\Local\uv\cache\builds-v0\.tmpS7hsPh\Lib\site-packages\numpy\_core\include" "-IC:\Users\runneradmin\AppData\Local\pypa\cibuildwheel\Cache\nuget-cpython\pythonarm64-freethreaded.3.14.7\tools\Include" "-DNDEBUG" "/MD" "/nologo" "/utf-8" "/W2" "/O2" "/Gw" "-DPy_GIL_DISABLED" "/showIncludes" "-DNPY_NO_DEPRECATED_API=NPY_2_0_API_VERSION" "/Fdreduce.cp314t-win_arm64.pyd.p\meson-generated_reduce.c.pdb" /Foreduce.cp314t-win_arm64.pyd.p/meson-generated_reduce.c.obj "/c" reduce.cp314t-win_arm64.pyd.p/reduce.c
    ..\bottleneck\src\bottleneck.h(7): fatal error C1083: Cannot open include file: 'numpy/arrayobject.h': No such file or directory

There's also a site-packages\numpy\_core\include include path. Has to be something simple, I just don't see it yet from browsing the log.

[rgommers]

The test-windows changes are a workflow config issue, cpython gets downloaded for x86-64 on an arm64 image, and then:

   Downloaded cpython-3.11.16-windows-x86_64-none (download)
...
          Need python for aarch64, but found x86_64

That is an unfortunate default uv behavior, you have to make it install an arm64 cpython

[neutrinoceros]

Thank you, I keep forgetting about this problem: astral-sh/uv#12906

[rgommers]

Oh wait, the wheel build issue does ring a bell: the problem here is that we're building all Windows wheels in a single job (which none of the other projects I work on do; it leads to issues like this one), and CPython didn't fix a free-threading header issue - that's why the 3.11 to 3.14 builds work and 3.14t chokes. From https://py-free-threading.github.io/installing-cpython/:

"Due to limitations of the Windows Python.org installer, using free-threaded Python installed from the Python.org installer may lead to trouble. In particular, if you install both a free-threaded and gil-enabled build of Python 3.13+ using the Python.org installer, both installs will share a site-packages folder. This can very quickly lead to broken environments if packages for both versions are simultaneously installed."

Let me figure out if there's an easy workaround.

[rgommers]

@neutrinoceros the problem is config-settings = { build-dir = "build" I think: that reuses the exact same build directory across all builds on win-arm64. That won't work in this case, because numpy is now installed in a different location. I think it's a recipe for problems. Separate jobs would fix it, or separate build directories. Reusing build dirs between jobs never seems healthy to me.

[rgommers]

Those fixups don't look right. I thought about it some more, and reusing a build directory is just not a valid thing to do between two fresh builds with different dependencies.

build-dir = "build" should be deleted from pyproject.toml. That way meson-python will simply create a fresh build dir, avoiding the problem. I think restructuring the jobs to have a single build per job is a bit more work, but better - it's a lot easier to understand, and safer.

I'll note that setuptools happens to work in this case, because within the build dir it does namespacing like so:

build/
    cp313-win_arm64/
    cp314-win_arm64/
    cp314t-win_arm64/

that's special-casing the interpreter, which makes it work - but in general it's also susceptible to changes between builds that aren't special-cased (e.g., a change to a compiler, codegen, etc.).

Meson, as a general-purpose build system, treats Python as any other dependency and not one with special-casing for how build caching and meson setup --reconfigure work.

[neutrinoceros]

I must not be understanding where the meson executable is even found. Is it included in the docker image ?
No, meson is a dependency of meson-python.

Let me clarify: the issue I'm seeing is that despite having meson==1.11.2 locked, uv is still somehow installing 1.12.0 in CI. I initially assume uv would respect my constraint, which meant meson 1.12.0 had to come from somewhere else, but having turned on uv's debug log I can see that the problem is really there.

[neutrinoceros]

Ah ! I think that's just because the build isolation intentionally ignores runtime constraints. Promoting the constraint to an actual build-time requirement should do the trick.

[neutrinoceros]

Okay wheels are working now.
The remaining errors are all on regular windows CI, and relate to interpreter selection in at least 3 different ways:

1. cp315 + x64combos are either unsupported by actions/setup-python, or misconfigured

The version '3.15' with architecture 'x64' was not found for Windows 2022.

2. build-time failures for x86

Need python for x86_64, but found x86
        Run-time dependency python found: NO  (tried pkgconfig and sysconfig)
  
        ..\meson.build:43:3: ERROR: Python dependency not found

3. A linker error on windows arm64

2026-09-13T10:08:26.4415431Z       C:\hostedtoolcache\windows\Python\3.11.9\arm64\python311.dll: file not
2026-09-13T10:08:26.4415776Z       recognized: file format not recognized
2026-09-13T10:08:26.4416050Z       collect2.exe: error: ld returned 1 exit status

I think 1 and 2 are relatively simple to fix. I'll give it a try now.

[neutrinoceros]

1. I got x64 jobs to work by downgrading Python to 3.14
2. I don't really understand the difference between x86 and x86_64, any chance this is a shortcoming from meson itself ?
3. I still have no idea how to fix the linker error on arm64

[rgommers]

2. I don't really understand the difference between x86 and x86_64, any chance this is a shortcoming from meson itself ?

x86 is a 32-bit interpreter. Need python for x86_64, but found x86 is the standard issue where we need/use the bus1/cabuild action to activate 32-bit MSVC, except in reverse - so the job is using that action when it shouldn't.

I still have no idea how to fix the linker error on arm64

It's picking up MinGW:

        C compiler for the host machine: cc (gcc 14.2.0 "cc
        (x86_64-posix-seh-rev2, Built by MinGW-Builds project) 14.2.0")

Pass --vsenv to the build to force MSVC, like in the wheel build jobs.

1. cp315 + x64combos are either unsupported by actions/setup-python, or misconfigured

Needs a pre-release set to true probably?

[neutrinoceros]

Thanks !

• arm64 is go
• windows CI is back to Python 3.15

x86 is a 32-bit interpreter. Need python for x86_64, but found x86 is the standard issue where we need/use the bus1/cabuild action to activate 32-bit MSVC, except in reverse - so the job is using that action when it shouldn't.

But bus1/cabuild isn't used at all in regular CI, it's just there for cibw, so I'm still confused. On the off chance that you got it mixed up, I tried adding the action to regular CI, and the error message is different, but it's still failing.

[rgommers]

Note that this duplicates a Python install from ./.github/bottleneck-action with setup-uv.

[rgommers]

Try inserting this in test-windows (copied from numpy):

      - name: pkg-config
        run: |
          python -m pip install pkgconf==2.5.1.post1

It's going wrong in the Python detection on x86-64, and the CI log has this warning:

 WARNING: Found pkg-config 'C:\\Strawberry\\perl\\bin\\pkg-config.BAT'

The windows-latest image has a completely broken pkg-config on the PATH and for some reason that I don't know about, GitHub doesn't get that fixed.

[neutrinoceros]

Interesting. The errors we get now look furiously like the ones we saw with bus1/cabuild