Add the MLX backend to the mv3 Apple demo - #260
Merged
Conversation
Follow-up to removing the MPS backend. The mv3 image-classification demo now offers MLX, ExecuTorch's Apple GPU backend, as a third option next to Core ML and XNNPACK. Both demo Xcode projects are repinned from the old swiftpm-1.1.0 SwiftPM branch to the nightly swiftpm-1.5.0.20260827, which is the first published package that carries the new backend_mlx product. (This will be repinned to the 1.5 release branch once that is cut.) In the mv3 demo: - export.py lowers MobileNet v3 to MLX with MLXPartitioner and writes mv3_mlx.pte, the same shape as the existing Core ML and XNNPACK exports. MLX lowering is ahead-of-time and does not import the mlx runtime package, so it runs on the same Linux export runner as the others. - the model selector gains an MLX case that loads mv3_mlx.pte. - the Xcode project links executorch::backend_mlx from the package and bundles mv3_mlx.pte, mirroring exactly how backend_coreml and mv3_coreml_all.pte are wired. - the export workflow's artifact list includes mv3_mlx.pte. Test Plan: Verified the wheel side of MLX end to end on macOS arm64 in a fresh venv: exported MobileNet v3 with MLXPartitioner (all ops delegated), ran it through the pybindings runtime, and matched eager to 1.2e-07. Also built a C++ app with find_package(executorch) that links executorch::backend_mlx and ran the same delegated program, exit 0. export.py parses. The export workflow is valid YAML and lists all four mv3 outputs. ClassificationController.swift parses with swiftc and its Mode switch is exhaustive over the three cases. Both Xcode projects were checked with a real plist parser (plutil -convert json). The mv3 project has 116 objects with zero ids referenced-but-undefined and zero defined-but-unreferenced. backend_mlx is wired into all eight places backend_coreml is (build files, framework phase, product dependency, product block for both the app and test targets) and mv3_mlx.pte into all six places mv3_coreml_all.pte is. The etLLM project change is only the one-line branch repin. plutil -lint passes on both. Known dependency: build-demo-ios in executorch stages the demo's .pte files with its own export scripts (portable, Core ML, XNNPACK). That job needs an MLX export step added before mv3_mlx.pte is produced in CI; until then the MLX button has no model in that build. Filed separately.
meta-cla
Bot
added
the
CLA Signed
metascroy
approved these changes
Aug 27, 2026
The pin named swiftpm-1.5.0.20260827 and the description claimed it was the first published package with the MLX product. It is not: no published branch through 20260828 declares backend_mlx, because the runtime halves only landed afterwards. The MLX backend (#22203) and its SwiftPM template half (#22261) are now on main and on the swiftpm template, so the next nightly is the first that will carry the product. Repin both demos to it. Nothing else is needed on the app side. The metallib resource bundle is a dependency of the backend_mlx product in the template, so SwiftPM attaches it automatically rather than the project having to bundle it. The demo already targets iOS 17, which matches the package floor.
The test target already bundles mv3_mlx.pte and links backend_mlx, in the same two build phases as the Core ML equivalents, but nothing loaded the model. So the delegate was shipped and never exercised: the partitioner only warns when it tags nothing, and an MLX runtime failure would have stayed green. Adds the fourth case alongside portable, Core ML and XNNPACK.
The simulator's Metal is a stub: it reports no working set and refuses to allocate a heap, which is the only way MLX allocates. So MLX cannot run there, and the delegate declines with a not-found error rather than crashing. The test asserted success anyway, so a platform that legitimately cannot run MLX turned into a red test. Skip that one case on the simulator and leave it running on a device. The picker offered MLX there too, which is a choice that can only fail. Offer it only where it can work. Also move the package to the latest release, which is the first one carrying the MLX Metal kernels for all three slices. Test Plan: Exported all four models and ran the suite on an iPhone 17 Pro simulator: three cases pass and the MLX case reports as skipped with its reason, where before it failed with "Resource not found". Built the app for a device to confirm MLX is still offered and still linked there.
The export script imported the MLX partitioner at module scope, so on a platform whose package does not carry that delegate the script died before writing anything. The nightly export job runs on Linux, where the published package ships the MLX Python code without its generated serializers, so the import fails there. That job declares its outputs with if-no-files-found: error, so one unbuildable model took the other three down with it. Import the delegate if it is there, and export the MLX model only in that case. The other three are written first, so they no longer depend on it. The workflow no longer lists the MLX model either, because that job only runs on Linux and never produces it. Also correct what the two simulator conditions say about themselves. MLX is unavailable on the simulator because the packaged simulator slice reports it so, not because Metal cannot allocate memory there. The behaviour is right and the stated reason was not. The README still described two backends, so it now describes three, and says the MLX model is exported on macOS only. Test Plan: Ran the export script twice against the current package. With the MLX import forced to fail, as on Linux, it exits 0 and writes mv3.pte, mv3_coreml_all.pte and mv3_xnnpack_fp32.pte, where before it wrote nothing. On macOS it writes all four, and the MLX one carries a real delegate. The MLX pin stays on the demo project, which links that product. Reverted it on the language model project, which does not, so that app keeps the package it already had.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add the MLX backend to the mv3 Apple demo
Follow-up to #259, which removed the deprecated MPS backend. This adds MLX,
ExecuTorch's Apple GPU backend, as a third option in the mv3 image-classification
demo, next to Core ML and XNNPACK.
What changed:
the nightly swiftpm-1.5.0.20260829. No earlier published branch carries the product:
every swiftpm-1.5.0.2026082x branch through the 28th declares zero backend_mlx,
because the runtime halves (Add the MLX backend to the Apple frameworks and SwiftPM package pytorch/executorch#22203 and #22261) only landed after
those nightlies ran. The 29th is the first that will have it. This will be repinned
to the 1.5 release branch once it is cut.
mv3_mlx.pte, the same shape as the existing Core ML and XNNPACK exports. MLX
lowering is ahead-of-time and does not import the mlx runtime package, so it runs on
the same Linux export runner as the others.
mv3_mlx.pte, mirroring how backend_coreml and mv3_coreml_all.pte are already wired.
Dependency, important: the iOS build (build-demo-ios, run from ExecuTorch's
test_ios_ci.sh) stages the demo's model files with its own export scripts, which were
portable, Core ML and XNNPACK only. That gap is now filled by
pytorch/executorch#22297, which lowers the model with MLXPartitioner inline and writes
mv3_mlx.pte. Lowering needs no mlx runtime package, so it runs on the existing runner.
So this PR now waits on exactly two things: #22297 landing, and the 20260829 nightly
publishing backend_mlx. Nothing further is needed on the app side, because the metallib
resource bundle is a dependency of the backend_mlx product in the template and SwiftPM
attaches it automatically.
Test Plan:
Verified the MLX path end to end on macOS arm64 in a fresh venv, using the macOS
wheel built at the tip of the merged runtime stack:
the resulting program carries the MLX delegate. Ran it through the pybindings
runtime and matched eager to a max error of 1.2e-07.
executorch::backend_mlx, and ran the same delegated program from C++, exit 0. otool
confirmed it links the shipped libexecutorch_backend_mlx.dylib and libexecutorch.dylib.
Static checks on this change:
over the three cases.
The mv3 project has 116 objects with zero ids referenced-but-undefined and zero
defined-but-unreferenced. backend_mlx is wired into all eight places backend_coreml
is, across the app and test targets, and mv3_mlx.pte into all six places
mv3_coreml_all.pte is. The etLLM project change is only the one-line branch repin.
plutil -lint passes on both.