Advanced Learning Path for running parallel vision inference driven by single M55 on an Alif Ensemble E8 DevKit using Zephyr, ExecuTorch, Ethos-U55, and Ethos-U85.

content/learning-paths/embedded-and-microcontrollers/alif-dual-npu-vision/_index.md

@@ -0,0 +1,75 @@
+---
+title: Run parallel vision inference on an Alif Ensemble E8 with Zephyr
+
+description: Build a power-conscious live camera demo that drives Ethos-U55 and Ethos-U85 from one Cortex-M55 MCU.
+
+minutes_to_complete: 90
+
+who_is_this_for: This is an advanced topic for embedded ML developers who want to run two ExecuTorch models concurrently on separate Ethos-U NPUs under Zephyr.
+
+learning_objectives:
+    - Explain how one MCU can coordinate two NPUs while avoiding the power and system cost of a second MCU or application processor
+    - Configure an Alif Ensemble E8 DevKit for native Zephyr camera, ISP, display, and dual-NPU operation
+    - Build, package, and flash an ExecuTorch application that targets Ethos-U55 and Ethos-U85
+    - Validate live camera capture, model results, and parallel inference timing
+
+prerequisites:
+    - Experience with C/C++, embedded systems, and Zephyr build concepts
+    - A development machine running macOS on Apple Silicon with Homebrew and the Xcode Command Line Tools installed
+    - An [Alif Ensemble E8 DevKit](https://alifsemi.com/support/kits/ensemble-e8devkit/) with an MT9M114 camera connected to J16 and an MW405 display
+    - Alif SEROM 1.105.65 and SERAM 1.110.0 installed on the board
+    - Alif SEToolkit 1.10 installed on the development machine
+
+author: Varun Chari
+
+generate_summary_faq: false
+rerun_summary: false
+rerun_faqs: false
+
+skilllevels: Advanced
+subjects: ML
+armips:
+    - Cortex-M55
+    - Ethos-U55
+    - Ethos-U85
+tools_software_languages:
+    - ExecuTorch
+    - Zephyr
+    - Python
+    - GCC
+operatingsystems:
+    - macOS
+    - RTOS
+
+further_reading:
+    - resource:
+        title: Alif Ensemble E8 DevKit support page
+        link: https://alifsemi.com/support/kits/ensemble-e8devkit/
+        type: website
+    - resource:
+        title: Alif SDK pull request 879
+        link: https://github.com/alifsemi/sdk-alif/pull/879
+        type: website
+    - resource:
+        title: Ethos-U core driver multi-variant merge
+        link: https://gitlab.arm.com/artificial-intelligence/ethos-u/ethos-u-core-driver/-/commit/b7cd193afde80afe8bbae9a26d2ca6586554f054
+        type: website
+    - resource:
+        title: ExecuTorch Arm Ethos-U NPU backend tutorial
+        link: https://docs.pytorch.org/executorch/stable/tutorial-arm-ethos-u.html
+        type: documentation
+    - resource:
+        title: Run image classification on an Alif Ensemble E8 DevKit using ExecuTorch and Ethos-U85
+        link: /learning-paths/embedded-and-microcontrollers/alif-image-classification/
+        type: documentation
+    - resource:
+        title: Dual-NPU live vision sample source
+        link: https://github.com/varunchariArm/sdk-alif/tree/dual-npu-main-integration/samples/modules/executorch/dual_npu_vision
+        type: website
+
+### FIXED, DO NOT MODIFY
+# ================================================================================
+weight: 1
+layout: "learningpathall"
+learning_path_main_page: "yes"
+---

content/learning-paths/embedded-and-microcontrollers/alif-dual-npu-vision/_next-steps.md

@@ -0,0 +1,8 @@
+---
+# ================================================================================
+#       FIXED, DO NOT MODIFY THIS FILE
+# ================================================================================
+weight: 21
+title: "Next Steps"
+layout: "learningpathall"
+---

content/learning-paths/embedded-and-microcontrollers/alif-dual-npu-vision/build-application.md

@@ -0,0 +1,106 @@
+---
+title: Build the dual-NPU application
+description: Configure the Zephyr overlays and build the firmware and model payload for the E8 high-performance core.
+weight: 4
+layout: "learningpathall"
+---
+
+The sample includes compiled PTE models for Ethos-U55 and Ethos-U85. It also includes a startup image that verifies both inference pipelines before the application starts the live camera.
+
+## Define the build inputs
+
+From the west workspace root, activate the Python environment and define the sample paths:
+
+```bash
+cd $HOME/alif-dual-npu
+source .venv/bin/activate
+APP=$PWD/sdk-alif/samples/modules/executorch/dual_npu_vision
+OD=$PWD/sdk-alif/samples/modules/tflite-micro/alif_object_detection
+MODULES=$(west list -f '{abspath}' | grep -v '/modules/lib/executorch$' | paste -sd';' -)
+MODULES="$MODULES;$PWD/modules/lib/executorch;$PWD/modules/ethos-u-core-driver-src"
+```
+
+The overlay order matters. The final `isp_route.overlay` file disables the CPI memory endpoint and routes camera frames exclusively through the ISP.
+
+## Build the firmware
+
+Run the complete build command:
+
+```bash
+west build \
+  -b alif_e8_dk/ae822fa0e5597xx0/rtss_hp \
+  -d build-dual-npu-vision \
+  "$APP" --pristine -- \
+  -DZEPHYR_MODULES="$MODULES" \
+  -DPYTHON_EXECUTABLE="$PWD/.venv-executorch/bin/python" \
+  -DPython3_EXECUTABLE="$PWD/.venv-executorch/bin/python" \
+  -DCMSIS_NN_LOCAL_PATH="$PWD/modules/cmsis-nn-src" \
+  -DDTC_OVERLAY_FILE="$APP/boards/dual_npu_e8.overlay;$OD/boards/alif_e8_dk_ae822fa0e5597xx0_rtss_hp.overlay;$OD/serial_camera.overlay;$OD/serial_camera_mt9m114.overlay;$OD/serial_camera_isp.overlay;$OD/serial_camera_mt9m114_isp.overlay;$APP/isp_route.overlay" \
+  -DOVERLAY_CONFIG="$APP/vision.conf"
+```
+
+A successful build ends with messages showing that Zephyr linked the ELF file and generated the binary.
+
+Confirm that the three expected outputs exist:
+
+```bash
+ls -lh build-dual-npu-vision/zephyr/zephyr.bin \
+  build-dual-npu-vision/zephyr/zephyr.elf \
+  build-dual-npu-vision/model_assets.bin
+```
+
+The files have these roles:
+
+| File | Purpose |
+| --- | --- |
+| `zephyr.bin` | Cortex-M55 Zephyr application |
+| `zephyr.elf` | Symbols and debug information |
+| `model_assets.bin` | U85 PTE, U55 PTE, startup image, and ImageNet labels |
+
+The checked-in PTE files let you build this Learning Path without regenerating models. The original artifacts were generated on a Linux development host, but the model inputs are public and do not depend on that host.
+
+Create a directory for the source weights:
+
+```bash
+cd $HOME/alif-dual-npu
+mkdir -p model-weights
+```
+
+Download the trained [SSD-Slim int8 model](https://github.com/emza-vs/ModelZoo/blob/59fcdb2aab865a8a8d93a9d419b3c5490a5508e4/Models/Object_detection/SSD/ssd_slim_120x160x1_v1_int8.tflite) and the official [torchvision MobileNetV2 checkpoint](https://download.pytorch.org/models/mobilenet_v2-7ebf99e0.pth). The SSD URL is pinned to the commit that added the model:
+
+```bash
+curl -L \
+  https://raw.githubusercontent.com/emza-vs/ModelZoo/59fcdb2aab865a8a8d93a9d419b3c5490a5508e4/Models/Object_detection/SSD/ssd_slim_120x160x1_v1_int8.tflite \
+  -o model-weights/ssd_slim_120x160x1_v1_int8.tflite
+curl -L \
+  https://download.pytorch.org/models/mobilenet_v2-7ebf99e0.pth \
+  -o model-weights/mobilenet_v2-7ebf99e0.pth
+```
+
+Verify both downloads before using them:
+
+```bash
+echo "64fcc31aa517798d0e798551418c85bc0a5ed03a75c45c4e47fc7ee41e5ea51f  model-weights/ssd_slim_120x160x1_v1_int8.tflite" | shasum -a 256 -c -
+echo "7ebf99e03e254b273379b23edca7ec0da9f48273b23a332b93c1c99d49e86e8f  model-weights/mobilenet_v2-7ebf99e0.pth" | shasum -a 256 -c -
+```
+
+The SSD source repository does not publish a PyTorch checkpoint. Install TensorFlow in the ExecuTorch Python environment, then use the sample's importer to convert the trained, quantized constants into the common PyTorch checkpoint:
+
+```bash
+.venv-executorch/bin/python -m pip install tensorflow==2.20.0
+.venv-executorch/bin/python \
+  "$APP/tools/import_ssd_slim_tflite.py" \
+  --source model-weights/ssd_slim_120x160x1_v1_int8.tflite \
+  --output model-weights/ssd_slim_common.pth
+```
+
+Generate both PTE files with the Alif Vela configuration included in the SDK:
+
+```bash
+"$APP/tools/generate_pte_models.sh" \
+  "$PWD/sdk-alif/samples/modules/executorch/ensemble_vela.ini" \
+  "$PWD/model-weights/ssd_slim_common.pth" \
+  "$PWD/model-weights/mobilenet_v2-7ebf99e0.pth"
+```
+
+The script emits `comparable_ssd_slim_u55.pte` and `mobilenet_v2_imagenet_u85.pte` in the sample's `models` directory. The export uses PT2E quantization, deterministic representative inputs, the ExecuTorch Ethos-U partitioner, and Vela. Both programs expose int8 tensors and require complete delegation, with no Cortex-M fallback operators. The SSD checkpoint is produced by the supplied offline weight-import tool. Both deployed files are native ExecuTorch PTE programs.