From 90ecf88b7d307fe4b9bcb0d45978c364b986ec7f Mon Sep 17 00:00:00 2001 From: Andrea Lacava Date: Fri, 9 Oct 2026 15:09:07 -0400 Subject: [PATCH 1/4] build: install libe3 like a regular library, the binding through pip `build_libe3 --install --enable-swig` installed the Python module into the site-packages of whatever interpreter CMake found, regardless of --prefix: the system one unless the right venv was active. Without root that install failed, and because the module's rules ran before the library's own, the failure stopped the install before liblibe3, the headers and libe3.pc were written; consumers then reported "Package 'libe3' not found". A library install now writes only the library. `cmake --install` installs the module only when -DLIBE3_PYTHON_INSTALL_DIR is given (CI's staged install keeps working), and its rules are included after libe3Install, so they cannot stop the library install. The binding is meant to be installed with pip, built against the installed library (next commit). The SWIG module definition moves to cmake/libe3SwigModule.cmake so the in-tree build and the pip build compile it the same way; it links Python3::Module (not libpython) when CMake has it. build_libe3: - --all-encodings enables ASN.1, JSON and Protocol Buffers, what a gNB needs to serve any dApp; - --install uses sudo only when the prefix is not writable, refreshes the linker cache after an install into /usr or /usr/local, and prints the PKG_CONFIG_PATH line for any other prefix; - the CMake check runs after --install-deps, which is what installs CMake on a fresh machine: before, `./build_libe3 -I` stopped with "CMake is required" on exactly the machine it is meant for. The README gives one install recipe, the one OAI and the dApp library point to. Checked in a fresh ubuntu:24.04 container as a user with sudo: -I on a machine without CMake, --all-encodings --install into /usr/local (one sudo, ldconfig run), pkg-config and find_package find it with no environment variable, nothing is written into any Python site-packages, and the in-tree --enable-swig build passes `ctest -L swig`. Assisted-by: Claude:claude-opus-5-5 --- CMakeLists.txt | 5 ++- README.md | 33 +++++++++++--- build_libe3 | 71 ++++++++++++++++++++++++------- cmake/libe3SWIG.cmake | 85 +++++++++---------------------------- cmake/libe3SwigModule.cmake | 55 ++++++++++++++++++++++++ swig/README.md | 21 +++++---- 6 files changed, 174 insertions(+), 96 deletions(-) create mode 100644 cmake/libe3SwigModule.cmake diff --git a/CMakeLists.txt b/CMakeLists.txt index 717ed63c..21b4d0b1 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -75,13 +75,16 @@ include(libe3Targets) include(libe3Tests) include(libe3Examples) include(libe3Docs) -include(libe3SWIG) # ============================================================================ # Installation # ============================================================================ include(libe3Install) +# After libe3Install on purpose: install rules run in the order they are defined, so a failure +# to write the Python module must not stop the library and libe3.pc from being installed. +include(libe3SWIG) + # ============================================================================ # Summary # ============================================================================ diff --git a/README.md b/README.md index c31034d6..e97f671a 100644 --- a/README.md +++ b/README.md @@ -131,6 +131,28 @@ cmake .. -DCMAKE_BUILD_TYPE=Release make -j$(nproc) ``` +### Installing libe3 + +This is the recipe the OAI E3 agent and the dApp library point to. It installs libe3 like any other +library, once and with `sudo`, into `/usr/local`; after that nothing that builds against it needs `sudo` +or extra environment variables. Every wire encoding is built, since a gNB has to serve whichever one its +dApp uses: + +```bash +git clone --branch https://github.com/wineslab/libe3 && cd libe3 +./build_libe3 -I # build dependencies (sudo), once +./build_libe3 --all-encodings --install # -> /usr/local, asks for sudo, refreshes the linker cache +pkg-config --modversion libe3 +``` + +- C and C++ projects (the OAI gNB) find it through `pkg-config` or `find_package(libe3)`, with no setup. +- The Python binding is a separate package, `libe3py`, installed with pip into any virtual environment, + without root: `pip install libe3py==` (or `pip install .` from this checkout). It is built + against the installed libe3 and must be the same version; the dApp library depends on it, so + `pip install dapps` brings it in. +- To install somewhere else, `--prefix DIR` (no `sudo` if `DIR` is yours); then `PKG_CONFIG_PATH` has to + include `DIR/lib/pkgconfig`, and pip needs `--config-settings=cmake.define.CMAKE_PREFIX_PATH=DIR`. + ### Build Options | Option | Default | Description | @@ -492,10 +514,9 @@ libe3 ships an optional SWIG-generated Python binding so the same C++ library ca # Install SWIG and Python development headers (Ubuntu) sudo apt-get install -y swig python3-dev -# Build + install into the active interpreter's site-packages (activate your venv first) -./build_libe3 --install --enable-swig \ - --cmake-opt "-DLIBE3_ENABLE_ASN1=ON -DLIBE3_ENABLE_JSON=ON" -python3 -c "import libe3py; print('libe3py OK')" +# Install libe3 first (see "Installing libe3"), then the binding, in a virtual environment +pip install libe3py # or, from this checkout: pip install . +python3 -c "import libe3py; print('libe3py OK', libe3py.__version__)" # Or, build only (no install) and run the smoke test (CTest label "swig") cmake -S . -B build -DLIBE3_ENABLE_SWIG=ON @@ -510,8 +531,8 @@ the full **`DAppSession`** dApp seam — a complete lifecycle (`start` / timeout_ms)`, designed for the sub-millisecond / high-throughput E3AP path (GIL released during blocking calls). SM payloads cross as native `bytes`; per-SM encoding stays in Python so existing Python SM implementations work -unchanged. `build_libe3 --install --enable-swig` installs `_libe3py.so` + -`libe3py.py` into `Python3_SITEARCH`. +unchanged. The binding is built by pip against the installed libe3 (`python/CMakeLists.txt`), links it +statically, and refuses an installed libe3 of another version. See **[`swig/README.md`](swig/README.md)** for the architecture, rationale, and a full Python usage example, and [`swig/e3_dapp_session.hpp`](swig/e3_dapp_session.hpp) diff --git a/build_libe3 b/build_libe3 index c31e2563..4d8a3b7c 100755 --- a/build_libe3 +++ b/build_libe3 @@ -111,6 +111,8 @@ Features: --enable-json Enable JSON encoding support (default: ON) --disable-json Disable JSON encoding --enable-protobuf Enable Protocol Buffers encoding support (default: OFF) + --all-encodings Enable ASN.1, JSON and Protocol Buffers (what a gNB that has to serve + any dApp needs; same as the three --enable-* options) --disable-protobuf Disable Protocol Buffers encoding --enable-sctp Enable the SCTP POSIX transport (default: OFF). Needs the SCTP headers: libsctp-dev (Debian/Ubuntu), @@ -118,9 +120,10 @@ Features: --install-deps does NOT install them; SCTP is optional and the deployed E3 links are IPC or TCP. --disable-sctp Disable the SCTP transport - --enable-swig Build + install the Python (SWIG) bindings, libe3py - (default: OFF). With --install, installs libe3py into - the active interpreter's site-packages. + --enable-swig Build the Python (SWIG) binding, libe3py, in build/swig to develop and + test it (default: OFF). It is not installed with the library: install + it with pip (pip install libe3py, or pip install . in this checkout) + after installing libe3. --disable-swig Disable the Python (SWIG) bindings --enable-tests Build unit tests (default: ON) --disable-tests Disable unit tests @@ -139,9 +142,11 @@ Sanitizers: Installation: ------------- -I, --install-deps Install required dependencies (requires sudo) - --install Install library after building + --install Install library after building (uses sudo only if the prefix is not + writable by you, then refreshes the linker cache with ldconfig) --uninstall Uninstall library using install manifest - --prefix DIR Installation prefix (default: /usr/local) + --prefix DIR Installation prefix (default: /usr/local). A prefix in your home, such as + ~/.local/libe3, needs no sudo; put its lib/pkgconfig on PKG_CONFIG_PATH. Testing: -------- @@ -488,11 +493,41 @@ run_tests() { install_library() { local build_path="$SCRIPT_DIR/$BUILD_DIR" cd "$build_path" - - echo_info "Installing libe3..." - sudo cmake --install . - + + local prefix + prefix="$(sed -n 's/^CMAKE_INSTALL_PREFIX:PATH=//p' CMakeCache.txt | head -n1)" + prefix="${prefix:-/usr/local}" + + # sudo only when the prefix (or the closest existing parent) is not writable by this user + local probe="$prefix" + while [ ! -e "$probe" ] && [ "$probe" != "/" ]; do probe="$(dirname "$probe")"; done + local runner=() + if [ ! -w "$probe" ]; then + runner=(sudo) + fi + + echo_info "Installing libe3 into $prefix${runner:+ (with sudo)}..." + "${runner[@]}" cmake --install . + + # A library in a directory the dynamic linker searches (/usr/local/lib) is only found at run + # time after the linker cache is refreshed. + if [ "$(uname -s)" = "Linux" ] && command -v ldconfig >/dev/null 2>&1 \ + && { [ "$prefix" = "/usr" ] || [ "$prefix" = "/usr/local" ]; }; then + "${runner[@]}" ldconfig || echo_warning "ldconfig failed; run it once by hand (sudo ldconfig)" + fi + echo_success "Installation completed!" + case "$prefix" in + /usr|/usr/local) ;; + *) + echo_info "libe3 is not in a system path. Before building something against it, run:" + echo " export PKG_CONFIG_PATH=$prefix/lib/pkgconfig:\$PKG_CONFIG_PATH" + ;; + esac + if [ "$ENABLE_SWIG" -eq 1 ]; then + echo_info "The Python binding is not installed with the library. In your virtual environment run:" + echo " pip install libe3py==$(cat "$SCRIPT_DIR/VERSION") # or: pip install $SCRIPT_DIR" + fi } uninstall_library() { @@ -676,6 +711,12 @@ main() { ENABLE_PROTOBUF=1 shift ;; + --all-encodings) + ENABLE_ASN1=1 + ENABLE_JSON=1 + ENABLE_PROTOBUF=1 + shift + ;; --enable-sctp) ENABLE_SCTP=1 shift @@ -823,12 +864,6 @@ main() { esac fi - # Check for required tools - if ! check_command cmake; then - echo_error "CMake is required but not found. Install with: --install-deps" - exit 1 - fi - # Install dependencies if requested if [ $INSTALL_DEPS -eq 1 ]; then install_dependencies @@ -838,6 +873,12 @@ main() { echo_info "Dependencies installed. Please try to build libe3." exit 0 fi + + # Checked after --install-deps, which is what installs it on a fresh machine + if ! check_command cmake; then + echo_error "CMake is required but not found. Install with: --install-deps" + exit 1 + fi # Clean if requested if [ $CLEAN_ALL -eq 1 ]; then diff --git a/cmake/libe3SWIG.cmake b/cmake/libe3SWIG.cmake index b162fb7b..f68c6fb0 100644 --- a/cmake/libe3SWIG.cmake +++ b/cmake/libe3SWIG.cmake @@ -6,11 +6,12 @@ # the minimal E3Agent view it wraps the batched dApp session # (swig/e3_dapp_session.{hpp,cpp}) — the low-latency DAppSession + E3Event. # -# When LIBE3_ENABLE_SWIG is on and the library is installed (build_libe3 -# --install --enable-swig), the module (_libe3py.so + libe3py.py) is installed -# into the active interpreter's site-packages (Python3_SITEARCH) so consumers -# can simply `import libe3py`. Activate the target venv before installing so the -# module lands in the environment that will import it. +# This in-tree build is for development and the swig tests (build/swig). Users +# install the binding with pip (`pip install libe3py`, or `pip install .` from +# this checkout), which builds the same module against the installed libe3; see +# python/CMakeLists.txt. `cmake --install` installs the module only when +# LIBE3_PYTHON_INSTALL_DIR is set, so a library install never writes into a +# Python's site-packages behind the user's back. # # SPDX-FileCopyrightText: Copyright (c) 2026 Northeastern University # SPDX-License-Identifier: Apache-2.0 @@ -19,70 +20,24 @@ if(NOT LIBE3_ENABLE_SWIG) return() endif() -find_package(SWIG 4.0 REQUIRED COMPONENTS python) -find_package(Python3 REQUIRED COMPONENTS Interpreter Development) - -include(UseSWIG) +include(libe3SwigModule) set(LIBE3_SWIG_OUTPUT_DIR "${CMAKE_BINARY_DIR}/swig") -file(MAKE_DIRECTORY "${LIBE3_SWIG_OUTPUT_DIR}") - -set_property(SOURCE "${CMAKE_CURRENT_SOURCE_DIR}/swig/libe3.i" PROPERTY CPLUSPLUS ON) -set_property(SOURCE "${CMAKE_CURRENT_SOURCE_DIR}/swig/libe3.i" PROPERTY - INCLUDE_DIRECTORIES - "${CMAKE_CURRENT_SOURCE_DIR}/include" - "${CMAKE_CURRENT_SOURCE_DIR}/swig") -set_property(SOURCE "${CMAKE_CURRENT_SOURCE_DIR}/swig/libe3.i" PROPERTY - SWIG_FLAGS - "-I${CMAKE_CURRENT_SOURCE_DIR}/include" - "-I${CMAKE_CURRENT_SOURCE_DIR}/swig" - # latrec.h refuses to compile without CLOCK_MONOTONIC declared (see - # its own #error), which SWIG's preprocessor never sees for real: it - # does not walk glibc's actual / chain the way - # the real compiler does afterwards on the generated wrapper. The - # numeric value is never evaluated by SWIG, only its definedness, so - # a placeholder satisfies the check without SWIG needing to resolve - # the real one. - "-DCLOCK_MONOTONIC=1") - -swig_add_library(libe3py - TYPE SHARED - LANGUAGE python - SOURCES - "${CMAKE_CURRENT_SOURCE_DIR}/swig/libe3.i" - "${CMAKE_CURRENT_SOURCE_DIR}/swig/e3_dapp_session.cpp" +libe3_add_python_module(libe3py + SOURCE_DIR "${CMAKE_CURRENT_SOURCE_DIR}" OUTPUT_DIR "${LIBE3_SWIG_OUTPUT_DIR}" -) - -set_target_properties(libe3py PROPERTIES - LIBRARY_OUTPUT_DIRECTORY "${LIBE3_SWIG_OUTPUT_DIR}" -) + LINK libe3::libe3 + INCLUDE "${CMAKE_CURRENT_SOURCE_DIR}/include") -target_include_directories(libe3py PRIVATE - ${CMAKE_CURRENT_SOURCE_DIR}/include - ${CMAKE_CURRENT_SOURCE_DIR}/swig - ${Python3_INCLUDE_DIRS} -) - -target_link_libraries(libe3py PRIVATE libe3::libe3 ${Python3_LIBRARIES}) - -# Install the Python module into the active interpreter's site-packages so -# `import libe3py` works after `build_libe3 --install --enable-swig`. The .so -# target name from UseSWIG is `libe3py` (file `_libe3py.so`); the generated -# libe3py.py shim lives in LIBE3_SWIG_OUTPUT_DIR after the build. -# -# LIBE3_PYTHON_INSTALL_DIR defaults to the interpreter's site-packages, which is -# an ABSOLUTE path — so CMAKE_INSTALL_PREFIX / --prefix does NOT relocate the -# Python module (see swig/README.md). Override this var to stage it elsewhere -# (CI does this to test the install without touching system site-packages). -set(LIBE3_PYTHON_INSTALL_DIR "${Python3_SITEARCH}" CACHE PATH - "Install directory for the libe3py Python module") -install(TARGETS libe3py - LIBRARY DESTINATION "${LIBE3_PYTHON_INSTALL_DIR}" -) -install(FILES "${LIBE3_SWIG_OUTPUT_DIR}/libe3py.py" - DESTINATION "${LIBE3_PYTHON_INSTALL_DIR}" -) +set(LIBE3_PYTHON_INSTALL_DIR "" CACHE PATH + "Install the libe3py module here with the library (empty: do not; users install it with pip)") +if(LIBE3_PYTHON_INSTALL_DIR) + install(TARGETS libe3py LIBRARY DESTINATION "${LIBE3_PYTHON_INSTALL_DIR}") + install(FILES "${LIBE3_SWIG_OUTPUT_DIR}/libe3py.py" DESTINATION "${LIBE3_PYTHON_INSTALL_DIR}") + message(STATUS "libe3py will be installed with the library into ${LIBE3_PYTHON_INSTALL_DIR}") +else() + message(STATUS "libe3py is built in ${LIBE3_SWIG_OUTPUT_DIR} and not installed; install it with pip (pip install .)") +endif() # Register a CTest smoke test that imports the module and constructs a # minimal E3Config + E3Agent. Only added when LIBE3_BUILD_TESTS is also on. diff --git a/cmake/libe3SwigModule.cmake b/cmake/libe3SwigModule.cmake new file mode 100644 index 00000000..29660a6e --- /dev/null +++ b/cmake/libe3SwigModule.cmake @@ -0,0 +1,55 @@ +# The libe3py SWIG module, shared by the in-tree build (libe3SWIG.cmake) and the +# pip build against an installed libe3 (python/CMakeLists.txt), so the two +# cannot drift apart. +# +# libe3_add_python_module( SOURCE_DIR OUTPUT_DIR LINK [INCLUDE ...]) +# +# Produces /_libe3py.so and /libe3py.py. The module links +# libe3 statically, so it needs no liblibe3.so at run time. +# +# SPDX-FileCopyrightText: Copyright (c) 2026 Northeastern University +# SPDX-License-Identifier: Apache-2.0 + +include_guard(GLOBAL) + +find_package(SWIG 4.0 REQUIRED COMPONENTS python) +if(CMAKE_VERSION VERSION_LESS 3.18) + find_package(Python3 REQUIRED COMPONENTS Interpreter Development) +else() + find_package(Python3 REQUIRED COMPONENTS Interpreter Development.Module) +endif() +include(UseSWIG) + +function(libe3_add_python_module target) + cmake_parse_arguments(ARG "" "SOURCE_DIR;OUTPUT_DIR;LINK" "INCLUDE" ${ARGN}) + set(_i "${ARG_SOURCE_DIR}/swig/libe3.i") + file(MAKE_DIRECTORY "${ARG_OUTPUT_DIR}") + set_property(SOURCE "${_i}" PROPERTY CPLUSPLUS ON) + set(_swig_inc "${ARG_SOURCE_DIR}/swig" ${ARG_INCLUDE}) + set_property(SOURCE "${_i}" PROPERTY INCLUDE_DIRECTORIES ${_swig_inc}) + set(_swig_flags "") + foreach(_d IN LISTS _swig_inc) + list(APPEND _swig_flags "-I${_d}") + endforeach() + # latrec.h refuses to compile without CLOCK_MONOTONIC declared (see its own + # #error), which SWIG's preprocessor never sees for real: it does not walk + # glibc's actual / chain the way the real compiler does + # afterwards on the generated wrapper. The numeric value is never evaluated + # by SWIG, only its definedness, so a placeholder satisfies the check. + list(APPEND _swig_flags "-DCLOCK_MONOTONIC=1") + set_property(SOURCE "${_i}" PROPERTY SWIG_FLAGS ${_swig_flags}) + + swig_add_library(${target} + TYPE SHARED + LANGUAGE python + SOURCES "${_i}" "${ARG_SOURCE_DIR}/swig/e3_dapp_session.cpp" + OUTPUT_DIR "${ARG_OUTPUT_DIR}") + set_target_properties(${target} PROPERTIES LIBRARY_OUTPUT_DIRECTORY "${ARG_OUTPUT_DIR}") + target_include_directories(${target} PRIVATE "${ARG_SOURCE_DIR}/swig" ${ARG_INCLUDE}) + if(TARGET Python3::Module) + target_link_libraries(${target} PRIVATE ${ARG_LINK} Python3::Module) + else() + target_include_directories(${target} PRIVATE ${Python3_INCLUDE_DIRS}) + target_link_libraries(${target} PRIVATE ${ARG_LINK} ${Python3_LIBRARIES}) + endif() +endfunction() diff --git a/swig/README.md b/swig/README.md index 94721860..ad8d28fc 100644 --- a/swig/README.md +++ b/swig/README.md @@ -103,19 +103,22 @@ while True: ## Build & install -The module is opt-in (`LIBE3_ENABLE_SWIG=OFF` by default). Build and install it -into the **active interpreter's** site-packages (activate your venv first): +Install libe3 once (`./build_libe3 --all-encodings --install`, see the main README), then the binding +with pip, in any virtual environment: ```bash -./build_libe3 --install --enable-swig \ - --cmake-opt "-DLIBE3_ENABLE_ASN1=ON -DLIBE3_ENABLE_JSON=ON" -python3 -c "import libe3py; print('libe3py OK')" +pip install libe3py # or, from the libe3 checkout: pip install . +python -c "import libe3py; print('libe3py OK', libe3py.__version__)" ``` -Requires `swig >= 4.0` and Python development headers. The install drops -`_libe3py.so` + `libe3py.py` into `Python3_SITEARCH` — note that this is an -absolute path, so `--prefix` does **not** relocate the Python module; override -`-DLIBE3_PYTHON_INSTALL_DIR=` to stage it elsewhere. +pip builds the module from these sources against the installed libe3 (`python/CMakeLists.txt`); it +needs `swig >= 4.0`, the Python headers and a C++17 compiler (`./build_libe3 -I` installs them), and it +stops with an error if the installed libe3 is not the same version. For a libe3 outside `/usr` and +`/usr/local`: `pip install libe3py --config-settings=cmake.define.CMAKE_PREFIX_PATH=`. + +For development, `./build_libe3 --enable-swig` builds the module in `build/swig` (run the smoke test with +`PYTHONPATH=build/swig python3 tests/test_swig_smoke.py`); `cmake --install` installs it only if +`-DLIBE3_PYTHON_INSTALL_DIR=` is given. ## Service-model definitions From d52171ad74aa709cba1cc0c6048e69fb280c8af6 Mon Sep 17 00:00:00 2001 From: Andrea Lacava Date: Fri, 9 Oct 2026 15:09:07 -0400 Subject: [PATCH 2/4] python: publish the binding as libe3py, built against the installed libe3 The Python binding could only be had by building libe3 with SWIG and installing it into the right interpreter. It is now a package, libe3py, that pip builds against the libe3 installed on the machine, in any virtual environment and without root: ./build_libe3 --all-encodings --install # once, sudo pip install libe3py # or pip install . from a checkout - pyproject.toml (scikit-build-core) takes its version from VERSION and builds python/CMakeLists.txt; the sdist carries only the SWIG sources, the shared module definition and the license files. - python/CMakeLists.txt finds the installed library with find_package(libe3 CONFIG), so the module links libe3 statically as before, with the feature macros (encodings, latrec) libe3 was built with, and needs no liblibe3.so at run time. It stops unless the installed libe3 is the same version as the package, which keeps library and binding in step. - tag-and-release builds the sdist on every release, publishes it to PyPI and then TestPyPI (trusted publishing, environments pypi and testpypi, as the dApp library does), and attaches it to the GitHub release. sdist only: a plain linux_x86_64 wheel is rejected by PyPI. - pr-tests' swig job pip-installs the sdist against its staged install and runs the smoke test on the result. Checked in a fresh ubuntu:24.04 container: the sdist builds; pip installs it in a venv without sudo against libe3 in /usr/local; the module has no liblibe3 or libpython run-time dependency; tests/test_swig_smoke.py passes on it; `pip install .` from the checkout works; a package of another version is refused with the mismatch message. actionlint reports nothing new on either workflow. Not checked: the publishing jobs (they need a release), and a libe3 outside the standard prefixes through CMAKE_PREFIX_PATH (CI covers it). Assisted-by: Claude:claude-opus-5-5 --- .github/workflows/pr-tests.yml | 13 +++++ .github/workflows/tag-and-release.yml | 76 ++++++++++++++++++++++++++- pyproject.toml | 44 ++++++++++++++++ python/CMakeLists.txt | 59 +++++++++++++++++++++ python/README.md | 22 ++++++++ 5 files changed, 213 insertions(+), 1 deletion(-) create mode 100644 pyproject.toml create mode 100644 python/CMakeLists.txt create mode 100644 python/README.md diff --git a/.github/workflows/pr-tests.yml b/.github/workflows/pr-tests.yml index b348e388..197b182d 100644 --- a/.github/workflows/pr-tests.yml +++ b/.github/workflows/pr-tests.yml @@ -316,6 +316,19 @@ jobs: cmake --build build-consume-pkgconfig -j $(nproc) ./build-consume-pkgconfig/consume + # The way users get the binding: the sdist that the release publishes, + # built by pip against the installed library. + - name: pip-install the libe3py sdist against the installed library + if: needs.changes.outputs.code == 'true' + run: | + set -euo pipefail + python3 -m venv "$RUNNER_TEMP/venv" + "$RUNNER_TEMP/venv/bin/pip" install build + "$RUNNER_TEMP/venv/bin/python" -m build --sdist -o "$RUNNER_TEMP/dist" . + "$RUNNER_TEMP/venv/bin/pip" install "$RUNNER_TEMP"/dist/libe3py-*.tar.gz \ + --config-settings=cmake.define.CMAKE_PREFIX_PATH="$PWD/stage" + "$RUNNER_TEMP/venv/bin/python" tests/test_swig_smoke.py + # NOTE: the MPMC Queue Benchmark moved to .github/workflows/mpmc-bench.yml, # where it is gated on changes to include/libe3/mpmc_queue.hpp so it only # runs when the queue's core file changes. It isn't in the required-checks diff --git a/.github/workflows/tag-and-release.yml b/.github/workflows/tag-and-release.yml index b137700d..f282d71c 100644 --- a/.github/workflows/tag-and-release.yml +++ b/.github/workflows/tag-and-release.yml @@ -253,12 +253,80 @@ jobs: name: libe3-docs path: libe3-docs-${{ needs.create-tag.outputs.version }}.zip + build-python-sdist: + name: Build the libe3py source distribution + needs: create-tag + runs-on: ubuntu-latest + if: ${{ needs.create-tag.outputs.tag_created == 'true' && github.repository == 'wineslab/libe3' }} + timeout-minutes: 10 + steps: + - uses: actions/checkout@v5 + with: + persist-credentials: false + - uses: actions/setup-python@v6 + with: + python-version: "3.12" + - name: Build the sdist + # sdist only: the module is compiled on the user's machine against the + # libe3 installed there (python/CMakeLists.txt), and a plain + # linux_x86_64 wheel would be rejected by PyPI anyway. Building the + # sdist compiles nothing. + run: | + python3 -m pip install build + python3 -m build --sdist + ls dist/libe3py-${{ needs.create-tag.outputs.version }}.tar.gz + - uses: actions/upload-artifact@v6 + with: + name: libe3py-sdist + path: dist/ + + publish-to-pypi: + name: Publish libe3py to PyPI + needs: + - build-python-sdist + - create-tag + runs-on: ubuntu-latest + if: ${{ needs.create-tag.outputs.tag_created == 'true' && github.repository == 'wineslab/libe3' }} + environment: + name: pypi + url: https://pypi.org/p/libe3py + permissions: + id-token: write + steps: + - uses: actions/download-artifact@v7 + with: + name: libe3py-sdist + path: dist/ + - uses: pypa/gh-action-pypi-publish@release/v1 + + publish-to-testpypi: + name: Publish libe3py to TestPyPI + needs: + - publish-to-pypi + - create-tag + runs-on: ubuntu-latest + if: ${{ needs.create-tag.outputs.tag_created == 'true' && github.repository == 'wineslab/libe3' }} + environment: + name: testpypi + url: https://test.pypi.org/p/libe3py + permissions: + id-token: write + steps: + - uses: actions/download-artifact@v7 + with: + name: libe3py-sdist + path: dist/ + - uses: pypa/gh-action-pypi-publish@release/v1 + with: + repository-url: https://test.pypi.org/legacy/ + github-release: - name: Upload Debian package to GitHub Release + name: Upload Debian packages and the libe3py sdist to GitHub Release needs: - build-deb-amd64 - build-deb-arm64 - build-docs + - build-python-sdist - create-tag runs-on: ubuntu-latest if: ${{ needs.create-tag.outputs.tag_created == 'true' }} @@ -284,6 +352,12 @@ jobs: name: libe3-docs path: dist/ + - name: Download the libe3py sdist + uses: actions/download-artifact@v7 + with: + name: libe3py-sdist + path: dist/ + - name: Create GitHub Release and upload assets env: GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} diff --git a/pyproject.toml b/pyproject.toml new file mode 100644 index 00000000..799e9c4b --- /dev/null +++ b/pyproject.toml @@ -0,0 +1,44 @@ +# SPDX-FileCopyrightText: Copyright (c) 2026 Northeastern University +# SPDX-License-Identifier: Apache-2.0 +# +# libe3py: the Python binding of libe3, built against the libe3 installed on the +# machine (see python/CMakeLists.txt). Install libe3 first, then `pip install libe3py`. +[build-system] +requires = ["scikit-build-core>=0.8", "swig>=4.0"] +build-backend = "scikit_build_core.build" + +[project] +name = "libe3py" +dynamic = ["version"] +description = "Python binding of libe3, the O-RAN E3 interface library" +readme = "python/README.md" +requires-python = ">=3.10" +license = "Apache-2.0" +license-files = ["LICENSE", "NOTICE"] +authors = [{ name = "Northeastern University" }] +classifiers = [ + "Programming Language :: Python :: 3", + "Programming Language :: C++", + "Operating System :: POSIX :: Linux", +] + +[project.urls] +Homepage = "https://github.com/wineslab/libe3" +Documentation = "https://github.com/wineslab/libe3/blob/main/swig/README.md" + +[tool.scikit-build] +cmake.version = ">=3.18" +cmake.build-type = "Release" +cmake.source-dir = "python" +build-dir = "build/{wheel_tag}" +wheel.packages = [] +sdist.exclude = ["*"] +sdist.include = [ + "VERSION", "LICENSE", "NOTICE", "pyproject.toml", + "python/**", "swig/**", "cmake/libe3SwigModule.cmake", +] + +[tool.scikit-build.metadata.version] +provider = "scikit_build_core.metadata.regex" +input = "VERSION" +regex = "^(?P\\d+\\.\\d+\\.\\d+)" diff --git a/python/CMakeLists.txt b/python/CMakeLists.txt new file mode 100644 index 00000000..6622687b --- /dev/null +++ b/python/CMakeLists.txt @@ -0,0 +1,59 @@ +# libe3py, the Python binding of libe3, built by pip (scikit-build-core, see +# ../pyproject.toml) against the libe3 already installed on the machine: +# +# sudo ./build_libe3 --all-encodings --install # once +# pip install libe3py # or: pip install . from the checkout +# +# The module is compiled from the same SWIG sources as the in-tree build and links +# the installed static libe3, with the feature macros (encodings, latrec) that +# libe3 was built with. It refuses an installed libe3 of another version, so the +# binding and the library always match. +# +# SPDX-FileCopyrightText: Copyright (c) 2026 Northeastern University +# SPDX-License-Identifier: Apache-2.0 + +cmake_minimum_required(VERSION 3.18) + +set(LIBE3_SOURCE_DIR "${CMAKE_CURRENT_SOURCE_DIR}/..") +file(READ "${LIBE3_SOURCE_DIR}/VERSION" LIBE3PY_VERSION) +string(STRIP "${LIBE3PY_VERSION}" LIBE3PY_VERSION) + +project(libe3py VERSION ${LIBE3PY_VERSION} LANGUAGES CXX) +set(CMAKE_CXX_STANDARD 17) +set(CMAKE_CXX_STANDARD_REQUIRED ON) + +find_package(libe3 CONFIG QUIET) +if(NOT libe3_FOUND) + message(FATAL_ERROR + "libe3py ${LIBE3PY_VERSION} is built against an installed libe3, and none was found.\n" + "Install libe3 ${LIBE3PY_VERSION} first (https://github.com/wineslab/libe3):\n" + " git clone --branch ${LIBE3PY_VERSION} https://github.com/wineslab/libe3 && cd libe3\n" + " ./build_libe3 -I && ./build_libe3 --all-encodings --install\n" + "For a libe3 outside /usr and /usr/local, pass its prefix: " + "pip install libe3py --config-settings=cmake.define.CMAKE_PREFIX_PATH=") +endif() +if(NOT libe3_VERSION VERSION_EQUAL LIBE3PY_VERSION) + message(FATAL_ERROR + "The installed libe3 is ${libe3_VERSION} (${libe3_DIR}), but this is libe3py ${LIBE3PY_VERSION}.\n" + "Install the libe3py that matches it (pip install \"libe3py==${libe3_VERSION}\"), or install libe3 ${LIBE3PY_VERSION}.") +endif() +message(STATUS "Building libe3py ${LIBE3PY_VERSION} against libe3 ${libe3_VERSION} from ${libe3_DIR}") + +list(APPEND CMAKE_MODULE_PATH "${LIBE3_SOURCE_DIR}/cmake") +include(libe3SwigModule) + +get_target_property(_libe3_inc libe3::libe3 INTERFACE_INCLUDE_DIRECTORIES) +set(_out "${CMAKE_CURRENT_BINARY_DIR}/swig") +libe3_add_python_module(libe3py + SOURCE_DIR "${LIBE3_SOURCE_DIR}" + OUTPUT_DIR "${_out}" + LINK libe3::libe3 + INCLUDE ${_libe3_inc}) + +if(DEFINED SKBUILD_PLATLIB_DIR) + set(_dest "${SKBUILD_PLATLIB_DIR}") +else() + set(_dest ".") +endif() +install(TARGETS libe3py LIBRARY DESTINATION "${_dest}") +install(FILES "${_out}/libe3py.py" DESTINATION "${_dest}") diff --git a/python/README.md b/python/README.md new file mode 100644 index 00000000..17823f1a --- /dev/null +++ b/python/README.md @@ -0,0 +1,22 @@ +# libe3py + +Python binding of [libe3](https://github.com/wineslab/libe3), the C++ library that implements the O-RAN +E3 interface between a RAN node and its dApps. The [dApp library](https://github.com/wineslab/dApp-library) +(`pip install dapps`) uses it for E3AP; see +[`swig/README.md`](https://github.com/wineslab/libe3/blob/main/swig/README.md) for the API. + +libe3py is built when you install it, against the libe3 installed on the machine, and it must be the same +version. Install libe3 once: + +```bash +git clone --branch https://github.com/wineslab/libe3 && cd libe3 +./build_libe3 -I # build dependencies, swig and python3-dev included +./build_libe3 --all-encodings --install # into /usr/local, asks for sudo +``` + +then, in any virtual environment and without root: + +```bash +pip install libe3py== +python -c "import libe3py; print(libe3py.__version__)" +``` From 3ce706c7a3caca7f6ec3c6222504288499d1ab40 Mon Sep 17 00:00:00 2001 From: Andrea Lacava Date: Fri, 9 Oct 2026 15:09:15 -0400 Subject: [PATCH 3/4] release: 0.2.3 libe3 is installed as a regular library and its Python binding is the libe3py package on PyPI, built against it; build_libe3 -I works on a machine without CMake. Assisted-by: Claude:claude-opus-5-5 --- VERSION | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/VERSION b/VERSION index ee1372d3..71790396 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -0.2.2 +0.2.3 From 416c7d079aebca79cc143030ce0d3e4c367c9628 Mon Sep 17 00:00:00 2001 From: Andrea Lacava Date: Sat, 10 Oct 2026 11:07:37 -0400 Subject: [PATCH 4/4] chore(pr-template): keep the sections a reviewer reads, leave the checks to CI The template asked contributors to tick a mandatory test checklist, a CI checklist and a workflow confirmation. Every box restates something CI already enforces and reports in the single CI report comment (Release and Debug builds and tests, the benchmark, the commit policy: trailers, linear history, per-commit builds, SPDX), so the boxes added work without adding a check. Keep the summary, the type of change, the linked issue and the twin-repo coordination, and say in CONTRIBUTING that CI is what checks. Assisted-by: Claude:claude-opus-5-5 --- .github/PULL_REQUEST_TEMPLATE.md | 30 ------------------------------ CONTRIBUTING.md | 2 +- 2 files changed, 1 insertion(+), 31 deletions(-) diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md index 01373d2e..8c31769a 100644 --- a/.github/PULL_REQUEST_TEMPLATE.md +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -1,6 +1,5 @@ ## Summary @@ -26,29 +25,6 @@ PRs that leave the mandatory boxes unchecked will not be reviewed. Closes # -## Mandatory test checklist - -These mirror what CI (`.github/workflows/pr-tests.yml`) enforces. **All boxes must be ticked before review.** - -- [ ] `./build_libe3 -c -d build -j $(nproc) -r -t` passes (Release build + tests) -- [ ] `./build_libe3 -c -d build -j $(nproc) -g -t` passes (Debug build + tests) -- [ ] `cd build && ctest --output-on-failure` is clean -- [ ] MPMC queue benchmark (`./build/test_bench_mpmc_queue`) shows no regression vs `main` -- [ ] `VERSION` bumped per [SemVer](https://semver.org/) if the public API or ABI changed -- [ ] If public headers under `include/` were touched, `./build_libe3 --docs` renders without new Doxygen warnings -- [ ] If new build dependencies were added, they are installed by `./build_libe3 -I` (update the script if needed) -- [ ] If the `libe3.pc` interface changed, downstream consumers (`dApp-openairinterface5g`) still link cleanly - -## CI checklist - -CI posts a single `CI report` comment on this PR once every workflow has finished; it carries the -verdict, a per-workflow table and the benchmark/E2E detail. Confirm against that comment: - -- [ ] The report's verdict is green for the head commit -- [ ] `Unit Tests` is green (Debug + Release matrix on `ubuntu-latest`) -- [ ] `Commit policy` is green (trailers + linear history + each commit builds/tests independently) -- [ ] `MPMC Queue Benchmark` shows no regression (only runs when `include/libe3/mpmc_queue.hpp` changes) - ## Twin-repo coordination libe3 is paired with [`dapps`](https://github.com/wineslab/dApp-library) and [`dApp-openairinterface5g`](https://github.com/wineslab/dApp-openairinterface5g). **We do not accept patches that break or reduce compatibility with the twin repositories.** @@ -56,9 +32,3 @@ libe3 is paired with [`dapps`](https://github.com/wineslab/dApp-library) and [`d - [ ] This PR does not change the E3 wire protocol or public ABI, OR a paired PR exists in each affected twin repo (link below). Paired PR(s): - -## Workflow confirmation - -- [ ] My branch is a linear, fast-forward-able descendant of `main` (rebased if `main` moved), with no merge commits. (See `CONTRIBUTING.md` § Pull Request Process.) -- [ ] Every commit builds and passes tests on its own (atomic, `git bisect`-safe) with a descriptive message. -- [ ] I have read and followed `CONTRIBUTING.md`. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 1f4266dc..3ba591e0 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -58,7 +58,7 @@ The following are **mandatory** for every contribution. PRs that do not meet the ### Pull requests -- All PRs must use `.github/PULL_REQUEST_TEMPLATE.md` and complete every checklist item. +- All PRs must use `.github/PULL_REQUEST_TEMPLATE.md` and fill in every section; CI (below) is what checks the build, the tests and the commit policy. - The following CI workflows must be green on the latest commit before review: - **`Unit Tests`** (`.github/workflows/pr-tests.yml`) — builds and runs `ctest --output-on-failure` for both `Debug` and `Release` on `ubuntu-latest`, plus the integration, all-encodings, and SWIG jobs. - **`Commit policy`** (`.github/workflows/commit-trailers.yml`) — validates the AI-assistant trailer policy on every commit (see [AI assistants](#ai-assistants)), that the branch is a linear, fast-forward-able descendant of `main` (no merge commits), that every commit builds and passes tests on its own, and that every file carries an SPDX header (`reuse lint`, see [Licensing headers](#licensing-headers)).