MINIFICPP-2859 Extend Conan usage and integrate it in Windows and MacOS CI jobs#2212
MINIFICPP-2859 Extend Conan usage and integrate it in Windows and MacOS CI jobs#2212lordgamez wants to merge 6 commits into
Conversation
There was a problem hiding this comment.
Pull request overview
This PR extends MiNiFi C++’s Conan-based dependency workflow and integrates it into macOS and Windows CI, shifting more third-party dependency handling toward Conan recipes and consistent CMake target usage.
Changes:
- Add/extend Conan recipes for multiple third-party dependencies (e.g., RocksDB, open62541, magic_enum, couchbase) and wire them into the build.
- Refactor CMake integration to prefer
Get*modules and namespaced imported targets (e.g.,yaml-cpp::yaml-cpp,gsl-lite::gsl-lite, etc.). - Update bootstrap and CI workflows to run
conan installand incorporate Conan-specific compatibility handling.
Reviewed changes
Copilot reviewed 90 out of 93 changed files in this pull request and generated 9 comments.
Show a summary per file
| File | Description |
|---|---|
| thirdparty/rocksdb/config.yml | Updates RocksDB thirdparty version mapping and adds license header. |
| thirdparty/rocksdb/all/conanfile.py | Adjusts RocksDB Conan build options (e.g., disables tools, skips thirdparty). |
| thirdparty/rocksdb/all/conandata.yml | Adds license header and updates source metadata for RocksDB. |
| thirdparty/open62541/open62541.patch | Removes old patch location (moved under recipe patches). |
| thirdparty/open62541/config.yml | Adds thirdparty version mapping for open62541. |
| thirdparty/open62541/all/patches/open62541.patch | Centralizes open62541 patch used by Conan recipe/build. |
| thirdparty/open62541/all/conanfile.py | Adds open62541 Conan recipe (now using OpenSSL). |
| thirdparty/open62541/all/conandata.yml | Adds open62541 sources + patch metadata for Conan recipe. |
| thirdparty/magic-enum/config.yml | Adds thirdparty version mapping for magic_enum. |
| thirdparty/magic-enum/all/conanfile.py | Adds magic_enum Conan recipe. |
| thirdparty/magic-enum/all/conandata.yml | Adds magic_enum source metadata for Conan recipe. |
| thirdparty/couchbase/config.yml | Adds thirdparty version mapping for couchbase SDK. |
| thirdparty/couchbase/all/patches/use_fmt_instead_of_spdlog_fmt.patch | Patch to prefer fmt headers instead of spdlog-bundled fmt. |
| thirdparty/couchbase/all/patches/remove-thirdparty.patch | Patch to remove couchbase vendored spdlog/asio usage in favor of provided deps. |
| thirdparty/couchbase/all/patches/c++23_fixes.patch | Patch to address C++23 incomplete-type/unique_ptr include strictness. |
| thirdparty/couchbase/all/ConanThirdPartyDependencies.cmake | Adds a Conan-oriented dependency discovery module for couchbase build. |
| thirdparty/couchbase/all/conanfile.py | Adds couchbase C++ client Conan recipe, wiring deps and CMake options. |
| thirdparty/couchbase/all/conandata.yml | Adds couchbase sources + patch metadata for Conan recipe. |
| minifi-api/CMakeLists.txt | Switches gsl-lite linkage to namespaced target. |
| minifi_main/CMakeLists.txt | Updates linked dependency target names and argparse acquisition. |
| LICENSE | Removes libuvc and mbedTLS notices consistent with dependency changes. |
| libminifi/test/schema-tests/CMakeLists.txt | Switches JSON schema validator include to Get* module. |
| libminifi/CMakeLists.txt | Updates core library link targets to namespaced targets and removes direct includes. |
| extensions/windows-event-log/CMakeLists.txt | Updates pugixml target usage to namespaced target. |
| extensions/standard-processors/controllers/XMLRecordSetWriter.cpp | Fixes pugixml text setting to use c_str() for string values. |
| extensions/standard-processors/CMakeLists.txt | Updates range-v3/asio/pugixml/jsoncons integration to Get* and namespaced targets. |
| extensions/sftp/CMakeLists.txt | Switches to GetLibSSH2 and namespaced libssh2 target. |
| extensions/prometheus/CMakeLists.txt | Switches to GetPrometheus and links via unified target list variable. |
| extensions/opc/CMakeLists.txt | Removes mbedTLS path and switches to GetOpen62541. |
| extensions/mqtt/CMakeLists.txt | Switches to GetPahoMqttC and updates target name for linking. |
| extensions/libarchive/CMakeLists.txt | Switches LZMA acquisition to GetLibLZMA. |
| extensions/kafka/rdkafka_utils.h | Updates librdkafka header include path to include subdir form. |
| extensions/kafka/PublishKafka.h | Updates librdkafka header include path. |
| extensions/kafka/KafkaConnection.h | Updates librdkafka header include path. |
| extensions/kafka/ConsumeKafka.h | Updates librdkafka header include path. |
| extensions/kafka/CMakeLists.txt | Switches librdkafka acquisition to Getlibrdkafka and links via RdKafka::rdkafka++. |
| extensions/couchbase/CMakeLists.txt | Switches to GetCouchbase and uses unified couchbase target name. |
| extension-framework/CMakeLists.txt | Updates core dependency link list to namespaced targets. |
| encrypt-config/CMakeLists.txt | Switches argparse acquisition to GetArgparse and namespaced targets. |
| core-framework/common/CMakeLists.txt | Updates core-framework common deps to namespaced targets. |
| core-framework/CMakeLists.txt | Updates core-framework deps to namespaced targets. |
| controller/CMakeLists.txt | Switches argparse acquisition to GetArgparse and namespaced targets. |
| conanfile.py | Expands Conan requirements/options and exports Conan-based CMake source selectors. |
| CONAN.md | Updates Conan build documentation (bootstrap usage, export vs create, remote add). |
| CMakeLists.txt | Refactors ccache launcher configuration, adds CMP0091 default, and adopts Get* modules + namespaced targets. |
| cmake/PugiXml.cmake | Updates pugixml version/hash used by FetchContent build. |
| cmake/PahoMqttC.cmake | Simplifies patching and removes old alias target for paho-mqtt-c. |
| cmake/Open62541.cmake | Switches encryption to OpenSSL, updates patch path, and updates dependencies. |
| cmake/MiNiFiOptions.cmake | Adds many new thirdparty “source” multi-options and removes SYSTEM option. |
| cmake/MbedTLS.cmake | Removes bundled mbedTLS module. |
| cmake/LibLZMA.cmake | Makes LibLZMA alias creation idempotent and adjusts module path handling. |
| cmake/Jsoncons.cmake | Adds jsoncons::jsoncons alias if missing. |
| cmake/GetYamlCpp.cmake | Adds Conan-vs-build selector wrapper for yaml-cpp. |
| cmake/GetSpdlog.cmake | Makes spdlog alias creation idempotent in Conan mode. |
| cmake/GetRocksDB.cmake | Makes RocksDB alias creation idempotent in Conan mode. |
| cmake/GetRangeV3.cmake | Adds Conan-vs-build selector wrapper for range-v3. |
| cmake/GetPugiXml.cmake | Adds Conan-vs-build selector wrapper for pugixml. |
| cmake/GetPrometheus.cmake | Adds Conan-vs-build selector wrapper and consolidates targets list. |
| cmake/GetPahoMqttC.cmake | Adds Conan-vs-build selector wrapper for paho-mqtt-c. |
| cmake/GetOpen62541.cmake | Adds Conan-vs-build selector wrapper for open62541. |
| cmake/GetMiMalloc.cmake | Adds Conan-vs-build selector wrapper for mimalloc. |
| cmake/GetMagicEnum.cmake | Adds Conan-vs-build selector wrapper for magic_enum. |
| cmake/GetLibSSH2.cmake | Adds Conan-vs-build selector wrapper for libssh2. |
| cmake/GetLibSodium.cmake | Adds Conan-vs-build selector wrapper for libsodium. |
| cmake/Getlibrdkafka.cmake | Adds Conan-vs-build selector wrapper for librdkafka. |
| cmake/GetLibLZMA.cmake | Adds Conan-vs-build selector wrapper for liblzma. |
| cmake/GetJsonSchemaValidator.cmake | Adds Conan-vs-build selector wrapper for json-schema-validator. |
| cmake/GetJsoncons.cmake | Adds Conan-vs-build selector wrapper for jsoncons. |
| cmake/GetJemalloc.cmake | Adds Conan-vs-build selector wrapper for jemalloc. |
| cmake/GetGslLite.cmake | Adds Conan-vs-build selector wrapper for gsl-lite. |
| cmake/GetCouchbase.cmake | Adds Conan-vs-build selector wrapper for couchbase C++ client. |
| cmake/GetCatch2.cmake | Makes Catch2 alias creation idempotent in Conan mode. |
| cmake/GetBenchmark.cmake | Adds Conan-vs-build selector wrapper for benchmark. |
| cmake/GetAsio.cmake | Replaces mbedTLS export handling with Conan-vs-build selector for Asio. |
| cmake/GetArgparse.cmake | Replaces unrelated FetchContent content with argparse Conan-vs-build selector. |
| cmake/FetchLibSSH2.cmake | Adds Libssh2::libssh2 alias for consistent linking. |
| cmake/Fetchlibrdkafka.cmake | Updates librdkafka version/hash and adds header install shim + alias. |
| cmake/Crc32c.cmake | Makes crc32c alias creation idempotent. |
| cmake/Couchbase.cmake | Updates Asio handling and patch paths; adds canonical target alias. |
| cmake/Couchbase_1_0_2.cmake | Removes obsolete couchbase 1.0.2 build module. |
| cmake/CivetWeb.cmake | Makes civetweb alias creation idempotent. |
| cmake/BundledRocksDB.cmake | Disables RocksDB tools in bundled build. |
| cmake/BundledLibSodium.cmake | Renames imported libsodium target to namespaced libsodium::libsodium. |
| cmake/BundledJemalloc.cmake | Updates jemalloc source/version and renames imported target to jemalloc::jemalloc. |
| cmake/BundledAwsSdkCpp.cmake | Normalizes CMake arg quoting/formatting for AWS SDK build. |
| cmake/BuildTests.cmake | Switches yaml-cpp linkage in test targets to namespaced target. |
| cmake/Asio.cmake | Adds asio::asio alias target. |
| bootstrap/requirements.txt | Adds Conan and PyYAML to bootstrap Python dependencies. |
| bootstrap/package_manager.py | Simplifies venv activation call in Windows environment setup. |
| bootstrap/minifi_option.py | Adds USE_CONAN and custom malloc handling to bootstrap option flow. |
| bootstrap/cli.py | Adds Conan install/export flow to bootstrap build orchestration. |
| .github/workflows/ci.yml | Enables Conan in macOS/Windows CI, adds compatibility plugin, and uploads packages. |
| .github/conan/compatibility.py | Adds a Conan compatibility extension for broader package reuse across settings. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
fgerlits
left a comment
There was a problem hiding this comment.
This looks great, thank you!
…OS CI jobs - Add conan support to bootstrap script - Add conan support for the following libraries: argparse, jemalloc, libsodium, Libssh2, benchmark, gsl-lite, json-schema-validator, mimalloc, pugixml, yaml-cpp, prometheus, range-v3, liblzma, paho-mqtt, asio, couchbase, librdkafka, magic_enum, open62541 - Remove unused uvc library - Remove mbedtls dependency and use openssl instead in open62541 - Disable with_tools option for rocksdb - Add conan CI integration to Windows and MacOS jobs
a7c04e1 to
1c559b0
Compare
|
Added fix for Kafka extension conan build to be in line with the compile options when built from source: 65aa73d |
https://issues.apache.org/jira/browse/MINIFICPP-2859
Thank you for submitting a contribution to Apache NiFi - MiNiFi C++.
In order to streamline the review of the contribution we ask you to ensure the following steps have been taken:
For all changes:
Is there a JIRA ticket associated with this PR? Is it referenced in the commit message?
Does your PR title start with MINIFICPP-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
Has your PR been rebased against the latest commit within the target branch (typically main)?
Is your initial contribution a single, squashed commit?
For code changes:
For documentation related changes:
Note:
Please ensure that once the PR is submitted, you check GitHub Actions CI results for build issues and submit an update to your PR as soon as possible.