MINIFICPP-2860 Add Conan support for the remaining thirdparty libraries#2214
MINIFICPP-2860 Add Conan support for the remaining thirdparty libraries#2214lordgamez wants to merge 3 commits into
Conversation
d72fa53 to
6f11394
Compare
| add_minifi_multi_option(MINIFI_KAFKA_SOURCE "Retrieves librdkafka from provided source" "BUILD;CONAN" "BUILD") | ||
| add_minifi_multi_option(MINIFI_MAGIC_ENUM_SOURCE "Retrieves magic_enum from provided source" "BUILD;CONAN" "BUILD") | ||
| add_minifi_multi_option(MINIFI_OPC_SOURCE "Retrieves open62541 from provided source" "BUILD;CONAN" "BUILD") | ||
| add_minifi_multi_option(MINIFI_OSSP_UUID_SOURCE "Retrieves ossp-uuid from provided source" "BUILD;CONAN" "BUILD") |
There was a problem hiding this comment.
i think we should introduce a deafult flag, so something like this
add_minifi_multi_option(MINIFI_DEFAULT_DEPENDENCY_SOURCE "Retrieves dependencies from provided source by default" "BUILD;CONAN" "BUILD")
add_minifi_multi_option(MINIFI_LZ4_SOURCE "Retrieves lz4 from provided source" "BUILD;CONAN" "${MINIFI_DEFAULT_DEPENDENCY_SOURCE}")
add_minifi_multi_option(MINIFI_LIBCURL_SOURCE "Retrieves LibCURL from provided source" "BUILD;CONAN" "${MINIFI_DEFAULT_DEPENDENCY_SOURCE}")
...
There was a problem hiding this comment.
Good idea, added in 538e062 with an additional change in the main conanfile.py, this makes it unable to miss adding the MINIFI_* value change to CONAN when adding a new library.
There was a problem hiding this comment.
Pull request overview
Adds Conan packaging support for a broad set of remaining third-party dependencies and wires the build system to select between bundled (FetchContent) and Conan-provided sources via new Get* CMake modules. This continues the Conan integration work referenced by MINIFICPP-2860 (and depends on #2212).
Changes:
- Introduces Conan recipes + metadata for multiple third-party libraries (e.g., grpc, google-cloud-cpp, aws-sdk-cpp, azure-sdk-cpp, rpmalloc, ossp-uuid, llama.cpp, kubernetes client, bustache, jolt test resources).
- Updates core/extension CMake to use new
Get*modules and modernizes some dependency/link target usage. - Extends the top-level Conan build (
conanfile.py, bootstrap CLI) with new feature flags/options and required dependencies.
Reviewed changes
Copilot reviewed 78 out of 97 changed files in this pull request and generated 10 comments.
Show a summary per file
| File | Description |
|---|---|
| thirdparty/rpmalloc/config.yml | Adds rpmalloc Conan recipe version mapping |
| thirdparty/rpmalloc/all/conanfile.py | Adds rpmalloc Conan recipe |
| thirdparty/rpmalloc/all/conandata.yml | Adds rpmalloc source URL/hash |
| thirdparty/rpmalloc/all/CMakeLists.txt | Adds CMake build shim for rpmalloc |
| thirdparty/ossp-uuid/config.yml | Adds ossp-uuid Conan recipe version mapping |
| thirdparty/ossp-uuid/all/patches/ossp-uuid-update-config-guess.patch | Normalizes patch header formatting |
| thirdparty/ossp-uuid/all/patches/ossp-uuid-no-prog.patch | Adds patch to avoid building/installing CLI program |
| thirdparty/ossp-uuid/all/patches/ossp-uuid-mac-fix.patch | Adds patch to improve MAC handling/portability |
| thirdparty/ossp-uuid/all/conanfile.py | Adds ossp-uuid Conan recipe |
| thirdparty/ossp-uuid/all/conandata.yml | Adds ossp-uuid sources + patch list |
| thirdparty/llamacpp/config.yml | Adds llama.cpp Conan recipe version mapping |
| thirdparty/llamacpp/all/patches/mtmd-fix.patch | Adds llama.cpp patch for mtmd + include fix |
| thirdparty/llamacpp/all/conanfile.py | Adds llama.cpp Conan recipe |
| thirdparty/llamacpp/all/conandata.yml | Adds llama.cpp sources + patches |
| thirdparty/kubernetes-client-c/config.yml | Adds kubernetes-client-c Conan recipe version mapping |
| thirdparty/kubernetes-client-c/all/patches/remove-findpackage.patch | Adds patch to remove upstream find_package usage |
| thirdparty/kubernetes-client-c/all/conanfile.py | Adds kubernetes C client Conan recipe |
| thirdparty/kubernetes-client-c/all/conandata.yml | Adds kubernetes-client-c sources |
| thirdparty/jolt-tests/config.yml | Adds jolt test resources recipe version mapping |
| thirdparty/jolt-tests/all/conanfile.py | Adds Conan package for jolt test resources |
| thirdparty/jolt-tests/all/conandata.yml | Adds jolt sources URL/hash |
| thirdparty/grpc/fix-memory-request-missing-string-include.patch | Removes obsolete patch file |
| thirdparty/grpc/fix-glob-missing-algorithm-include.patch | Removes obsolete patch file |
| thirdparty/grpc/config.yml | Adds grpc Conan recipe version mapping |
| thirdparty/grpc/all/target_info/grpc_1.82.0.yml | Adds grpc target metadata used for packaging |
| thirdparty/grpc/all/patches/fix-protobuf-find-package.patch | Adds grpc patch for protobuf discovery |
| thirdparty/grpc/all/patches/fix-msvc-auto-return-type-template-arg.patch | Adds grpc patch for MSVC compilation issue |
| thirdparty/grpc/all/conanfile.py | Adds grpc Conan recipe |
| thirdparty/grpc/all/conandata.yml | Adds grpc sources + patches |
| thirdparty/grpc/all/conan_cmake_project_include.cmake | Adds grpc CMake project include workaround |
| thirdparty/grpc/all/cmake/grpc_plugin_template.cmake.in | Adds template for grpc plugin imported targets |
| thirdparty/google-cloud-cpp/remove_installation.patch | Removes obsolete patch file |
| thirdparty/google-cloud-cpp/config.yml | Adds google-cloud-cpp Conan recipe version mapping |
| thirdparty/google-cloud-cpp/all/patches/remove-find_package.patch | Adds patch adjusting upstream dependency discovery |
| thirdparty/google-cloud-cpp/all/patches/nlohmann_lib_as_interface.patch | Adds patch to enforce nlohmann_json usage |
| thirdparty/google-cloud-cpp/all/patches/c++23_fixes.patch | Adds patch for C++23 compatibility |
| thirdparty/google-cloud-cpp/all/conanfile.py | Adds google-cloud-cpp Conan recipe (storage-focused) |
| thirdparty/google-cloud-cpp/all/conandata.yml | Adds google-cloud-cpp sources + patches |
| thirdparty/bustache/config.yml | Adds bustache Conan recipe version mapping |
| thirdparty/bustache/all/patches/remove_installs.patch | Adds patch to remove upstream install rules |
| thirdparty/bustache/all/patches/fix-deprecated-literal-operator.patch | Adds patch for newer compiler literal operator |
| thirdparty/bustache/all/patches/add-append.patch | Adds patch for append helper needed by MiNiFi |
| thirdparty/bustache/all/conanfile.py | Adds bustache Conan recipe |
| thirdparty/bustache/all/conandata.yml | Adds bustache sources + patches |
| thirdparty/azure-sdk-cpp/fix-managed-identity.patch | Removes obsolete patch file |
| thirdparty/azure-sdk-cpp/config.yml | Adds azure-sdk-cpp Conan recipe version mapping |
| thirdparty/azure-sdk-cpp/all/patches/wil.patch | Adds patch to avoid upstream wil find_package |
| thirdparty/azure-sdk-cpp/all/patches/include-cinttypes-for-uint8_t-gcc15-fix.patch | Adds patch for missing includes |
| thirdparty/azure-sdk-cpp/all/patches/fix-openssl-helper.patch | Adds patch for openssl helper implementation |
| thirdparty/azure-sdk-cpp/all/conanfile.py | Adds azure-sdk-cpp Conan recipe |
| thirdparty/azure-sdk-cpp/all/conandata.yml | Adds azure-sdk-cpp sources + patches |
| thirdparty/aws-sdk-cpp/config.yml | Adds aws-sdk-cpp Conan recipe version mapping |
| thirdparty/aws-sdk-cpp/all/patches/s2n.patch | Adds patch to use OpenSSL targets for s2n |
| thirdparty/aws-sdk-cpp/all/patches/fix-finding-s2n.patch | Adds patch to force aws_use_package(s2n) |
| thirdparty/aws-sdk-cpp/all/patches/dll-export-injection.patch | Adds patch for Windows DLL export behavior |
| thirdparty/aws-sdk-cpp/all/patches/aws-c-cal.patch | Adds patch to adjust OpenSSL discovery/install bits |
| thirdparty/aws-sdk-cpp/all/conanfile.py | Adds aws-sdk-cpp Conan recipe |
| thirdparty/aws-sdk-cpp/all/conandata.yml | Adds aws-sdk-cpp source + patches list |
| extensions/standard-processors/tests/CMakeLists.txt | Switches to GetJoltTests include |
| extensions/llamacpp/CMakeLists.txt | Switches to GetLlamaCpp + updates link targets |
| extensions/kubernetes/CMakeLists.txt | Switches to GetKubernetesClientC include |
| extensions/grafana-loki/CMakeLists.txt | Switches to GetGrpc + shared proto generation module |
| extensions/gcp/tests/CMakeLists.txt | Adjusts test include/link logic to support Conan GTest |
| extensions/gcp/CMakeLists.txt | Switches to GetGoogleCloudCpp include |
| extensions/bustache/CMakeLists.txt | Switches to GetBustache include |
| extensions/azure/CMakeLists.txt | Switches to GetAzureSdkCpp include |
| extensions/aws/CMakeLists.txt | Switches to GetAwsSdkCpp include |
| conanfile.py | Adds new Conan options and requirements for the new libs |
| CMakeLists.txt | Switches some deps to Get* modules (rpmalloc, ossp-uuid, date) |
| cmake/WholeArchive.cmake | Enhances whole-archive handling for Conan interface targets |
| cmake/Protobuf.cmake | Updates bundled protobuf version |
| cmake/MiNiFiOptions.cmake | Adds new multi-options for selecting BUILD vs CONAN sources |
| cmake/LlamaCpp.cmake | Fixes patch path and adds alias targets |
| cmake/KubernetesClientC.cmake | Fixes patch path for Kubernetes client C |
| cmake/Grpc.cmake | Updates grpc tag and patch paths; adds gRPC::grpc++ alias |
| cmake/GoogleCloudCpp.cmake | Fixes patch paths; exposes test gtest link libs |
| cmake/GetWinFlexBison.cmake | Adds BUILD/CONAN switch for winflexbison |
| cmake/GetRpMalloc.cmake | Adds BUILD/CONAN switch for rpmalloc |
| cmake/GetOSSPUUID.cmake | Adds BUILD/CONAN switch for ossp-uuid |
| cmake/GetLlamaCpp.cmake | Adds BUILD/CONAN switch for llama.cpp |
| cmake/GetKubernetesClientC.cmake | Adds BUILD/CONAN switch for kubernetes client C |
| cmake/GetJoltTests.cmake | Adds BUILD/CONAN switch for jolt test resources |
| cmake/GetGrpc.cmake | Adds BUILD/CONAN switch for grpc |
| cmake/GetGoogleCloudCpp.cmake | Adds BUILD/CONAN switch for google-cloud-cpp (+ GTest handling) |
| cmake/GetDate.cmake | Adds BUILD/CONAN switch for date (+ tzdata staging on Windows) |
| cmake/GetBustache.cmake | Adds BUILD/CONAN switch for bustache |
| cmake/GetAzureSdkCpp.cmake | Adds BUILD/CONAN switch for azure SDK |
| cmake/GetAwsSdkCpp.cmake | Adds BUILD/CONAN switch for aws SDK |
| cmake/GenerateGrafanaLokiProto.cmake | Centralizes Loki proto generation |
| cmake/ExpressionLanguage.cmake | Switches Windows winflexbison setup to GetWinFlexBison |
| cmake/Date.cmake | Removes Windows tzdata staging logic (moved to GetDate) |
| bootstrap/cli.py | Extends Conan option mapping + updates conan settings flags |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Add Conan support for abseil, protobuf, grpc, gcp, ossp-uuid, bustache, jolt, rpmalloc, kubernetes, azure-sdk, llamacpp, date, winflexbison, and aws-sdk-cpp libraries
6f11394 to
7d03200
Compare
|
Added fix for kubernetes library built with conan to use static library instead of shared library: 7d03200 |
https://issues.apache.org/jira/browse/MINIFICPP-2860
Depends on #2212
Thank you for submitting a contribution to Apache NiFi - MiNiFi C++.
In order to streamline the review of the contribution we ask you to ensure the following steps have been taken:
For all changes:
Is there a JIRA ticket associated with this PR? Is it referenced in the commit message?
Does your PR title start with MINIFICPP-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
Has your PR been rebased against the latest commit within the target branch (typically main)?
Is your initial contribution a single, squashed commit?
For code changes:
For documentation related changes:
Note:
Please ensure that once the PR is submitted, you check GitHub Actions CI results for build issues and submit an update to your PR as soon as possible.