Skip to content
Open
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ concurrency:
env:
DOCKER_CMAKE_FLAGS: -DDOCKER_VERIFY_THREAD=3 -DUSE_SHARED_LIBS= -DSTRICT_GSL_CHECKS=AUDIT -DCI_BUILD=ON -DENABLE_AWS=ON -DENABLE_KAFKA=ON -DENABLE_MQTT=ON -DENABLE_AZURE=ON -DENABLE_SQL=ON \
-DENABLE_SPLUNK=ON -DENABLE_GCP=ON -DENABLE_OPC=ON -DENABLE_PYTHON_SCRIPTING=ON -DENABLE_LUA_SCRIPTING=ON -DENABLE_KUBERNETES=ON -DENABLE_TEST_PROCESSORS=ON -DENABLE_PROMETHEUS=ON \
-DENABLE_ELASTICSEARCH=ON -DENABLE_GRAFANA_LOKI=ON -DENABLE_COUCHBASE=ON -DENABLE_LLAMACPP=ON -DDOCKER_BUILD_ONLY=ON -DMINIFI_PERFORMANCE_TESTS=ON
-DENABLE_ELASTICSEARCH=ON -DENABLE_GRAFANA_LOKI=ON -DENABLE_COUCHBASE=ON -DENABLE_LLAMACPP=ON -DENABLE_LMDB=ON -DDOCKER_BUILD_ONLY=ON -DMINIFI_PERFORMANCE_TESTS=ON
CCACHE_DIR: ${{ GITHUB.WORKSPACE }}/.ccache
jobs:
macos_xcode:
Expand Down Expand Up @@ -41,6 +41,7 @@ jobs:
-DENABLE_LIBARCHIVE=ON
-DENABLE_LLAMACPP=ON
-DENABLE_KAFKA=ON
-DENABLE_LMDB=ON
-DENABLE_LUA_SCRIPTING=ON
-DENABLE_LZMA=ON
-DENABLE_MQTT=ON
Expand Down Expand Up @@ -146,6 +147,7 @@ jobs:
-DENABLE_LIBARCHIVE=ON
-DENABLE_KAFKA=ON
-DENABLE_LLAMACPP=ON
-DENABLE_LMDB=ON
-DENABLE_LUA_SCRIPTING=ON
-DENABLE_LZMA=ON
-DENABLE_MQTT=ON
Expand Down Expand Up @@ -244,10 +246,11 @@ jobs:
-DENABLE_EXECUTE_PROCESS=ON
-DENABLE_GCP=ON
-DENABLE_GRAFANA_LOKI=ON
-DENABLE_KAFKA=ON
-DENABLE_KUBERNETES=ON
-DENABLE_LIBARCHIVE=ON
-DENABLE_LLAMACPP=ON
-DENABLE_KAFKA=ON
-DENABLE_LMDB=ON
-DENABLE_LUA_SCRIPTING=ON
-DENABLE_LZMA=ON
-DENABLE_MQTT=ON
Expand Down
5 changes: 3 additions & 2 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -742,15 +742,16 @@ endif()

cpack_add_component_group(extensions DISPLAY_NAME "Extensions" EXPANDED)
set(EXTENSIONS_ENABLED_BY_DEFAULT (
minifi-archive-extensions
minifi-aws
minifi-azure
minifi-civet-extensions
minifi-elasticsearch
minifi-gcp
minifi-grafana-loki
minifi-archive-extensions
minifi-mqtt-extensions
minifi-kafka
minifi-lmdb
minifi-mqtt-extensions
minifi-pdh
minifi-prometheus
minifi-rocksdb-repos
Expand Down
14 changes: 13 additions & 1 deletion CONFIGURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -625,7 +625,7 @@ The Flow File Repository can be configured with the `nifi.flowfile.repository.cl
# in minifi.properties
nifi.flowfile.repository.class.name=NoOpRepository # VolatileFlowFileRepository can also be used which is an alias for NoOpRepository

The Content Repository can be configured with the `nifi.content.repository.class.name` property. If not specified, it uses the `DatabaseContentRepository` class by default, which persists the content in a RocksDB database. `DatabaseContentRepository` is also the default value specified in the minifi.properties file. Alternatively it can be configured to use a `VolatileContentRepository` that keeps the state in memory (so the state gets lost upon restart), or the `FileSystemRepository` to keep the state in regular files.
The Content Repository can be configured with the `nifi.content.repository.class.name` property. If not specified, it uses the `DatabaseContentRepository` class by default, which persists the content in a RocksDB database. `DatabaseContentRepository` is also the default value specified in the minifi.properties file. Alternatively it can be configured to use a `VolatileContentRepository` that keeps the state in memory (so the state gets lost upon restart), the `FileSystemRepository` to keep the state in regular files, or `LmdbContentRepository` that uses LMDB database as an alternative to RocksDB.

**NOTE:** RocksDB database has a limit of 4GB for the size of a database object. Due to this if you expect to process larger flow files than 4GB you should use the `FileSystemRepository`. The downside of using `FileSystemRepository` is that it does not have the transactional guarantees of the RocksDB repository implementation.

Expand Down Expand Up @@ -745,6 +745,18 @@ RocksDB options can also be overridden for a specific repository using the `nifi
nifi.provenance.repository.rocksdb.options.use_direct_reads=false
nifi.state.storage.rocksdb.options.use_direct_io_for_flush_and_compaction=false

### Configuring LMDB content repository

There is an alternative content repository that can be used: the LMDB database. When `LmdbContentRepository` is set, LMDB is used for storing content, which is a memory-mapped, fast (especially for reads), low-footprint, key-value database. It can be a good alternative on memory-limited edge devices, but each use case should be evaluated separately. The caveats of using this database are the following:

- Single writer only — concurrent writes are serialized, which can be a bottleneck for write-heavy workloads
- Database file never shrinks automatically — occasional large flow files could keep the database size permanently high on disk and in the reserved virtual address space
- No compression - stores everything uncompressed on disk
- The maximum database size must be set upfront, which is done in the minifi.properties file with the following property, with the current default value set to 10 GB:

# in minifi.properties
nifi.content.repository.lmdb.max.db.size=10 GB

#### Shared database

It is also possible to use a single database to store multiple repositories with the `minifidb://` scheme.
Expand Down
44 changes: 44 additions & 0 deletions cmake/LMDB.cmake
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.

include(FetchContent)

set(PATCH_FILE "${CMAKE_SOURCE_DIR}/thirdparty/lmdb/add-cmake-file.patch")
set(PC ${Bash_EXECUTABLE} -c "set -x &&\
(\\\"${Patch_EXECUTABLE}\\\" -p1 -R -s -f --dry-run -i \\\"${PATCH_FILE}\\\" || \\\"${Patch_EXECUTABLE}\\\" -p1 -N -i \\\"${PATCH_FILE}\\\")")

FetchContent_Declare(
lmdb
URL https://github.com/LMDB/lmdb/archive/refs/tags/LMDB_1.0.0-branch.tar.gz
URL_HASH SHA256=8d3e790194e43a72f172f34c442ea4737b2d1433fc0983f2ef70bae999bc2d28
PATCH_COMMAND "${PC}"
SOURCE_SUBDIR "libraries/liblmdb"
SYSTEM
)

if (WIN32)
get_directory_property(MINIFI_SAVED_COMPILE_DEFS COMPILE_DEFINITIONS)
remove_definitions(-DWIN32_LEAN_AND_MEAN)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you explain why you did this?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LMDB fails to compile on Windows when WIN32_LEAN_AND_MEAN is defined, which is automatically added to the compile definitions in CMakeLists.txt so it is included in all thirdparties used with FetchContent, so it needs to be removed separately for LMDB on Windows.

@szaszm szaszm Jun 22, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like to make absolutely sure that we only undefine WIN32_LEAN_AND_MEAN while compiling LMDB, or find another workaround. If you look up what it does, it prevents windows.h from including a bunch of additional headers that usually end up unused, and may end up conflicting with other headers, while also increasing compile times. Alternatively, we could look up the transitively included headers that LMDB relies on, and include them explicitly with target_compile_options(target PRIVATE -include foo.h)

Ideally Microsoft would've made the lightweight header the default, with the option to opt in to additional features, but for historical reasons, the heavyweight header is the default, and you can opt out of the extra features. (bloat)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When I checked this according to my understanding this change should only impact the lmdb subdirectory, so it should not impact any other compilation. Anyway I can still check if there is another workaround.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There were 2 issues:

  • Defining WIN32_LEAN_AND_MEAN removed winternl.h needed for NTSTATUS usage, that needed to be added explicitly
  • After including winternl.h the function pointer names in mdb.c clashed with the Windows symbols, those needed to be renamed

Added patch in: 52d93b2

endif()

FetchContent_MakeAvailable(lmdb)

if (WIN32)
set_directory_properties(PROPERTIES COMPILE_DEFINITIONS "${MINIFI_SAVED_COMPILE_DEFS}")
endif()

set(LMDB_INCLUDE_DIR "${lmdb_SOURCE_DIR}/libraries/liblmdb")
1 change: 1 addition & 0 deletions cmake/MiNiFiOptions.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,7 @@ endif()
add_minifi_option(ENABLE_ALL "Enables all extensions" OFF)
add_minifi_option(ENABLE_CIVET "Enables CivetWeb components." ON)
add_minifi_option(ENABLE_ROCKSDB "Enables the RocksDB extension." ON)
add_minifi_option(ENABLE_LMDB "Enables the LMDB extension." ON)
add_minifi_option(ENABLE_LIBARCHIVE "Enables the lib archive extensions." ON)
add_minifi_option(ENABLE_LZMA "Enables the liblzma build" ON)
add_minifi_option(ENABLE_BZIP2 "Enables the bzip2 build" ON)
Expand Down
3 changes: 3 additions & 0 deletions conf/minifi.properties.in
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,9 @@ nifi.content.repository.class.name=DatabaseContentRepository
# nifi.database.content.repository.rocksdb.compaction.period=2 min
# nifi.database.content.repository.optimize.for.small.db.cache.size=8 MB

## Relates to the internal workings of the LMDB backend
# nifi.content.repository.lmdb.max.db.size=10 GB

# setting this value to "0" enables synchronous deletion
# nifi.database.content.repository.purge.period = 1 sec

Expand Down
37 changes: 37 additions & 0 deletions extensions/lmdb/CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
#
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
#

if (NOT (ENABLE_ALL OR ENABLE_LMDB))
return()
endif()

include(LMDB)

include(${CMAKE_SOURCE_DIR}/extensions/ExtensionHeader.txt)

file(GLOB SOURCES "*.cpp")

add_minifi_library(minifi-lmdb SHARED ${SOURCES})

target_include_directories(minifi-lmdb PUBLIC ${LMDB_INCLUDE_DIR})
target_link_libraries(minifi-lmdb PUBLIC lmdb)
target_link_libraries(minifi-lmdb PUBLIC minifi-api minifi-extension-framework)
target_link_libraries(minifi-lmdb PRIVATE $<LINK_ONLY:core-minifi>)

register_extension(minifi-lmdb "LMDB" LMDB "This Enables persistent provenance, flowfile, and content repositories using LMDB" "extensions/lmdb/tests")
Comment thread
lordgamez marked this conversation as resolved.
Outdated
Loading
Loading