[samson30]

Relates to debezium/dbz#1809.

Description

Moves kstreams/poc-ddd-aggregates onto the shared parent POM, as agreed in debezium/dbz#1809, using the same repository-root Docker build as #503.

• pom.xml now uses debezium-examples-parent: Java 21 (was 11) and the compiler version come from the parent, and Kafka moves from 1.0.0 to the parent's 4.3.1 (kafka-streams, kafka-clients).
• Removes the unused kafka_2.11 broker dependency and the Confluent repository. jackson-databind is now declared directly (BOM managed, 2.21.2), since the serdes use it and it previously only came in through kafka_2.11.
• StreamingAggregatesDDD is updated for the Kafka Streams 4.x API: Consumed moved to the kstream package, Serialized (removed in Kafka 3.0) is replaced by Grouped, and CACHE_MAX_BYTES_BUFFERING_CONFIG (deprecated since 3.4) is replaced by STATESTORE_CACHE_MAX_BYTES_CONFIG.
• Dockerfile: the build context is now the repository root (docker-compose.yaml), with a Dockerfile.dockerignore limiting it to the root pom.xml and this module's files. The JDK stage uses eclipse-temurin:21-jdk instead of downloading OpenJDK 11, the build stage uses maven:3.9-eclipse-temurin-21, and the jlink module list was regenerated with jdeps for the new dependencies. Dependencies are now copied with -DincludeScope=runtime, because Kafka 4.x declares slf4j-api, zstd-jni, lz4-java and snappy-java as runtime dependencies and they would otherwise be missing from the image.
• CI workflow: adds actions/setup-java with Temurin 21 and includes the root pom.xml in the Maven cache key.

Verification (local, JDK 21)

• mvn clean install succeeds; dependency:tree shows kafka-streams and kafka-clients 4.3.1 and jackson-databind 2.21.2.
• docker compose build aggregator succeeds, with a 23 kB build context.
• Ran kafka, mysql, connect_source and aggregator and registered mysql-source.json: the aggregator prints the customer and address aggregates for customers 1001 to 1004, updating customer 1001 in MySQL produces an updated aggregate, and final_ddd_aggregates contains the JSON aggregates.

Notes

• The aggregator prints SLF4J(W): No SLF4J providers were found. Before this change the only binding came in through kafka_2.11 (slf4j-log4j12) without a log4j configuration, so the example had no log output then either; its results are printed with Printed.toSysOut(). I can add a binding such as slf4j-simple if you would like Kafka Streams logs.
• The MongoDB sink part of the example (connect_sink and debezium-mongodb/Dockerfile) is unchanged; this PR only covers the Maven module.

[Naros]

Hi @samson30 thanks for the PR. I've left a few points below. Could you also add a test.yaml file that is used by the end-to-end testing. See the other examples on how the test.yaml is constructed based on README steps.

[samson30]

Hi @samson30 thanks for the PR. I've left a few points below. Could you also add a test.yaml file that is used by the end-to-end testing. See the other examples on how the test.yaml is constructed based on README steps.

Thanks @Naros! I have added kstreams/test.yaml and wired it into the kstreams workflow. It registers the MySQL source connector, starts the aggregator once the source topics exist, checks the customer 1001 aggregate on final_ddd_aggregates, then updates the customer in MySQL and checks that the updated aggregate is published. It passes locally.

While testing, I also removed the build: section from connect_source in the compose file. That service only runs the MySQL source connector, but because the aggregator depends on it, docker compose build aggregator also built the MongoDB Dockerfile (based on Connect 2.0) and tagged the result as quay.io/debezium/connect:3.7.

The test doesn't cover the MongoDB sink part of the README, since that builds a Connect 2.0 image and compiles the hpgrahsl connector from source. If you'd like, I can look at moving it to Debezium's own MongoDB sink connector in a follow-up PR.

[Naros]

Thanks for all the work @samson30, merged & applied! 🎉