Skip to content

[fix][broker] Propagate topic policy initialization failures - #26259

Open
Praveenkumar76 wants to merge 1 commit into
apache:masterfrom
cognitree:fix/26137-topic-policy-load-failure
Open

[fix][broker] Propagate topic policy initialization failures#26259
Praveenkumar76 wants to merge 1 commit into
apache:masterfrom
cognitree:fix/26137-topic-policy-load-failure

Conversation

@Praveenkumar76

@Praveenkumar76 Praveenkumar76 commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

Fixes #26137

Motivation

Topic policy loading is performed during topic initialization through AbstractTopic.initTopicPolicy().

Although BrokerService may read topic policies earlier in the topic-loading flow, a separate policy read is performed later by initTopicPolicy(). The earlier read can succeed while this later read fails because of a transient metadata-store error, a __change_events read failure, or a policy-cache initialization timeout.

Previously, PersistentTopic.initialize() and NonPersistentTopic.initialize() handled this later failure with a terminal exceptionally(...) stage that returned null. This converted the failed initialization future into a successful one, allowing the topic to finish loading without its topic-level policies applied.

The topic could then temporarily or indefinitely use namespace- or broker-level fallback values. This is particularly unsafe when a topic has longer retention or TTL settings than its namespace, because messages can become eligible for deletion or expiration earlier than configured.

This PR chooses fail-closed behavior for failures from initTopicPolicy(): the failure is logged and propagated instead of allowing the topic to finish initialization with unapplied topic policies.

Modifications

  • Replaced the failure-recovery handlers around initTopicPolicy() in PersistentTopic.initialize() and NonPersistentTopic.initialize() with whenComplete(...).
  • Kept policy-load failure logging while preserving the exceptional completion of the initialization future.
  • Scoped the failure logging specifically to initTopicPolicy() rather than unrelated initialization operations.
  • Preserved the existing defensive reset of isEncryptionRequired when persistent-topic policy initialization fails.
  • Added focused regression tests for persistent and non-persistent topics.

Verifying this change

This change added tests and can be verified as follows:

  • Added PersistentTopicTest.testInitializeFailsWhenTopicPolicyLoadingFails.
  • Added NonPersistentTopicTest.testInitializeFailsWhenTopicPolicyLoadingFails.
  • Both tests mock a successful policy-listener registration followed by a failed topic-policy fetch and verify that the topic initialization future completes exceptionally.
./gradlew :pulsar-broker:test \
  --tests "org.apache.pulsar.broker.service.PersistentTopicTest.testInitializeFailsWhenTopicPolicyLoadingFails" \
  --tests "org.apache.pulsar.broker.service.nonpersistent.NonPersistentTopicTest.testInitializeFailsWhenTopicPolicyLoadingFails" \
  -PskipJavaVersionCheck \
  -DtestRetryCount=0 \
  --no-daemon \
  --no-configuration-cache

Does this pull request potentially affect one of the following parts:

  • Dependencies (add or upgrade a dependency)
  • The public API
  • The schema
  • The default values of configurations
  • The threading model
  • The binary protocol
  • The REST endpoints
  • The admin CLI options
  • The metrics
  • Anything that affects deployment

This changes topic-loading behavior when the later initTopicPolicy() operation fails. Instead of loading the topic with namespace- or broker-level fallback policies, topic initialization now fails.

@Praveenkumar76
Praveenkumar76 force-pushed the fix/26137-topic-policy-load-failure branch from b5987ce to 84bb5f1 Compare August 1, 2026 06:41
@Praveenkumar76
Praveenkumar76 marked this pull request as ready for review August 2, 2026 07:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] Topic policy loading errors are ignored which can result in data loss

1 participant