fix: Allow tombstones without a key property - #502
Merged
Conversation
jsonbailey
force-pushed
the
jb/sdk-2941/tombstone-key-check
branch
2 times, most recently
from
August 19, 2026 17:27
030c519 to
a245d0e
Compare
FeatureFlag and Segment validated the required "key" property before the deleted-item early-out, so a persisted tombstone written with only a version could not be decoded. Other LaunchDarkly SDKs (Node, .NET, Java) write keyless tombstones to a persistent store, so a shared store broke Python's all-flags read. The key requirement now applies only to items that are not deleted. A tombstone still requires a version, and .key falls back to an empty string so callers never fail. The stored data still round-trips verbatim.
jsonbailey
force-pushed
the
jb/sdk-2941/tombstone-key-check
branch
from
August 19, 2026 17:36
a245d0e to
d99edf5
Compare
kinyoklion
approved these changes
Aug 19, 2026
keelerm84
approved these changes
Aug 21, 2026
The Consul and DynamoDB cores rebuilt the all-items map from the key
inside each item, and threw away the key the item was stored under. A
deleted item written by another SDK has no key of its own, so the read
failed with KeyError('key') and broke all_flag_state.
Both cores now use the key they already have: the Consul KV path and the
DynamoDB sort key. The Redis cores were already correct.
keelerm84
approved these changes
Aug 26, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Symptom
A customer running the Python SDK and the Node server SDK against one shared Redis store saw
all_flags_state()start returningFeatureFlagsState(valid=False)with zero flags — the first time any flag was deleted, and for every subsequent read. Individualvariation()calls kept working; only the all-flags read broke.Root cause
FeatureFlag.__init__(ldclient/impl/model/feature_flag.py:107) andSegment.__init__(ldclient/impl/model/segment.py:76) validated the requiredkeyproperty before the deleted-item early-out:A persisted tombstone with no inner
keytherefore raisedValueError: error in flag/segment data: required property "key" is missing.CachingStoreWrapper._cache_put_all(ldclient/feature_store_helpers.py:130-138) decodes every item in the set with no per-item error handling, so one such record fails the entire all-flags read, andall_flags_state()returns an invalid state with no flags (ldclient/client.py:629-635).Cross-SDK context
Keyless tombstones are the norm, not a corruption:
{"version":N,"deleted":true}with no key (js-core,packages/store/node-server-sdk-redis/src/RedisCore.ts:163-168). .NET and Java do the same.deletedbefore validating the key. .NET's guard is literallyif (key is null && !deleted).$deleted), which already decoded fine here.Python was the only SDK that required the inner key on a tombstone. The key is redundant in a persistent store anyway: the store already knows it, because it is the Redis hash field the item is stored under.
Fix
The
keyrequirement now applies only to items that are not deleted. A tombstone:version;key's type if one is present, so the Go/Relay shape keeps working;.keyto an empty string when no key is present, so callers reading.keynever hit anAttributeError(__slots__is declared on these classes).The
versioncheck moves above the deleted branch so it stays a single check. One knock-on: a live record missing bothkeyandversionnow reportsversionas the missing property instead ofkey. Both are still required, only the property named in the error changes, and no test asserted the old wording.No key is synthesized into the stored dict.
ModelEntity.to_json_dict()returnsself._dataand the FDv2Store.commit()path depends on that, so the original data still round-trips verbatim — a test asserts this.Tests
ldclient/testing/impl/test_model_decode.py— for bothFEATURESandSEGMENTS: a keyless tombstone decodes and round-trips unchanged; the Go/Relay$deletedshape decodes (regression guard); a tombstone still requiresversion; a live record with no key still raisesValueError, including the explicit"deleted": Falsecase.ldclient/testing/test_feature_store_helpers.pyandtest_async_feature_store_helpers.py— a wrapper whose core holds a valid flag plus a keyless tombstone returns the valid flag fromall()without raising, the tombstone is filtered out, andget()on it returnsNone.No existing test asserted the old strictness for tombstones, so nothing had to be updated.
make test(1434 passed, 276 skipped) andmake lintboth pass.Out of scope
Deliberately kept small and single-purpose. These are tracked separately and are not in this PR:
_cache_put_all, so one bad record cannot fail a whole set;json.loadsguards in the store cores;Related
Note
Overview
Fixes shared persistent-store setups (e.g. Python + Node on Redis) where keyless deletion tombstones (
{"version":N,"deleted":true}) causedall_flags_state()to return invalid empty state because decoding required an innerkey.Model:
FeatureFlagandSegmentnow validateversionfirst; deleted items no longer requirekey(optional placeholder like$deletedstill works)..keyis''when absent, andto_json_dict()still round-trips the original payload without synthesizing a key.Stores: Consul and DynamoDB
get_all_internalindex results by the database/storage key (hash field / KV path / sort key), notitem['key'], matching Redis and other SDKs.Tests: Decode/round-trip cases for FEATURES and SEGMENTS; caching wrapper tolerance; integration
write_raw_item+test_all_reads_tombstone_with_no_keyfor Consul, DynamoDB, and Redis.Reviewed by Cursor Bugbot for commit 37bddbb. Bugbot is set up for automated code reviews on this repo. Configure here.