Make the JSON ingestion spec extension fully opt-in#1249
Open
jwils wants to merge 2 commits into
Open
Conversation
jwils
requested review from
BrianSigafoos-SQ,
bsorbo,
jwondrusch and
myronmarston
as code owners
jwils
force-pushed
the
joshuaw/indexer-ingestion-adapters
branch
from
e8e1a67 to
1631628
Compare
jwils
force-pushed
the
joshuaw/lazy-json-ingestion-spec-default
branch
from
81cb325 to
5e11583
Compare
jwils
changed the title
jwils
force-pushed
the
joshuaw/lazy-json-ingestion-spec-default
branch
from
8dc35fb to
39af63e
Compare
jwils
force-pushed
the
joshuaw/indexer-ingestion-adapters
branch
2 times, most recently
from
4a79422 to
b1abdb0
Compare
jwils
force-pushed
the
joshuaw/lazy-json-ingestion-spec-default
branch
from
39af63e to
6a733f1
Compare
jwils
force-pushed
the
joshuaw/indexer-ingestion-adapters
branch
from
b1abdb0 to
908b4c9
Compare
jwils
requested review from
ellisandrews-toast,
marcdaniels-toast and
rossroberts-toast
as code owners
jwils
force-pushed
the
joshuaw/indexer-ingestion-adapters
branch
from
908b4c9 to
247ac28
Compare
jwils
force-pushed
the
joshuaw/lazy-json-ingestion-spec-default
branch
from
e37cb08 to
ec47261
Compare
jwils
force-pushed
the
joshuaw/lazy-json-ingestion-spec-default
branch
from
efab4d7 to
f136921
Compare
myronmarston
force-pushed
the
joshuaw/lazy-json-ingestion-spec-default
branch
from
01bcbd3 to
2a45ad6
Compare
Keep JSON ingestion schema support opt-in instead of baking it into the indexer helpers. Specs that actually ingest JSON data now use :ingests_json_data, which composes datastore, indexer, and :json_ingestion_schema_definition support. Schema-only specs use :json_ingestion_schema_definition directly. This keeps elasticgraph-indexer tests from assuming JSON as the only ingestion format and leaves room for future non-JSON ingestion paths. The json_ingestion gem applies the shared JSON schema-definition support by convention from its spec helper, so its specs no longer need a separate helper that duplicates APIExtension wiring. Two subtleties matter for keeping the full suite green: - `define_derived_metadata` does not reliably expand a derived tag declared on a `shared_context` into the groups produced when the context is combined with `include_examples` (RSpec raises inside `update_inherited_metadata`; see rspec/rspec#333). So the shared contexts that ingest JSON data list the component tags (`:json_ingestion_schema_definition`, `:uses_datastore`, `:builds_indexer`) literally rather than via `:ingests_json_data`, with a comment referencing the upstream issue so the workaround can be collapsed once it is fixed. Dropping the (redundant) `:capture_logs` tag alone does not avoid the crash, and reverting either shared context to `:ingests_json_data` reintroduces it in the full suite. - The elasticgraph-apollo and elasticgraph-health_check bundles do not include elasticgraph-json_ingestion, and their specs use prebuilt on-disk artifacts, so they stay on `:uses_datastore`/`:builds_indexer` rather than opting into the JSON schema-definition extension they cannot load. Generated with Claude Code
myronmarston
force-pushed
the
joshuaw/lazy-json-ingestion-spec-default
branch
4 times, most recently
from
ce6ba87 to
b214bfe
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
On #1205, @myronmarston suggested that the
DEFAULT_SCHEMA_DEFINITION_EXTENSION_MODULESavailability probe inspec_support/schema_definition_helpers.rb(abegin/rescue LoadErrorevaluated at file load) could go away in a follow-up once the extraction settled. The review here then asked the natural follow-on question: is there anything left that really needs the defaulting, or is this a good time to force tests to opt in?Answer: nothing needs the defaulting anymore. This PR removes it entirely.
What
spec_support/schema_definition_helpers.rbno longer applies any extension modules by default —define_schemain specs builds a plain core schema unless the caller passesextension_modules:.generate_schema_artifactsincludes the extension only when loading the repository's main test schema (config/schema.rb), which uses the JSON ingestion schema definition DSL. Schemas defined via a block get no extensions unless requested.build_indexerincludes the extension when it generates schema artifacts itself, since the indexer can only ingest JSON events today. The require is lazy so spec bundles withoutelasticgraph-json_ingestion(apollo, warehouse, health_check, query_interceptor) can still build an indexer from an externally builtdatastore_core:.schema_definition_extension_modules:/extension_modules:at their build call sites.t.json_schemacalls in unit specs that never index are removed, along with aJsonSafeLongplaceholder assertion inelasticgraph-graphqlthat duplicated coverage now living inelasticgraph-json_ingestion."JSONIngestionSchemaDefinitionHelpers"shared context (modeled onelasticgraph-warehouse'swarehouse_schema_support) now appliesAPIExtensionto every schema defined inelasticgraph-json_ingestion's own specs, so those call sites don't each passextension_modules:explicitly.Verification
script/run_gem_specsforelasticgraph-schema_definition,elasticgraph-json_ingestion,elasticgraph-indexer,elasticgraph-graphql,elasticgraph-datastore_core,elasticgraph-admin,elasticgraph-health_check, andelasticgraph-query_interceptorscript/type_checkscript/quick_buildStack
Current PR is marked with
->.