Skip to content

[DOC] warn against explain ALL in production - #15833

Open
nvliyuan wants to merge 1 commit into
NVIDIA:mainfrom
nvliyuan:docs-warn-explain-all-production
Open

[DOC] warn against explain ALL in production#15833
nvliyuan wants to merge 1 commit into
NVIDIA:mainfrom
nvliyuan:docs-warn-explain-all-production

Conversation

@nvliyuan

Copy link
Copy Markdown
Collaborator

No issue; documentation clarification requested after a production workload generated enough spark.rapids.sql.explain=ALL diagnostics to make the driver unresponsive.

Description

Clarifies that spark.rapids.sql.explain=ALL is intended only for debugging and must not be enabled for production workloads because it can generate substantial driver log output.

Changes include:

  • Update the canonical RapidsConf help text to warn about driver performance and responsiveness risks.
  • Recommend the default NOT_ON_GPU level, or NONE, for production workloads.
  • Apply the same warning to the supported-operations guidance, which previously recommended ALL without a production caveat.
  • Synchronize the generated docs/configs.md and docs/supported_ops.md output with their Scala sources.

Validation:

  • git diff --check
  • IDE lints on all four modified files
  • Verified modified Scala source lines remain within the project line-length limit
  • Verified both generated documentation pages contain the corresponding production warning
  • Local Maven validation was not available because this machine has no Java runtime; GitHub CI will run compilation, Scalastyle, and generated-doc checks

Checklists

Documentation

  • Updated for new or modified user-facing features or behaviors
  • No user-facing change

Testing

  • Added or modified tests to cover new code paths
  • Covered by existing tests
    (Please provide the names of the existing tests in the PR description.)
  • Not required

Performance

  • Tests ran and results are added in the PR description
  • Issue filed with a link in the PR description
  • Not required: this changes only static help text and generated documentation; configuration parsing and runtime behavior are unchanged

Document the driver logging risk and direct production workloads to the safer default diagnostics levels.

Signed-off-by: liyuan <yuali@nvidia.com>
@nvliyuan nvliyuan added the documentation Improvements or additions to documentation label Aug 31, 2026
@nvliyuan
nvliyuan requested review from sameerz and viadea August 31, 2026 06:18
@nvliyuan nvliyuan self-assigned this Aug 31, 2026
@greptile-apps

greptile-apps Bot commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

Updates the spark.rapids.sql.explain documentation to warn that ALL is debugging-only and can make production drivers unresponsive.

  • Recommends NOT_ON_GPU or NONE for production workloads.
  • Synchronizes the generated configuration and supported-operations pages with their Scala sources.

Confidence Score: 5/5

The PR appears safe to merge because it consistently updates static guidance without changing configuration defaults, parsing, or runtime behavior.

The Scala documentation sources and checked-in generated pages are synchronized, and no actionable failure remains.

Important Files Changed

Filename Overview
sql-plugin/src/main/scala/com/nvidia/spark/rapids/RapidsConf.scala Extends the canonical EXPLAIN configuration help text with a production warning while preserving its default and runtime behavior.
sql-plugin/src/main/scala/com/nvidia/spark/rapids/TypeChecks.scala Updates the supported-operations documentation generator with matching guidance for production workloads.
docs/configs.md Synchronizes the generated configuration entry exactly with the updated RapidsConf help text.
docs/supported_ops.md Synchronizes the generated supported-operations guidance exactly with its Scala generator.

Reviews (1): Last reviewed commit: "[DOC] warn against explain ALL in produc..." | Re-trigger Greptile

@nvliyuan
nvliyuan requested a review from nartal1 August 31, 2026 06:20
@sameerz

sameerz commented Sep 2, 2026

Copy link
Copy Markdown
Collaborator

build

2 similar comments
@nvliyuan

nvliyuan commented Sep 2, 2026

Copy link
Copy Markdown
Collaborator Author

build

@sameerz

sameerz commented Sep 2, 2026

Copy link
Copy Markdown
Collaborator

build

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants