[Do Not Merge/ POC only] planner: support rewrite join to apply rule#67693
[Do Not Merge/ POC only] planner: support rewrite join to apply rule#67693Reminiscent wants to merge 1 commit into
Conversation
|
This cherry pick PR is for a release branch and has not yet been approved by triage owners. To merge this cherry pick:
DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
ti-chi-bot
Bot
added
release-note-none
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
ti-chi-bot
Bot
added
size/XXL
|
Hi @Reminiscent. Thanks for your PR. PRs from untrusted users cannot be marked as trusted with I understand the commands that are listed here. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
📝 WalkthroughWalkthroughThe PR adds support for the Changes
Estimated code review effort🎯 4 (Complex) | ⏱️ ~50 minutes Possibly related PRs
Suggested labels
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 3 | ❌ 2❌ Failed checks (1 warning, 1 inconclusive)
✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
/ok-to-test |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
tests/integrationtest/t/planner/core/casetest/hint/join_to_apply.test (1)
120-128: Add a no-hint control for the aggregation barrier case.This only snapshots the hinted plan, so it does not actually prove the new rule is a no-op here. Adding the same
EXPLAINwithoutJOIN_TO_APPLY()would make regressions unambiguous if this barrier shape ever starts rewriting indirectly.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/integrationtest/t/planner/core/casetest/hint/join_to_apply.test` around lines 120 - 128, Add a no-hint control by duplicating the EXPLAIN block without the JOIN_TO_APPLY() hint so the test asserts the hinted plan is unchanged versus the baseline; specifically, add an additional "explain format = 'brief'" + SELECT ... (the same query joining jt_outer and the aggregated jt_inner using max and group by i.k) but remove the /*+ JOIN_TO_APPLY() */ hint so the test compares the hinted snapshot against the no-hint baseline and will catch any indirect rewriting of the aggregation barrier.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@pkg/planner/core/rule_join_to_apply.go`:
- Around line 93-145: tryRewriteJoinToApply currently emits statement-level
warnings via join.SCtx().GetSessionVars().StmtCtx.SetHintWarning during per-join
checks (e.g., when no correlatable keys or no usable TiKV path), which causes
misleading warnings for mixed queries; stop emitting warnings inside
tryRewriteJoinToApply and instead return applicability info to the caller (e.g.,
change tryRewriteJoinToApply signature to return (base.LogicalPlan, bool,
string) or an enum/reason) so the traversal driver can aggregate results across
all joins and call StmtCtx.SetHintWarning only once if no join in the whole
statement is eligible; update call sites that invoke tryRewriteJoinToApply to
collect reasons/flags and emit the single statement-level warning after the full
rewrite pass.
---
Nitpick comments:
In `@tests/integrationtest/t/planner/core/casetest/hint/join_to_apply.test`:
- Around line 120-128: Add a no-hint control by duplicating the EXPLAIN block
without the JOIN_TO_APPLY() hint so the test asserts the hinted plan is
unchanged versus the baseline; specifically, add an additional "explain format =
'brief'" + SELECT ... (the same query joining jt_outer and the aggregated
jt_inner using max and group by i.k) but remove the /*+ JOIN_TO_APPLY() */ hint
so the test compares the hinted snapshot against the no-hint baseline and will
catch any indirect rewriting of the aggregation barrier.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
Run ID: 82fcb7d4-eb23-4a29-8bf7-d06da70ad147
📒 Files selected for processing (12)
pkg/executor/adapter.gopkg/parser/hintparser.gopkg/parser/hintparser.ypkg/parser/hintparser_test.gopkg/parser/misc.gopkg/planner/core/optimizer.gopkg/planner/core/planbuilder.gopkg/planner/core/rule/logical_rules.gopkg/planner/core/rule_join_to_apply.gopkg/util/hint/hint.gotests/integrationtest/r/planner/core/casetest/hint/join_to_apply.resulttests/integrationtest/t/planner/core/casetest/hint/join_to_apply.test
| func tryRewriteJoinToApply(join *logicalop.LogicalJoin) (base.LogicalPlan, bool) { | ||
| if join.JoinType != logicalop.LeftOuterJoin || !join.SCtx().GetSessionVars().StmtCtx.EnableJoinToApply { | ||
| return join, false | ||
| } | ||
|
|
||
| if containsLogicalCTE(join.Children()[1]) { | ||
| // Apply under CTE needs extra builder-time CTE bookkeeping that this | ||
| // rule intentionally does not reproduce in the first rollout. | ||
| join.SCtx().GetSessionVars().StmtCtx.SetHintWarning("JOIN_TO_APPLY() is inapplicable because the right subtree contains a CTE.") | ||
| return join, false | ||
| } | ||
|
|
||
| var ( | ||
| window *logicalop.LogicalWindow | ||
| dataSource *logicalop.DataSource | ||
| ) | ||
| correlatableKeys := make([]correlatableJoinKey, 0, len(join.EqualConditions)) | ||
| remainingKeys := make([]*expression.ScalarFunction, 0, len(join.EqualConditions)) | ||
| for _, eq := range join.EqualConditions { | ||
| outerCol, innerCol := extractJoinKeyCols(eq, join.Children()[0].Schema(), join.Children()[1].Schema()) | ||
| if outerCol == nil || innerCol == nil { | ||
| remainingKeys = append(remainingKeys, eq) | ||
| continue | ||
| } | ||
|
|
||
| trace, ok := traceWindowBypassPath(join.Children()[1], innerCol, false) | ||
| if !ok { | ||
| remainingKeys = append(remainingKeys, eq) | ||
| continue | ||
| } | ||
| if window == nil { | ||
| window = trace.window | ||
| dataSource = trace.dataSource | ||
| } else if window != trace.window || dataSource != trace.dataSource { | ||
| remainingKeys = append(remainingKeys, eq) | ||
| continue | ||
| } | ||
| correlatableKeys = append(correlatableKeys, correlatableJoinKey{ | ||
| eq: eq, | ||
| outerCol: outerCol, | ||
| innerCol: innerCol, | ||
| resolvedCol: trace.partitionCol, | ||
| accessCol: trace.accessCol, | ||
| }) | ||
| } | ||
|
|
||
| if len(correlatableKeys) == 0 { | ||
| join.SCtx().GetSessionVars().StmtCtx.SetHintWarning("JOIN_TO_APPLY() is inapplicable because no join key matches the window partition columns.") | ||
| return join, false | ||
| } | ||
| if !dataSourceHasUsableTiKVAccessPath(dataSource, correlatableKeys) { | ||
| join.SCtx().GetSessionVars().StmtCtx.SetHintWarning("JOIN_TO_APPLY() is inapplicable because the inner datasource has no usable TiKV access path on the correlated key.") | ||
| return join, false |
There was a problem hiding this comment.
Defer JOIN_TO_APPLY() warnings until the rewrite pass finishes.
JOIN_TO_APPLY() is a statement-level hint, but this method warns on each left outer join that it cannot rewrite. In a mixed query, one join can be rewritten successfully while another unrelated left join still leaves a misleading “inapplicable” warning behind. Please track applicability across the whole traversal and only warn when the statement ends up with no eligible rewrite target.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@pkg/planner/core/rule_join_to_apply.go` around lines 93 - 145,
tryRewriteJoinToApply currently emits statement-level warnings via
join.SCtx().GetSessionVars().StmtCtx.SetHintWarning during per-join checks
(e.g., when no correlatable keys or no usable TiKV path), which causes
misleading warnings for mixed queries; stop emitting warnings inside
tryRewriteJoinToApply and instead return applicability info to the caller (e.g.,
change tryRewriteJoinToApply signature to return (base.LogicalPlan, bool,
string) or an enum/reason) so the traversal driver can aggregate results across
all joins and call StmtCtx.SetHintWarning only once if no join in the whole
statement is eligible; update call sites that invoke tryRewriteJoinToApply to
collect reasons/flags and emit the single statement-level warning after the full
rewrite pass.
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## release-8.5 #67693 +/- ##
================================================
Coverage ? 42.3257%
================================================
Files ? 1534
Lines ? 427355
Branches ? 0
================================================
Hits ? 180881
Misses ? 230374
Partials ? 16100
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
|
@Reminiscent: The following tests failed, say
Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
|
/hold DO NOT unhold this without my approval. |
ti-chi-bot
Bot
added
the
do-not-merge/hold
Reminiscent
changed the title
2 participants
What problem does this PR solve?
Issue Number: close #51116
Problem Summary:
support rewrite join to apply rule
What changed and how does it work?
Check List
Tests
Side effects
Documentation
Release note
Please refer to Release Notes Language Style Guide to write a quality release note.
Summary by CodeRabbit
JOIN_TO_APPLY()optimizer hint enabling optimized execution plans for LEFT JOIN queries containing window functions, allowing the planner to convert eligible join patterns into Apply operations for improved performance.