-
Notifications
You must be signed in to change notification settings - Fork 6.2k
pkg/planner: add order-aware logical join reorder rule #67305
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 6 commits
Commits
Show all changes
21 commits
Select commit
Hold shift + click to select a range
9157837
pkg/planner: add order-aware logical join reorder rule
AilinKid d628f8f
pkg/planner: refine order-aware join reorder propagation
AilinKid 44b80a4
pkg/planner: add barrier cases for order-aware join reorder
AilinKid 25ed694
pkg/planner: address order-aware review feedback
AilinKid bbeabe9
pkg/planner: address order-aware CodeRabbit feedback
AilinKid bdcdafe
pkg/planner: refine order-aware leading recursion
AilinKid 0f1e02a
Merge remote-tracking branch 'origin/master' into order-aware-reorder
AilinKid 14e9080
pkg/planner: add order-aware alternative logical plan round
AilinKid ad50288
pkg/planner: add order-aware alternative round casetest
AilinKid 19b7800
.
AilinKid 8f84191
pkg/server: drop unused optimizor testdata runfiles
AilinKid f4d7070
pkg/planner: stop masking straight-join reorder flags
AilinKid 68a1ff9
.
AilinKid 2f9da9c
Apply suggestion from @AilinKid
AilinKid ff12f12
.
AilinKid 8f15e0f
.
AilinKid ed4ba5f
.
AilinKid 598ca8b
.
AilinKid 2bdd26e
.
AilinKid 3d34d46
planner: rename PlanSatisfiesOrdering to DsSatisfiesOrdering
AilinKid 7cea238
planner: address order-aware join reorder review comments
AilinKid File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
20 changes: 20 additions & 0 deletions
20
pkg/planner/core/casetest/rule/testdata/order_aware_join_reorder_suite_in.json
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,20 @@ | ||
| [ | ||
| { | ||
| "name": "TestOrderAwareCDCJoinReorder", | ||
| "cases": [ | ||
| "SELECT t6.id, t7.payload, t8.payload FROM t7 JOIN t8 ON t7.id = t8.id JOIN t6 ON t6.id = t7.id WHERE t6.category = 'hot' ORDER BY t6.id LIMIT 2", | ||
| "SELECT /*+ LEADING(t7, t8, t6) */ t6.id, t7.payload, t8.payload FROM t7 JOIN t8 ON t7.id = t8.id JOIN t6 ON t6.id = t7.id WHERE t6.category = 'hot' ORDER BY t6.id LIMIT 2", | ||
| "SELECT /*+ TIDB_INLJ(t7, t8, t9) */ t6.id, t7.payload, t8.payload, t9.payload FROM t7 JOIN t8 ON t7.id = t8.id JOIN t9 ON t8.id = t9.id JOIN t6 ON t6.id = t7.id WHERE t6.category = 'hot' ORDER BY t6.id LIMIT 2", | ||
| "SELECT /*+ TIDB_INLJ(t7, t8, t9) LEADING(t7, t8, t9, t6) */ t6.id, t7.payload, t8.payload, t9.payload FROM t7 JOIN t8 ON t7.id = t8.id JOIN t9 ON t8.id = t9.id JOIN t6 ON t6.id = t7.id WHERE t6.category = 'hot' ORDER BY t6.id LIMIT 2" | ||
| ] | ||
| }, | ||
| { | ||
| "name": "TestOrderAwareJoinReorderPushSelection", | ||
| "cases": [ | ||
| "explain format = 'plan_tree' select t6.id, t7.payload, t8.payload from t7 join t8 on t7.id = t8.id join t6 on t6.id = t7.id where t6.category = 'hot' order by t6.id limit 2", | ||
| "explain format = 'plan_tree' select /*+ LEADING(t7, t8, t6) */ t6.id, t7.payload, t8.payload from t7 join t8 on t7.id = t8.id join t6 on t6.id = t7.id where t6.category = 'hot' order by t6.id limit 2", | ||
| "explain format = 'plan_tree' select /*+ TIDB_INLJ(t7, t8, t9) */ t6.id, t7.payload, t8.payload, t9.payload from t7 join t8 on t7.id = t8.id join t9 on t8.id = t9.id join t6 on t6.id = t7.id where t6.category = 'hot' order by t6.id limit 2", | ||
| "explain format = 'plan_tree' select /*+ TIDB_INLJ(t7, t8, t9) LEADING(t7, t8, t9, t6) */ t6.id, t7.payload, t8.payload, t9.payload from t7 join t8 on t7.id = t8.id join t9 on t8.id = t9.id join t6 on t6.id = t7.id where t6.category = 'hot' order by t6.id limit 2" | ||
| ] | ||
| } | ||
| ] |
164 changes: 164 additions & 0 deletions
164
pkg/planner/core/casetest/rule/testdata/order_aware_join_reorder_suite_out.json
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,164 @@ | ||
| [ | ||
| { | ||
| "Name": "TestOrderAwareCDCJoinReorder", | ||
|
AilinKid marked this conversation as resolved.
|
||
| "Cases": [ | ||
| { | ||
| "SQL": "SELECT t6.id, t7.payload, t8.payload FROM t7 JOIN t8 ON t7.id = t8.id JOIN t6 ON t6.id = t7.id WHERE t6.category = 'hot' ORDER BY t6.id LIMIT 2", | ||
| "Plan": [ | ||
| "Projection root test.t6.id, test.t7.payload, test.t8.payload", | ||
| "└─TopN root test.t6.id, offset:0, count:2", | ||
| " └─Projection root test.t7.payload, test.t8.payload, test.t6.id", | ||
| " └─MergeJoin root inner join, left key:test.t7.id, right key:test.t8.id", | ||
| " ├─TableReader(Build) root data:TableFullScan", | ||
| " │ └─TableFullScan cop[tikv] table:t8 keep order:true", | ||
| " └─MergeJoin(Probe) root inner join, left key:test.t6.id, right key:test.t7.id", | ||
| " ├─TableReader(Build) root data:TableFullScan", | ||
| " │ └─TableFullScan cop[tikv] table:t7 keep order:true", | ||
| " └─IndexReader(Probe) root index:IndexRangeScan", | ||
| " └─IndexRangeScan cop[tikv] table:t6, index:idx_category_id_payload(category, id, payload) range:[\"hot\",\"hot\"], keep order:true" | ||
| ], | ||
| "Result": [ | ||
| "1 100 1000", | ||
| "2 200 2000" | ||
| ] | ||
| }, | ||
| { | ||
| "SQL": "SELECT /*+ LEADING(t7, t8, t6) */ t6.id, t7.payload, t8.payload FROM t7 JOIN t8 ON t7.id = t8.id JOIN t6 ON t6.id = t7.id WHERE t6.category = 'hot' ORDER BY t6.id LIMIT 2", | ||
| "Plan": [ | ||
| "Projection root test.t6.id, test.t7.payload, test.t8.payload", | ||
| "└─Limit root offset:0, count:2", | ||
| " └─MergeJoin root inner join, left key:test.t7.id, right key:test.t6.id", | ||
| " ├─IndexReader(Build) root index:IndexRangeScan", | ||
| " │ └─IndexRangeScan cop[tikv] table:t6, index:idx_category_id_payload(category, id, payload) range:[\"hot\",\"hot\"], keep order:true", | ||
| " └─MergeJoin(Probe) root inner join, left key:test.t7.id, right key:test.t8.id", | ||
| " ├─TableReader(Build) root data:TableFullScan", | ||
| " │ └─TableFullScan cop[tikv] table:t8 keep order:true", | ||
| " └─TableReader(Probe) root data:TableFullScan", | ||
| " └─TableFullScan cop[tikv] table:t7 keep order:true" | ||
| ], | ||
| "Result": [ | ||
| "1 100 1000", | ||
| "2 200 2000" | ||
| ] | ||
| }, | ||
| { | ||
| "SQL": "SELECT /*+ TIDB_INLJ(t7, t8, t9) */ t6.id, t7.payload, t8.payload, t9.payload FROM t7 JOIN t8 ON t7.id = t8.id JOIN t9 ON t8.id = t9.id JOIN t6 ON t6.id = t7.id WHERE t6.category = 'hot' ORDER BY t6.id LIMIT 2", | ||
| "Plan": [ | ||
| "Projection root test.t6.id, test.t7.payload, test.t8.payload, test.t9.payload", | ||
| "└─Limit root offset:0, count:2", | ||
| " └─Projection root test.t7.payload, test.t8.payload, test.t9.payload, test.t6.id", | ||
| " └─IndexJoin root inner join, inner:TableReader, outer key:test.t8.id, inner key:test.t9.id, equal cond:eq(test.t8.id, test.t9.id)", | ||
| " ├─IndexJoin(Build) root inner join, inner:TableReader, outer key:test.t7.id, inner key:test.t8.id, equal cond:eq(test.t7.id, test.t8.id)", | ||
| " │ ├─IndexJoin(Build) root inner join, inner:TableReader, outer key:test.t6.id, inner key:test.t7.id, equal cond:eq(test.t6.id, test.t7.id)", | ||
| " │ │ ├─IndexReader(Build) root index:IndexRangeScan", | ||
| " │ │ │ └─IndexRangeScan cop[tikv] table:t6, index:idx_category_id_payload(category, id, payload) range:[\"hot\",\"hot\"], keep order:true", | ||
| " │ │ └─TableReader(Probe) root data:TableRangeScan", | ||
| " │ │ └─TableRangeScan cop[tikv] table:t7 range: decided by [test.t6.id], keep order:false", | ||
| " │ └─TableReader(Probe) root data:TableRangeScan", | ||
| " │ └─TableRangeScan cop[tikv] table:t8 range: decided by [test.t7.id], keep order:false", | ||
| " └─TableReader(Probe) root data:TableRangeScan", | ||
| " └─TableRangeScan cop[tikv] table:t9 range: decided by [test.t8.id], keep order:false" | ||
| ], | ||
| "Result": [ | ||
| "1 100 1000 10000", | ||
| "2 200 2000 20000" | ||
| ] | ||
| }, | ||
| { | ||
| "SQL": "SELECT /*+ TIDB_INLJ(t7, t8, t9) LEADING(t7, t8, t9, t6) */ t6.id, t7.payload, t8.payload, t9.payload FROM t7 JOIN t8 ON t7.id = t8.id JOIN t9 ON t8.id = t9.id JOIN t6 ON t6.id = t7.id WHERE t6.category = 'hot' ORDER BY t6.id LIMIT 2", | ||
| "Plan": [ | ||
| "Projection root test.t6.id, test.t7.payload, test.t8.payload, test.t9.payload", | ||
| "└─Limit root offset:0, count:2", | ||
| " └─MergeJoin root inner join, left key:test.t7.id, right key:test.t6.id", | ||
| " ├─IndexReader(Build) root index:IndexRangeScan", | ||
| " │ └─IndexRangeScan cop[tikv] table:t6, index:idx_category_id_payload(category, id, payload) range:[\"hot\",\"hot\"], keep order:true", | ||
| " └─IndexJoin(Probe) root inner join, inner:TableReader, outer key:test.t8.id, inner key:test.t9.id, equal cond:eq(test.t8.id, test.t9.id)", | ||
| " ├─IndexJoin(Build) root inner join, inner:TableReader, outer key:test.t7.id, inner key:test.t8.id, equal cond:eq(test.t7.id, test.t8.id)", | ||
| " │ ├─TableReader(Build) root data:TableFullScan", | ||
| " │ │ └─TableFullScan cop[tikv] table:t7 keep order:true", | ||
| " │ └─TableReader(Probe) root data:TableRangeScan", | ||
| " │ └─TableRangeScan cop[tikv] table:t8 range: decided by [test.t7.id], keep order:false", | ||
| " └─TableReader(Probe) root data:TableRangeScan", | ||
| " └─TableRangeScan cop[tikv] table:t9 range: decided by [test.t8.id], keep order:false" | ||
| ], | ||
| "Result": [ | ||
| "1 100 1000 10000", | ||
| "2 200 2000 20000" | ||
| ] | ||
| } | ||
| ] | ||
| }, | ||
| { | ||
| "Name": "TestOrderAwareJoinReorderPushSelection", | ||
| "Cases": [ | ||
| { | ||
| "SQL": "explain format = 'plan_tree' select t6.id, t7.payload, t8.payload from t7 join t8 on t7.id = t8.id join t6 on t6.id = t7.id where t6.category = 'hot' order by t6.id limit 2", | ||
| "Plan": [ | ||
| "Projection root test.t6.id, test.t7.payload, test.t8.payload", | ||
| "└─TopN root test.t6.id, offset:0, count:2", | ||
| " └─Projection root test.t7.payload, test.t8.payload, test.t6.id", | ||
| " └─MergeJoin root inner join, left key:test.t7.id, right key:test.t8.id", | ||
| " ├─TableReader(Build) root data:TableFullScan", | ||
| " │ └─TableFullScan cop[tikv] table:t8 keep order:true", | ||
| " └─MergeJoin(Probe) root inner join, left key:test.t6.id, right key:test.t7.id", | ||
| " ├─TableReader(Build) root data:TableFullScan", | ||
| " │ └─TableFullScan cop[tikv] table:t7 keep order:true", | ||
| " └─IndexReader(Probe) root index:IndexRangeScan", | ||
| " └─IndexRangeScan cop[tikv] table:t6, index:idx_category_id_payload(category, id, payload) range:[\"hot\",\"hot\"], keep order:true" | ||
| ] | ||
| }, | ||
| { | ||
| "SQL": "explain format = 'plan_tree' select /*+ LEADING(t7, t8, t6) */ t6.id, t7.payload, t8.payload from t7 join t8 on t7.id = t8.id join t6 on t6.id = t7.id where t6.category = 'hot' order by t6.id limit 2", | ||
| "Plan": [ | ||
| "Projection root test.t6.id, test.t7.payload, test.t8.payload", | ||
| "└─Limit root offset:0, count:2", | ||
| " └─MergeJoin root inner join, left key:test.t7.id, right key:test.t6.id", | ||
| " ├─IndexReader(Build) root index:IndexRangeScan", | ||
| " │ └─IndexRangeScan cop[tikv] table:t6, index:idx_category_id_payload(category, id, payload) range:[\"hot\",\"hot\"], keep order:true", | ||
| " └─MergeJoin(Probe) root inner join, left key:test.t7.id, right key:test.t8.id", | ||
| " ├─TableReader(Build) root data:TableFullScan", | ||
| " │ └─TableFullScan cop[tikv] table:t8 keep order:true", | ||
| " └─TableReader(Probe) root data:TableFullScan", | ||
| " └─TableFullScan cop[tikv] table:t7 keep order:true" | ||
| ] | ||
| }, | ||
| { | ||
| "SQL": "explain format = 'plan_tree' select /*+ TIDB_INLJ(t7, t8, t9) */ t6.id, t7.payload, t8.payload, t9.payload from t7 join t8 on t7.id = t8.id join t9 on t8.id = t9.id join t6 on t6.id = t7.id where t6.category = 'hot' order by t6.id limit 2", | ||
| "Plan": [ | ||
| "Projection root test.t6.id, test.t7.payload, test.t8.payload, test.t9.payload", | ||
| "└─Limit root offset:0, count:2", | ||
| " └─Projection root test.t7.payload, test.t8.payload, test.t9.payload, test.t6.id", | ||
| " └─IndexJoin root inner join, inner:TableReader, outer key:test.t8.id, inner key:test.t9.id, equal cond:eq(test.t8.id, test.t9.id)", | ||
| " ├─IndexJoin(Build) root inner join, inner:TableReader, outer key:test.t7.id, inner key:test.t8.id, equal cond:eq(test.t7.id, test.t8.id)", | ||
| " │ ├─IndexJoin(Build) root inner join, inner:TableReader, outer key:test.t6.id, inner key:test.t7.id, equal cond:eq(test.t6.id, test.t7.id)", | ||
| " │ │ ├─IndexReader(Build) root index:IndexRangeScan", | ||
| " │ │ │ └─IndexRangeScan cop[tikv] table:t6, index:idx_category_id_payload(category, id, payload) range:[\"hot\",\"hot\"], keep order:true", | ||
| " │ │ └─TableReader(Probe) root data:TableRangeScan", | ||
| " │ │ └─TableRangeScan cop[tikv] table:t7 range: decided by [test.t6.id], keep order:false", | ||
| " │ └─TableReader(Probe) root data:TableRangeScan", | ||
| " │ └─TableRangeScan cop[tikv] table:t8 range: decided by [test.t7.id], keep order:false", | ||
| " └─TableReader(Probe) root data:TableRangeScan", | ||
| " └─TableRangeScan cop[tikv] table:t9 range: decided by [test.t8.id], keep order:false" | ||
| ] | ||
| }, | ||
| { | ||
| "SQL": "explain format = 'plan_tree' select /*+ TIDB_INLJ(t7, t8, t9) LEADING(t7, t8, t9, t6) */ t6.id, t7.payload, t8.payload, t9.payload from t7 join t8 on t7.id = t8.id join t9 on t8.id = t9.id join t6 on t6.id = t7.id where t6.category = 'hot' order by t6.id limit 2", | ||
| "Plan": [ | ||
| "Projection root test.t6.id, test.t7.payload, test.t8.payload, test.t9.payload", | ||
| "└─Limit root offset:0, count:2", | ||
| " └─MergeJoin root inner join, left key:test.t7.id, right key:test.t6.id", | ||
| " ├─IndexReader(Build) root index:IndexRangeScan", | ||
| " │ └─IndexRangeScan cop[tikv] table:t6, index:idx_category_id_payload(category, id, payload) range:[\"hot\",\"hot\"], keep order:true", | ||
| " └─IndexJoin(Probe) root inner join, inner:TableReader, outer key:test.t8.id, inner key:test.t9.id, equal cond:eq(test.t8.id, test.t9.id)", | ||
| " ├─IndexJoin(Build) root inner join, inner:TableReader, outer key:test.t7.id, inner key:test.t8.id, equal cond:eq(test.t7.id, test.t8.id)", | ||
| " │ ├─TableReader(Build) root data:TableFullScan", | ||
| " │ │ └─TableFullScan cop[tikv] table:t7 keep order:true", | ||
| " │ └─TableReader(Probe) root data:TableRangeScan", | ||
| " │ └─TableRangeScan cop[tikv] table:t8 range: decided by [test.t7.id], keep order:false", | ||
| " └─TableReader(Probe) root data:TableRangeScan", | ||
| " └─TableRangeScan cop[tikv] table:t9 range: decided by [test.t8.id], keep order:false" | ||
| ] | ||
| } | ||
| ] | ||
| } | ||
| ] | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.