ddl,tables: only write the index when it meets partial index condition by YangKeao · Pull Request #62762 · pingcap/tidb

YangKeao · 2025-08-01T03:30:07Z

This is the second PR for partial index

#63448
#62759
-> #62762

What problem does this PR solve?

Issue Number: close #62761

Problem Summary:

What changed and how does it work?

Modify the tables.index to parse/store the expression of partialConditionExpression.
Add a MeetPartialCondition to check whether it'll meet the condition.
Add some tests about the DML during the DDL on the same table.
Modify the row count of admin show ddl job from the added count to the scanned count.
Support admin check table for partial index.

Check List

Tests

Unit test
Integration test
Manual test (add detailed scripts or steps below)
No need to test
- I checked and no code files have been changed.

Side effects

Performance regression: Consumes more CPU
Performance regression: Consumes more Memory
Breaking backward compatibility

Documentation

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

None

codecov · 2025-08-01T03:47:13Z

Codecov Report

❌ Patch coverage is 52.76873% with 290 lines in your changes missing coverage. Please review.
✅ Project coverage is 74.9055%. Comparing base (f2abf53) to head (5e8e090).
⚠️ Report is 3 commits behind head on master.

Additional details and impacted files

@@               Coverage Diff                @@
##             master     #62762        +/-   ##
================================================
+ Coverage   72.7181%   74.9055%   +2.1874%     
================================================
  Files          1864       1888        +24     
  Lines        505545     514158      +8613     
================================================
+ Hits         367623     385133     +17510     
+ Misses       115594     105286     -10308     
- Partials      22328      23739      +1411

Flag	Coverage Δ
integration	`48.1293% <27.2413%> (?)`
unit	`72.5268% <46.5798%> (+0.2327%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
dumpling	`52.8700% <ø> (ø)`
parser	`∅ <ø> (∅)`
br	`62.9573% <ø> (+16.7468%)`	⬆️

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

YangKeao · 2025-08-11T06:11:06Z

/retest

YangKeao · 2025-10-30T14:28:31Z

/retest

YangKeao · 2025-10-30T15:01:24Z

/retest

YangKeao · 2025-10-30T18:02:18Z

I have no idea why mysql-test failed for now. It seems that the newest master commit also failed.

YangKeao · 2025-11-03T06:44:32Z

/retest

wjhuang2016 · 2025-11-03T12:42:54Z

 		for i, index := range indexes {
+			// If the `IndexRecordChunk.conditionPushed` is true and we have only 1 index, the `indexConditionCheckers`
+			// will not be initialized.
+			if index.Meta().HasCondition() && indexConditionCheckers != nil {


Check indexConditionCheckers != nil is enough.

It's not enough.

indexConditionCheckers is initialized with make([]func(row chunk.Row) (bool, error), len(w.indexes)). We should either check index.Meta().HasCondition() or indexConditionCheckers[i] != nil (they are the same meaning).

wjhuang2016

rest LGTM

YangKeao · 2025-11-06T13:35:37Z

/hold

Waiting for @fzzf678 to review.

winoros

for the planner part.

ti-chi-bot · 2025-11-06T13:44:36Z

[LGTM Timeline notifier]

Timeline:

2025-11-05 12:43:55.226073926 +0000 UTC m=+274084.669103804: ☑️ agreed by wjhuang2016.
2025-11-06 13:44:35.117833051 +0000 UTC m=+364124.560862931: ☑️ agreed by winoros.

ti-chi-bot · 2025-11-10T07:47:21Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: fzzf678, Leavrth, winoros, wjhuang2016

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~OWNERS~~ [Leavrth,winoros,wjhuang2016]
~~br/OWNERS~~ [Leavrth]
~~pkg/ddl/OWNERS~~ [wjhuang2016]
~~pkg/infoschema/OWNERS~~ [wjhuang2016]
~~pkg/meta/OWNERS~~ [wjhuang2016]
~~pkg/planner/OWNERS~~ [winoros]
~~pkg/table/OWNERS~~ [wjhuang2016]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

fzzf678 · 2025-11-10T07:47:25Z

/unhold

YangKeao · 2025-11-10T12:18:45Z

/retest

YangKeao · 2025-11-11T04:34:29Z

/retest

YangKeao · 2025-11-11T11:28:33Z

/retest

YangKeao · 2025-11-11T12:00:06Z

/retest

ti-chi-bot · 2025-11-11T12:30:18Z

@YangKeao: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
non-block/pull-integration-e2e-test-next-gen	`512d005`	link	false	`/test pull-integration-e2e-test-next-gen`
non-block/pull-unit-test-next-gen	`512d005`	link	false	`/test pull-unit-test-next-gen`
non-block/pull-mysql-client-test-next-gen	`512d005`	link	false	`/test pull-mysql-client-test-next-gen`
non-block/pull-mysql-test-next-gen	`512d005`	link	false	`/test pull-mysql-test-next-gen`
non-block/pull-integration-realcluster-test-next-gen	`12863b6`	link	false	`/test pull-integration-realcluster-test-next-gen`

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Signed-off-by: Yang Keao <yangkeao@chunibyo.icu>

ti-chi-bot · 2026-06-01T05:51:46Z

In response to a cherrypick label: new pull request created to branch release-8.5: #68833.
But this PR has conflicts, please resolve them!

ti-chi-bot Bot added release-note-none Denotes a PR that doesn't merit a release note. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. sig/planner SIG: Planner labels Aug 1, 2025

YangKeao mentioned this pull request Aug 1, 2025

ddl,parser: modify schema to store the partial condition #62759

Merged

13 tasks

YangKeao force-pushed the partial-index-2 branch 7 times, most recently from ac0a733 to 6009afe Compare August 8, 2025 04:11

YangKeao force-pushed the partial-index-2 branch from 6009afe to 9785325 Compare August 11, 2025 04:16

YangKeao force-pushed the partial-index-2 branch 13 times, most recently from 9f57ee7 to db0e6ef Compare August 27, 2025 15:51

YangKeao force-pushed the partial-index-2 branch 3 times, most recently from 4f0082c to 7807dc6 Compare August 28, 2025 12:24

wjhuang2016 reviewed Nov 3, 2025

View reviewed changes

wjhuang2016 approved these changes Nov 5, 2025

View reviewed changes

winoros reviewed Nov 6, 2025

View reviewed changes

Comment thread br/tests/br_partial_index/run.sh Outdated

winoros approved these changes Nov 6, 2025

View reviewed changes

fzzf678 reviewed Nov 7, 2025

View reviewed changes

Comment thread pkg/ddl/backfilling_operators.go Outdated

Comment thread pkg/ddl/index_cop.go

Leavrth approved these changes Nov 10, 2025

View reviewed changes

fzzf678 approved these changes Nov 10, 2025

View reviewed changes

support writing data / deleting data with partial index

5e8e090

Signed-off-by: Yang Keao <yangkeao@chunibyo.icu>

This was referenced Jan 21, 2026

parser: modify parser to support partial index (#63448) #65698

Closed

ddl,parser: modify schema to store the partial condition (#62759) #65699

Closed

ddl,tables: only write the index when it meets partial index condition (#62762) #65700

Merged

YangKeao mentioned this pull request Jun 1, 2026

parser: modify parser to support partial index (#63448) #68823

Merged

13 tasks

This was referenced Jun 1, 2026

parser: modify parser to support partial index (#63448) #68830

Open

ddl,parser: modify schema to store the partial condition (#62759) #68831

Open

ddl,tables: only write the index when it meets partial index condition (#62762) #68833

Open

Conversation

YangKeao commented Aug 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

What changed and how does it work?

Check List

Release note

Uh oh!

codecov Bot commented Aug 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

YangKeao commented Aug 11, 2025

Uh oh!

YangKeao commented Oct 30, 2025

Uh oh!

YangKeao commented Oct 30, 2025

Uh oh!

YangKeao commented Oct 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

YangKeao commented Nov 3, 2025

Uh oh!

Uh oh!

wjhuang2016 Nov 3, 2025

Choose a reason for hiding this comment

Uh oh!

YangKeao Nov 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

wjhuang2016 left a comment

Choose a reason for hiding this comment

Uh oh!

YangKeao commented Nov 6, 2025

Uh oh!

Uh oh!

winoros left a comment

Choose a reason for hiding this comment

Uh oh!

ti-chi-bot Bot commented Nov 6, 2025

[LGTM Timeline notifier]

Uh oh!

Uh oh!

Uh oh!

ti-chi-bot Bot commented Nov 10, 2025

Uh oh!

fzzf678 commented Nov 10, 2025

Uh oh!

YangKeao commented Nov 10, 2025

Uh oh!

YangKeao commented Nov 11, 2025

Uh oh!

YangKeao commented Nov 11, 2025

Uh oh!

YangKeao commented Nov 11, 2025

Uh oh!

ti-chi-bot Bot commented Nov 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ti-chi-bot commented Jun 1, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

YangKeao commented Aug 1, 2025 •

edited

Loading

codecov Bot commented Aug 1, 2025 •

edited

Loading

YangKeao commented Oct 30, 2025 •

edited

Loading

YangKeao Nov 3, 2025 •

edited

Loading

ti-chi-bot Bot commented Nov 11, 2025 •

edited

Loading