planner: stop pushing TopN down through Projection if it has undeterministic functions #53362

qw4990 · 2024-05-17T08:22:00Z

What problem does this PR solve?

Issue Number: close #37986

Problem Summary: planner: stop pushing TopN down through Projection if it has undeterministic functions

What changed and how does it work?

planner: stop pushing TopN down through Projection if it has undeterministic functions

Check List

Tests

Unit test
Integration test
Manual test (add detailed scripts or steps below)
No need to test
- I checked and no code files have been changed.

Side effects

Performance regression: Consumes more CPU
Performance regression: Consumes more Memory
Breaking backward compatibility

Documentation

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

None

tiprow · 2024-05-17T08:22:15Z

Hi @qw4990. Thanks for your PR.

PRs from untrusted users cannot be marked as trusted with /ok-to-test in this repo meaning untrusted PR authors can never trigger tests themselves. Collaborators can still trigger tests on the PR using /test all.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

qw4990 · 2024-05-17T08:45:01Z

/test pull-mysql-client-test

tiprow · 2024-05-17T08:45:24Z

@qw4990: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

In response to this:

/test pull-mysql-client-test

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

codecov · 2024-05-17T09:35:24Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 74.6884%. Comparing base (397a460) to head (396608b).
Report is 5 commits behind head on master.

Additional details and impacted files

@@               Coverage Diff                @@
##             master     #53362        +/-   ##
================================================
+ Coverage   72.5545%   74.6884%   +2.1339%     
================================================
  Files          1505       1505                
  Lines        429830     430006       +176     
================================================
+ Hits         311861     321165      +9304     
+ Misses        98694      89003      -9691     
- Partials      19275      19838       +563

Flag	Coverage Δ
integration	`49.4350% <55.5555%> (?)`
unit	`71.4094% <100.0000%> (-0.0370%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
dumpling	`53.9957% <ø> (ø)`
parser	`∅ <ø> (∅)`
br	`50.4368% <ø> (+9.0328%)`	⬆️

hawkingrei · 2024-05-17T10:01:04Z

/ok-to-test

time-and-fate

Please fix the failed cases.

pkg/planner/core/rule_topn_push_down.go

time-and-fate · 2024-05-20T04:20:32Z

pkg/planner/core/rule_topn_push_down.go

-			by.Expr = expression.FoldConstant(exprCtx, expression.ColumnSubstitute(exprCtx, by.Expr, p.schema, p.Exprs))
+			substituted := expression.ColumnSubstitute(exprCtx, by.Expr, p.schema, p.Exprs)


Why is FoldConstant() removed?

My fault, it has been updated now.

pkg/planner/core/rule_topn_push_down.go

AilinKid

Rest LGTM

AilinKid · 2024-05-20T05:18:39Z

pkg/planner/core/rule_topn_push_down.go

-			by.Expr = expression.FoldConstant(exprCtx, expression.ColumnSubstitute(exprCtx, by.Expr, p.schema, p.Exprs))
+			substituted := expression.ColumnSubstitute(exprCtx, by.Expr, p.schema, p.Exprs)


Co-authored-by: Zhou Kunqin <25057648+time-and-fate@users.noreply.github.com>

ti-chi-bot · 2024-05-20T06:47:26Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: AilinKid, time-and-fate

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~pkg/planner/OWNERS~~ [AilinKid,time-and-fate]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

ti-chi-bot · 2024-05-20T06:47:30Z

[LGTM Timeline notifier]

Timeline:

2024-05-20 06:21:05.703428775 +0000 UTC m=+2066219.460564332: ☑️ agreed by AilinKid.
2024-05-20 06:47:29.333737419 +0000 UTC m=+2067803.090872992: ☑️ agreed by time-and-fate.

qw4990 · 2024-05-20T07:28:02Z

/test check-dev

tiprow · 2024-05-20T07:28:24Z

@qw4990: The specified target(s) for /test were not found.
The following commands are available to trigger required jobs:

/test fast_test_tiprow
/test tidb_parser_test

Use /test all to run all jobs.

In response to this:

/test check-dev

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

qw4990 · 2024-05-20T13:57:37Z

/test unit-test

tiprow · 2024-05-20T13:57:59Z

@qw4990: The specified target(s) for /test were not found.
The following commands are available to trigger required jobs:

/test fast_test_tiprow
/test tidb_parser_test

Use /test all to run all jobs.

In response to this:

/test unit-test

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

…inistic functions (pingcap#53362) close pingcap#37986

Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>

ti-chi-bot · 2024-05-20T23:55:42Z

In response to a cherrypick label: new pull request created to branch release-7.5: #53412.

…inistic functions (#53362) (#53412) close #37986

…inistic functions (pingcap#53362) close pingcap#37986

Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>

ti-chi-bot · 2024-06-04T15:23:01Z

In response to a cherrypick label: new pull request created to branch release-6.5: #53811.

…inistic functions (#53362) (#53811) close #37986

fixup

ea40bc4

ti-chi-bot bot added do-not-merge/needs-tests-checked release-note-none sig/planner SIG: Planner size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels May 17, 2024

qw4990 added 2 commits May 17, 2024 16:30

fixup

10193e6

fixup

8a3e40b

ti-chi-bot bot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels May 17, 2024

fixup

27ec9c1

ti-chi-bot bot removed the do-not-merge/needs-tests-checked label May 17, 2024

ti-chi-bot bot added the ok-to-test label May 17, 2024

time-and-fate reviewed May 17, 2024

View reviewed changes

pkg/planner/core/rule_topn_push_down.go Outdated Show resolved Hide resolved

fixup

05aab37

time-and-fate reviewed May 20, 2024

View reviewed changes

AilinKid reviewed May 20, 2024

View reviewed changes

qw4990 and others added 3 commits May 20, 2024 14:16

Update pkg/planner/core/rule_topn_push_down.go

7bbfaf9

Co-authored-by: Zhou Kunqin <25057648+time-and-fate@users.noreply.github.com>

fixup

2d5b5f0

Merge remote-tracking branch 'upstream/master' into fix-37986

396608b

AilinKid approved these changes May 20, 2024

View reviewed changes

ti-chi-bot bot added approved needs-1-more-lgtm labels May 20, 2024

time-and-fate approved these changes May 20, 2024

View reviewed changes

ti-chi-bot bot added the lgtm label May 20, 2024

ti-chi-bot bot removed the needs-1-more-lgtm label May 20, 2024

qw4990 added the type/bug-fix This PR fixes a bug. label May 20, 2024

ti-chi-bot bot merged commit b3d740f into pingcap:master May 20, 2024
23 checks passed

terry1purcell pushed a commit to terry1purcell/tidb that referenced this pull request May 20, 2024

planner: stop pushing TopN down through Projection if it has undeterm…

ace0d7d

…inistic functions (pingcap#53362) close pingcap#37986

ti-chi-bot added the needs-cherry-pick-release-7.5 label May 20, 2024

ti-chi-bot pushed a commit to ti-chi-bot/tidb that referenced this pull request May 20, 2024

This is an automated cherry-pick of pingcap#53362

3ba4eb7

Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>

ti-chi-bot mentioned this pull request May 20, 2024

planner: stop pushing TopN down through Projection if it has undeterministic functions (#53362) #53412

Merged

13 tasks

ti-chi-bot bot pushed a commit that referenced this pull request May 21, 2024

planner: stop pushing TopN down through Projection if it has undeterm…

2ec5f23

…inistic functions (#53362) (#53412) close #37986

RidRisR pushed a commit to RidRisR/tidb that referenced this pull request May 23, 2024

planner: stop pushing TopN down through Projection if it has undeterm…

a6c3107

…inistic functions (pingcap#53362) close pingcap#37986

ti-chi-bot added the needs-cherry-pick-release-6.5 label Jun 4, 2024

ti-chi-bot pushed a commit to ti-chi-bot/tidb that referenced this pull request Jun 4, 2024

This is an automated cherry-pick of pingcap#53362

c7b28e9

Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>

ti-chi-bot mentioned this pull request Jun 4, 2024

planner: stop pushing TopN down through Projection if it has undeterministic functions (#53362) #53811

Merged

13 tasks

ti-chi-bot bot pushed a commit that referenced this pull request Jun 10, 2024

planner: stop pushing TopN down through Projection if it has undeterm…

236b1b7

…inistic functions (#53362) (#53811) close #37986

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

planner: stop pushing TopN down through Projection if it has undeterministic functions #53362

planner: stop pushing TopN down through Projection if it has undeterministic functions #53362

qw4990 commented May 17, 2024 •

edited by hawkingrei

tiprow bot commented May 17, 2024

qw4990 commented May 17, 2024

tiprow bot commented May 17, 2024

codecov bot commented May 17, 2024 •

edited

hawkingrei commented May 17, 2024

time-and-fate left a comment

time-and-fate May 20, 2024 •

edited

AilinKid May 20, 2024

qw4990 May 20, 2024

AilinKid left a comment

AilinKid May 20, 2024

ti-chi-bot bot commented May 20, 2024

ti-chi-bot bot commented May 20, 2024

qw4990 commented May 20, 2024

tiprow bot commented May 20, 2024

qw4990 commented May 20, 2024

tiprow bot commented May 20, 2024

ti-chi-bot commented May 20, 2024

ti-chi-bot commented Jun 4, 2024

		by.Expr = expression.FoldConstant(exprCtx, expression.ColumnSubstitute(exprCtx, by.Expr, p.schema, p.Exprs))
		substituted := expression.ColumnSubstitute(exprCtx, by.Expr, p.schema, p.Exprs)

planner: stop pushing TopN down through Projection if it has undeterministic functions #53362

planner: stop pushing TopN down through Projection if it has undeterministic functions #53362

Conversation

qw4990 commented May 17, 2024 • edited by hawkingrei

What problem does this PR solve?

What changed and how does it work?

Check List

Release note

tiprow bot commented May 17, 2024

qw4990 commented May 17, 2024

tiprow bot commented May 17, 2024

codecov bot commented May 17, 2024 • edited

Codecov Report

hawkingrei commented May 17, 2024

time-and-fate left a comment

Choose a reason for hiding this comment

time-and-fate May 20, 2024 • edited

Choose a reason for hiding this comment

AilinKid May 20, 2024

Choose a reason for hiding this comment

qw4990 May 20, 2024

Choose a reason for hiding this comment

AilinKid left a comment

Choose a reason for hiding this comment

AilinKid May 20, 2024

Choose a reason for hiding this comment

ti-chi-bot bot commented May 20, 2024

ti-chi-bot bot commented May 20, 2024

[LGTM Timeline notifier]

qw4990 commented May 20, 2024

tiprow bot commented May 20, 2024

qw4990 commented May 20, 2024

tiprow bot commented May 20, 2024

ti-chi-bot commented May 20, 2024

ti-chi-bot commented Jun 4, 2024

qw4990 commented May 17, 2024 •

edited by hawkingrei

codecov bot commented May 17, 2024 •

edited

time-and-fate May 20, 2024 •

edited