Skip to content

[fix] (topn) fix uncleared block in topn_next()#39119

Merged
xiaokang merged 1 commit intoapache:masterfrom
csun5285:fix_topn
Aug 11, 2024
Merged

[fix] (topn) fix uncleared block in topn_next()#39119
xiaokang merged 1 commit intoapache:masterfrom
csun5285:fix_topn

Conversation

@csun5285
Copy link
Copy Markdown
Contributor

@csun5285 csun5285 commented Aug 8, 2024 •
edited
Loading

Proposed changes

BUG

When reusing blocks, the extra columns were not cleaned up.

Issue Number: close #xxx

@doris-robot
Copy link
Copy Markdown

doris-robot commented Aug 8, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@csun5285
Copy link
Copy Markdown
Contributor Author

csun5285 commented Aug 8, 2024

run buildall

@github-actions github-actions bot added the doing label Aug 8, 2024
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Aug 8, 2024

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link
Copy Markdown

doris-robot commented Aug 8, 2024

TPC-H: Total hot run time: 39632 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 8c8a01aa7c5836874b1e6fd5e32e8c2e0827bd55, data reload: false

------ Round 1 ----------------------------------
q1	17653	4515	4332	4332
q2	2013	172	193	172
q3	10544	1152	1121	1121
q4	10166	665	790	665
q5	7537	2615	2560	2560
q6	224	140	140	140
q7	971	587	597	587
q8	9220	1937	1947	1937
q9	8889	6583	6594	6583
q10	7063	2196	2209	2196
q11	456	240	250	240
q12	401	218	218	218
q13	17767	2971	2960	2960
q14	288	237	232	232
q15	530	484	462	462
q16	486	385	372	372
q17	990	711	698	698
q18	8150	7580	7365	7365
q19	6270	1113	1063	1063
q20	679	336	326	326
q21	5418	4412	4724	4412
q22	1140	1012	991	991
Total cold run time: 116855 ms
Total hot run time: 39632 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4552	4282	4297	4282
q2	372	277	269	269
q3	2869	2653	2633	2633
q4	1858	1630	1612	1612
q5	5298	5292	5332	5292
q6	225	137	136	136
q7	2068	1665	1692	1665
q8	3192	3375	3372	3372
q9	8475	8374	8412	8374
q10	3398	3167	3155	3155
q11	624	504	486	486
q12	801	601	599	599
q13	16248	2956	2982	2956
q14	306	277	275	275
q15	517	477	484	477
q16	497	419	415	415
q17	1780	1480	1477	1477
q18	7662	7530	7558	7530
q19	1739	1572	1551	1551
q20	2032	1780	1766	1766
q21	9721	5164	5042	5042
q22	1113	1000	1014	1000
Total cold run time: 75347 ms
Total hot run time: 54364 ms

@doris-robot
Copy link
Copy Markdown

doris-robot commented Aug 8, 2024

TPC-DS: Total hot run time: 201389 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 8c8a01aa7c5836874b1e6fd5e32e8c2e0827bd55, data reload: false

query1	955	413	397	397
query2	6456	2061	1936	1936
query3	6639	207	216	207
query4	34283	23103	23023	23023
query5	4204	493	496	493
query6	286	200	177	177
query7	4587	295	289	289
query8	250	200	200	200
query9	8500	2380	2346	2346
query10	967	903	898	898
query11	17868	15069	15054	15054
query12	149	99	92	92
query13	1633	387	366	366
query14	10577	6204	6979	6204
query15	332	334	330	330
query16	7575	449	475	449
query17	1306	544	532	532
query18	1729	371	343	343
query19	232	173	175	173
query20	110	108	108	108
query21	204	109	98	98
query22	4260	4041	4000	4000
query23	33791	33149	33046	33046
query24	12227	2983	2964	2964
query25	638	361	356	356
query26	1804	149	146	146
query27	3025	275	274	274
query28	7456	2000	1996	1996
query29	1052	403	440	403
query30	283	147	142	142
query31	989	746	747	746
query32	91	56	54	54
query33	739	288	273	273
query34	985	467	472	467
query35	988	874	870	870
query36	1122	953	929	929
query37	167	79	77	77
query38	4394	4144	4205	4144
query39	1452	1368	1369	1368
query40	267	115	111	111
query41	46	45	42	42
query42	116	94	100	94
query43	516	482	461	461
query44	1246	736	745	736
query45	420	391	367	367
query46	1122	811	772	772
query47	1835	1764	1797	1764
query48	373	296	301	296
query49	1175	460	416	416
query50	817	399	405	399
query51	6869	6714	6683	6683
query52	101	92	92	92
query53	253	183	184	183
query54	1014	442	438	438
query55	77	75	76	75
query56	263	250	246	246
query57	1161	1074	1030	1030
query58	245	227	238	227
query59	3061	3075	2834	2834
query60	288	280	266	266
query61	99	93	96	93
query62	848	649	651	649
query63	213	183	180	180
query64	10685	2383	1906	1906
query65	3248	3112	3171	3112
query66	1320	340	358	340
query67	15216	15277	14885	14885
query68	4566	538	543	538
query69	434	366	380	366
query70	1169	1173	1199	1173
query71	408	264	263	263
query72	17943	16483	16803	16483
query73	761	326	328	326
query74	9299	8798	8788	8788
query75	3424	2664	2697	2664
query76	2650	995	1022	995
query77	513	320	317	317
query78	10347	10079	8929	8929
query79	2968	523	519	519
query80	1783	521	507	507
query81	595	231	238	231
query82	723	139	135	135
query83	272	170	164	164
query84	285	93	86	86
query85	1995	353	352	352
query86	487	303	293	293
query87	4676	4586	4534	4534
query88	4296	2505	2517	2505
query89	402	287	288	287
query90	1880	196	197	196
query91	152	125	133	125
query92	66	54	51	51
query93	4703	534	533	533
query94	942	307	311	307
query95	363	273	270	270
query96	618	285	273	273
query97	3192	3050	3060	3050
query98	221	205	203	203
query99	1666	1259	1282	1259
Total cold run time: 320998 ms
Total hot run time: 201389 ms

@doris-robot
Copy link
Copy Markdown

doris-robot commented Aug 8, 2024

ClickBench: Total hot run time: 30.71 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 8c8a01aa7c5836874b1e6fd5e32e8c2e0827bd55, data reload: false

query1	0.05	0.04	0.04
query2	0.08	0.04	0.04
query3	0.22	0.05	0.05
query4	1.68	0.07	0.07
query5	0.50	0.49	0.48
query6	1.12	0.73	0.72
query7	0.02	0.02	0.01
query8	0.05	0.06	0.04
query9	0.55	0.49	0.48
query10	0.54	0.55	0.53
query11	0.15	0.11	0.11
query12	0.16	0.13	0.12
query13	0.61	0.59	0.58
query14	0.76	0.78	0.78
query15	0.85	0.82	0.81
query16	0.35	0.37	0.38
query17	1.03	0.96	0.96
query18	0.24	0.22	0.22
query19	1.78	1.65	1.70
query20	0.02	0.01	0.00
query21	15.40	0.75	0.65
query22	4.83	7.05	2.03
query23	18.27	1.34	1.24
query24	2.11	0.24	0.22
query25	0.16	0.09	0.08
query26	0.29	0.21	0.20
query27	0.45	0.22	0.22
query28	13.34	1.02	1.00
query29	12.59	3.33	3.32
query30	0.24	0.05	0.05
query31	2.90	0.39	0.37
query32	3.27	0.49	0.46
query33	2.86	2.91	2.89
query34	17.07	4.40	4.40
query35	4.44	4.46	4.48
query36	0.66	0.49	0.49
query37	0.20	0.16	0.15
query38	0.15	0.15	0.15
query39	0.04	0.03	0.04
query40	0.15	0.12	0.12
query41	0.10	0.04	0.05
query42	0.05	0.05	0.04
query43	0.05	0.04	0.04
Total cold run time: 110.38 s
Total hot run time: 30.71 s

airborne12
airborne12 approved these changes Aug 9, 2024
View reviewed changes
Copy link
Copy Markdown
Member

@airborne12 airborne12 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Aug 9, 2024
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Aug 9, 2024

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Aug 9, 2024

PR approved by anyone and no changes requested.

zzzxl1993
zzzxl1993 approved these changes Aug 9, 2024
View reviewed changes
Copy link
Copy Markdown
Contributor

@zzzxl1993 zzzxl1993 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

xiaokang
xiaokang approved these changes Aug 11, 2024
View reviewed changes
Copy link
Copy Markdown
Contributor

@xiaokang xiaokang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@xiaokang xiaokang merged commit 476a159 into apache:master Aug 11, 2024
xiaokang pushed a commit to xiaokang/doris that referenced this pull request Aug 11, 2024
@csun5285 @xiaokang
[fix] (topn) fix uncleared block in topn_next() (apache#39119)
03f74b4 
When reusing blocks in topn, the extra columns were not cleaned up.
@xiaokang xiaokang mentioned this pull request Aug 11, 2024
Merged
xiaokang added a commit that referenced this pull request Aug 11, 2024
@xiaokang
[fix] (topn) fix uncleared block in topn_next() #39119 (#39199)
2e5493d
This was referenced Aug 12, 2024
Merged
Merged
airborne12 pushed a commit that referenced this pull request Aug 13, 2024
@csun5285
[fix] (topn) fix uncleared block in topn_next() (#39119) (#39224)
a6155a5 
## Proposed changes

pick from master #39119
airborne12 pushed a commit that referenced this pull request Aug 13, 2024
@csun5285
[fix] (topn) fix uncleared block in topn_next() (#39119) (#39226)
970cadd 
## Proposed changes

pick from master #39119
wyxxxcat pushed a commit to wyxxxcat/doris that referenced this pull request Aug 14, 2024
@csun5285 @wyxxxcat
[fix] (topn) fix uncleared block in topn_next() (apache#39119)
81713e1 
When reusing blocks in topn, the extra columns were not cleaned up.
@gavinchou gavinchou mentioned this pull request Aug 19, 2024
Closed
GoGoWen pushed a commit to GoGoWen/incubator-doris that referenced this pull request Aug 27, 2024
@csun5285
[fix] (topn) fix uncleared block in topn_next() (apache#39119) (apach…
9bbe777 
…e#39226)

## Proposed changes

pick from master apache#39119
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@xiaokang xiaokang xiaokang approved these changes

@airborne12 airborne12 airborne12 approved these changes

@zzzxl1993 zzzxl1993 zzzxl1993 approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/2.0.15-merged dev/2.1.6-merged dev/3.0.1-merged reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@csun5285 @doris-robot @xiaokang @airborne12 @zzzxl1993