Skip to content

[cherry-pick](branch-20) fix partition-topn calculate partition input rows have error (#39100) (#39581)#40003

Merged
HappenLee merged 1 commit intoapache:branch-2.0from
zhangstar333:branch-2.0
Aug 31, 2024
Merged

[cherry-pick](branch-20) fix partition-topn calculate partition input rows have error (#39100) (#39581)#40003
HappenLee merged 1 commit intoapache:branch-2.0from
zhangstar333:branch-2.0

Conversation

@zhangstar333
Copy link
Copy Markdown
Contributor

Proposed changes

cherry-pick from master #39100 #39581

@doris-robot
Copy link
Copy Markdown

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@zhangstar333
Copy link
Copy Markdown
Contributor Author

run buildall

@github-actions
Copy link
Copy Markdown
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link
Copy Markdown

TeamCity be ut coverage result:
Function Coverage: 37.86% (8140/21499)
Line Coverage: 29.60% (66995/226332)
Region Coverage: 29.09% (34559/118819)
Branch Coverage: 24.99% (17806/71256)
Coverage Report: http://coverage.selectdb-in.cc/coverage/bd097bea8878c728bc38fa406a7c6bafdafc9941_bd097bea8878c728bc38fa406a7c6bafdafc9941/report/index.html

@doris-robot
Copy link
Copy Markdown

TPC-H: Total hot run time: 50134 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit bd097bea8878c728bc38fa406a7c6bafdafc9941, data reload: false

------ Round 1 ----------------------------------
q1	17821	4409	4580	4409
q2	2070	151	149	149
q3	10440	1953	1884	1884
q4	10386	1306	1304	1304
q5	8551	4027	3909	3909
q6	231	124	121	121
q7	2049	1632	1622	1622
q8	9301	2777	2733	2733
q9	11036	10558	10444	10444
q10	8637	3523	3511	3511
q11	417	242	253	242
q12	462	293	302	293
q13	18361	3949	4156	3949
q14	363	329	323	323
q15	521	463	466	463
q16	678	572	583	572
q17	1131	936	949	936
q18	7424	7057	7003	7003
q19	1711	1548	1530	1530
q20	551	289	290	289
q21	4451	4081	4054	4054
q22	488	394	398	394
Total cold run time: 117080 ms
Total hot run time: 50134 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4306	4268	4277	4268
q2	326	227	226	226
q3	4179	4119	4132	4119
q4	2765	2726	2760	2726
q5	7197	7180	7115	7115
q6	235	119	117	117
q7	3282	2842	2851	2842
q8	4364	4476	4514	4476
q9	17422	16824	16881	16824
q10	4255	4279	4272	4272
q11	748	699	691	691
q12	1013	845	827	827
q13	6823	3737	3730	3730
q14	458	414	423	414
q15	490	469	455	455
q16	722	679	673	673
q17	3789	3869	3819	3819
q18	8806	8738	8732	8732
q19	1701	1698	1677	1677
q20	2407	2102	2091	2091
q21	8399	8355	8451	8355
q22	1054	939	994	939
Total cold run time: 84741 ms
Total hot run time: 79388 ms

@doris-robot
Copy link
Copy Markdown

TPC-DS: Total hot run time: 200744 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit bd097bea8878c728bc38fa406a7c6bafdafc9941, data reload: false

query1	925	401	436	401
query2	6554	2153	1955	1955
query3	6920	213	202	202
query4	21650	17972	17998	17972
query5	19742	6541	6619	6541
query6	382	212	235	212
query7	4867	315	306	306
query8	258	265	256	256
query9	3105	2666	2595	2595
query10	433	291	322	291
query11	11429	10717	10742	10717
query12	118	80	73	73
query13	5600	670	657	657
query14	18568	12933	13485	12933
query15	382	218	231	218
query16	6449	283	260	260
query17	1618	1433	888	888
query18	2291	421	413	413
query19	207	154	143	143
query20	79	80	78	78
query21	192	94	98	94
query22	5294	4971	5108	4971
query23	32329	31695	32090	31695
query24	6874	6582	6529	6529
query25	521	427	432	427
query26	645	166	160	160
query27	1996	293	291	291
query28	6135	2288	2260	2260
query29	2830	2749	2851	2749
query30	249	170	162	162
query31	927	736	714	714
query32	70	60	59	59
query33	410	269	262	262
query34	845	465	477	465
query35	1148	952	911	911
query36	1412	1218	1308	1218
query37	92	58	59	58
query38	3036	2915	2877	2877
query39	1370	1328	1322	1322
query40	304	96	89	89
query41	40	38	35	35
query42	86	80	82	80
query43	628	536	610	536
query44	1164	714	733	714
query45	244	232	226	226
query46	1228	998	984	984
query47	1779	1878	1727	1727
query48	977	696	682	682
query49	640	370	372	370
query50	855	600	616	600
query51	4698	4679	4641	4641
query52	89	91	87	87
query53	444	334	322	322
query54	2649	2450	2478	2450
query55	100	88	81	81
query56	239	224	210	210
query57	1144	1054	1219	1054
query58	217	214	187	187
query59	3298	3207	3559	3207
query60	217	212	232	212
query61	105	96	101	96
query62	818	452	461	452
query63	479	341	341	341
query64	2755	1545	1449	1449
query65	3636	3546	3541	3541
query66	786	384	379	379
query67	17234	16328	15035	15035
query68	10582	686	663	663
query69	566	339	348	339
query70	1977	1539	1365	1365
query71	417	300	325	300
query72	6591	3482	3515	3482
query73	747	332	318	318
query74	6267	5788	5944	5788
query75	5361	3706	3676	3676
query76	6843	1129	1203	1129
query77	1181	253	257	253
query78	12853	11704	11583	11583
query79	8807	655	644	644
query80	1157	420	407	407
query81	497	243	237	237
query82	1020	97	98	97
query83	173	135	135	135
query84	261	70	70	70
query85	900	324	318	318
query86	347	290	296	290
query87	3256	3038	3039	3038
query88	4530	2318	2295	2295
query89	480	308	295	295
query90	1972	211	208	208
query91	161	121	123	121
query92	60	53	56	53
query93	6996	602	560	560
query94	709	208	203	203
query95	1116	1065	1069	1065
query96	642	333	323	323
query97	6694	6335	6404	6335
query98	219	175	171	171
query99	2976	861	863	861
Total cold run time: 321163 ms
Total hot run time: 200744 ms

@doris-robot
Copy link
Copy Markdown

ClickBench: Total hot run time: 30.71 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit bd097bea8878c728bc38fa406a7c6bafdafc9941, data reload: false

query1	0.02	0.03	0.03
query2	0.07	0.03	0.03
query3	0.25	0.04	0.05
query4	1.79	0.07	0.06
query5	0.54	0.52	0.53
query6	1.22	0.61	0.62
query7	0.01	0.02	0.01
query8	0.04	0.03	0.02
query9	0.52	0.51	0.48
query10	0.54	0.55	0.54
query11	0.12	0.08	0.09
query12	0.11	0.09	0.09
query13	0.62	0.62	0.63
query14	0.80	0.82	0.78
query15	0.78	0.76	0.77
query16	0.38	0.36	0.38
query17	1.00	1.02	1.03
query18	0.23	0.24	0.25
query19	1.92	1.77	1.85
query20	0.01	0.02	0.01
query21	15.45	0.55	0.55
query22	2.05	1.76	1.45
query23	17.20	0.85	0.91
query24	4.84	3.14	1.27
query25	0.37	0.09	0.08
query26	0.80	0.16	0.17
query27	0.04	0.03	0.03
query28	5.86	0.80	0.70
query29	12.63	2.22	2.31
query30	0.60	0.52	0.51
query31	2.81	0.38	0.37
query32	3.40	0.50	0.50
query33	3.10	3.07	3.10
query34	15.26	4.82	4.77
query35	4.89	4.86	4.85
query36	1.04	1.01	1.03
query37	0.06	0.05	0.04
query38	0.04	0.02	0.02
query39	0.02	0.01	0.02
query40	0.16	0.14	0.15
query41	0.07	0.02	0.01
query42	0.02	0.01	0.01
query43	0.02	0.02	0.02
Total cold run time: 101.7 s
Total hot run time: 30.71 s

@doris-robot
Copy link
Copy Markdown

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit bd097bea8878c728bc38fa406a7c6bafdafc9941 with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       22.3 seconds inserted 10000000 Rows, about 448K ops/s

Copy link
Copy Markdown
Contributor

@HappenLee HappenLee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@HappenLee HappenLee merged commit 5e82c2b into apache:branch-2.0 Aug 31, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants