Skip to content

[Improvement](schema scan) Use async scanner for schema scanners#38403

Merged
Gabriel39 merged 5 commits intoapache:masterfrom
Gabriel39:imp_0726
Jul 31, 2024
Merged

[Improvement](schema scan) Use async scanner for schema scanners#38403
Gabriel39 merged 5 commits intoapache:masterfrom
Gabriel39:imp_0726

Conversation

@Gabriel39
Copy link
Copy Markdown
Contributor

@Gabriel39 Gabriel39 commented Jul 26, 2024

Proposed changes

Issue Number: close #xxx

@doris-robot
Copy link
Copy Markdown

doris-robot commented Jul 26, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@Gabriel39
Copy link
Copy Markdown
Contributor Author

Gabriel39 commented Jul 26, 2024

run buildall

github-actions[bot]
github-actions bot reviewed Jul 26, 2024
View reviewed changes
Copy link
Copy Markdown
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

Comment thread be/src/exec/schema_scanner/schema_active_queries_scanner.cpp Outdated
}

Status SchemaActiveQueriesScanner::_get_active_queries_block_from_fe() {
void SchemaActiveQueriesScanner::_get_next_block_internal() {
Copy link
Copy Markdown
Contributor

@github-actions github-actions bot Jul 26, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: function '_get_next_block_internal' exceeds recommended size/complexity thresholds [readability-function-size]

void SchemaActiveQueriesScanner::_get_next_block_internal() {
                                 ^
Additional context

be/src/exec/schema_scanner/schema_active_queries_scanner.cpp:53: 97 lines including whitespace and comments (threshold 80)

void SchemaActiveQueriesScanner::_get_next_block_internal() {
                                 ^

@Gabriel39
Copy link
Copy Markdown
Contributor Author

Gabriel39 commented Jul 26, 2024

run buildall

1 similar comment
@Gabriel39
Copy link
Copy Markdown
Contributor Author

Gabriel39 commented Jul 26, 2024

run buildall

@doris-robot
Copy link
Copy Markdown

doris-robot commented Jul 26, 2024

TPC-H: Total hot run time: 39114 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f2e98b509bb7ddbbcab7caa048081b718e5aa134, data reload: false

------ Round 1 ----------------------------------
q1	17627	4458	4278	4278
q2	2006	200	215	200
q3	10421	1217	1048	1048
q4	10140	720	673	673
q5	7583	2723	2680	2680
q6	224	137	141	137
q7	965	595	587	587
q8	9217	1905	1929	1905
q9	8904	6629	6557	6557
q10	8821	3775	3844	3775
q11	447	251	252	251
q12	490	231	222	222
q13	18387	2996	2963	2963
q14	269	231	229	229
q15	531	488	479	479
q16	484	389	374	374
q17	981	646	744	646
q18	8165	7433	7328	7328
q19	4707	932	1008	932
q20	687	340	326	326
q21	4968	3231	3277	3231
q22	365	297	293	293
Total cold run time: 116389 ms
Total hot run time: 39114 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4414	4281	4283	4281
q2	371	265	265	265
q3	2989	2810	2898	2810
q4	1969	1702	1682	1682
q5	5623	5590	5523	5523
q6	229	132	139	132
q7	2173	1798	1839	1798
q8	3308	3431	3402	3402
q9	8798	8770	8796	8770
q10	4128	3865	3851	3851
q11	607	485	489	485
q12	757	614	608	608
q13	15865	3192	3191	3191
q14	323	282	281	281
q15	552	483	500	483
q16	480	440	426	426
q17	1814	1560	1504	1504
q18	8128	8152	7939	7939
q19	3093	1636	1655	1636
q20	2181	1939	1874	1874
q21	8445	4947	4826	4826
q22	601	527	512	512
Total cold run time: 76848 ms
Total hot run time: 56279 ms

@doris-robot
Copy link
Copy Markdown

doris-robot commented Jul 26, 2024

TPC-DS: Total hot run time: 171736 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f2e98b509bb7ddbbcab7caa048081b718e5aa134, data reload: false

query1	911	384	389	384
query2	6438	1925	1838	1838
query3	6654	203	213	203
query4	28500	17481	17161	17161
query5	3665	480	497	480
query6	276	177	174	174
query7	4575	288	289	288
query8	269	204	190	190
query9	8458	2458	2448	2448
query10	448	283	270	270
query11	12081	10008	10246	10008
query12	117	83	82	82
query13	1622	353	360	353
query14	10478	7622	6717	6717
query15	222	162	175	162
query16	7583	476	484	476
query17	1306	546	527	527
query18	1935	279	274	274
query19	183	137	140	137
query20	91	81	82	81
query21	198	99	96	96
query22	4148	3994	3926	3926
query23	34009	33991	33597	33597
query24	10557	2963	2936	2936
query25	604	397	425	397
query26	706	149	148	148
query27	2272	280	274	274
query28	6116	2079	2055	2055
query29	792	430	415	415
query30	259	157	159	157
query31	985	731	771	731
query32	93	56	59	56
query33	688	334	319	319
query34	872	502	489	489
query35	863	762	770	762
query36	1118	970	960	960
query37	150	88	98	88
query38	2998	2855	2856	2855
query39	865	807	815	807
query40	193	114	109	109
query41	44	46	43	43
query42	120	95	94	94
query43	525	473	458	458
query44	1102	721	715	715
query45	208	177	171	171
query46	1071	729	756	729
query47	1828	1735	1765	1735
query48	384	298	293	293
query49	846	409	412	409
query50	790	398	396	396
query51	6776	6609	6698	6609
query52	96	90	84	84
query53	255	177	187	177
query54	842	445	440	440
query55	75	74	71	71
query56	295	286	305	286
query57	1117	1039	1050	1039
query58	278	274	295	274
query59	2835	2628	2689	2628
query60	307	297	289	289
query61	96	97	93	93
query62	774	666	615	615
query63	210	174	175	174
query64	9147	2276	1692	1692
query65	3191	3089	3080	3080
query66	757	342	346	342
query67	15223	14701	15052	14701
query68	4897	548	536	536
query69	578	400	333	333
query70	1136	1026	1145	1026
query71	438	298	287	287
query72	7709	5594	5786	5594
query73	767	329	328	328
query74	6259	5666	5610	5610
query75	3441	2665	2664	2664
query76	3044	908	966	908
query77	605	318	318	318
query78	9734	8977	8920	8920
query79	1597	533	530	530
query80	2348	493	488	488
query81	641	216	217	216
query82	824	139	131	131
query83	311	167	169	167
query84	270	78	78	78
query85	1548	310	303	303
query86	483	307	302	302
query87	3195	3071	3096	3071
query88	4214	2466	2456	2456
query89	399	288	287	287
query90	1964	191	199	191
query91	129	101	101	101
query92	64	47	52	47
query93	2179	523	516	516
query94	1045	281	312	281
query95	352	262	265	262
query96	590	277	277	277
query97	3147	3038	3017	3017
query98	223	202	204	202
query99	1671	1222	1265	1222
Total cold run time: 279292 ms
Total hot run time: 171736 ms

@doris-robot
Copy link
Copy Markdown

doris-robot commented Jul 26, 2024

ClickBench: Total hot run time: 31.28 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit f2e98b509bb7ddbbcab7caa048081b718e5aa134, data reload: false

query1	0.05	0.04	0.03
query2	0.09	0.04	0.04
query3	0.23	0.04	0.04
query4	1.66	0.08	0.07
query5	0.50	0.50	0.48
query6	1.12	0.72	0.73
query7	0.02	0.01	0.01
query8	0.04	0.04	0.04
query9	0.56	0.48	0.48
query10	0.54	0.55	0.54
query11	0.16	0.11	0.11
query12	0.15	0.12	0.13
query13	0.59	0.58	0.58
query14	0.76	0.81	0.77
query15	0.84	0.80	0.83
query16	0.37	0.37	0.37
query17	0.98	1.01	0.98
query18	0.22	0.21	0.22
query19	1.81	1.68	1.74
query20	0.01	0.00	0.00
query21	15.40	0.77	0.66
query22	3.99	6.56	2.68
query23	18.41	1.33	1.22
query24	2.17	0.23	0.22
query25	0.16	0.08	0.09
query26	0.31	0.21	0.20
query27	0.46	0.23	0.23
query28	13.21	1.02	1.00
query29	12.61	3.34	3.29
query30	0.25	0.05	0.06
query31	2.87	0.39	0.38
query32	3.28	0.48	0.48
query33	2.88	2.96	2.92
query34	16.92	4.35	4.39
query35	4.43	4.43	4.42
query36	0.66	0.48	0.46
query37	0.19	0.16	0.15
query38	0.16	0.14	0.15
query39	0.04	0.03	0.03
query40	0.15	0.12	0.13
query41	0.09	0.05	0.05
query42	0.06	0.04	0.05
query43	0.05	0.04	0.04
Total cold run time: 109.45 s
Total hot run time: 31.28 s

github-actions[bot]
github-actions bot reviewed Jul 26, 2024
View reviewed changes
Copy link
Copy Markdown
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

Comment thread be/src/exec/schema_scanner/schema_active_queries_scanner.cpp Outdated
}

Status SchemaActiveQueriesScanner::_get_active_queries_block_from_fe() {
void SchemaActiveQueriesScanner::_get_next_block_internal() {
Copy link
Copy Markdown
Contributor

@github-actions github-actions bot Jul 26, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: function '_get_next_block_internal' exceeds recommended size/complexity thresholds [readability-function-size]

void SchemaActiveQueriesScanner::_get_next_block_internal() {
                                 ^
Additional context

be/src/exec/schema_scanner/schema_active_queries_scanner.cpp:53: 98 lines including whitespace and comments (threshold 80)

void SchemaActiveQueriesScanner::_get_next_block_internal() {
                                 ^

@Gabriel39
Copy link
Copy Markdown
Contributor Author

Gabriel39 commented Jul 26, 2024

run buildall

@doris-robot
Copy link
Copy Markdown

doris-robot commented Jul 26, 2024

TPC-H: Total hot run time: 39404 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 762e81cafb9c10349402fce13acbabe09b1438b0, data reload: false

------ Round 1 ----------------------------------
q1	18031	4690	4429	4429
q2	2577	207	203	203
q3	11124	1303	1071	1071
q4	11230	768	704	704
q5	8322	2778	2727	2727
q6	226	139	136	136
q7	976	606	590	590
q8	9229	1922	1909	1909
q9	8661	6583	6553	6553
q10	8700	3766	3816	3766
q11	496	252	246	246
q12	395	222	218	218
q13	17752	2989	2990	2989
q14	275	234	235	234
q15	522	488	471	471
q16	507	404	384	384
q17	970	640	627	627
q18	8126	7607	7355	7355
q19	5930	1096	1015	1015
q20	655	315	332	315
q21	4947	3171	3304	3171
q22	354	297	291	291
Total cold run time: 120005 ms
Total hot run time: 39404 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4593	4228	4241	4228
q2	394	279	273	273
q3	2988	2798	2737	2737
q4	1885	1633	1634	1633
q5	5294	5345	5306	5306
q6	215	130	128	128
q7	2187	1754	1734	1734
q8	3217	3397	3340	3340
q9	8459	8417	8372	8372
q10	3921	3659	3660	3659
q11	582	490	499	490
q12	793	623	593	593
q13	16508	2977	3016	2977
q14	312	264	276	264
q15	523	470	479	470
q16	471	406	422	406
q17	1758	1471	1455	1455
q18	7622	7544	7493	7493
q19	1646	1577	1622	1577
q20	1977	1793	1781	1781
q21	4770	4707	4606	4606
q22	596	501	495	495
Total cold run time: 70711 ms
Total hot run time: 54017 ms

@doris-robot
Copy link
Copy Markdown

doris-robot commented Jul 26, 2024

TPC-DS: Total hot run time: 172537 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 762e81cafb9c10349402fce13acbabe09b1438b0, data reload: false

query1	900	368	365	365
query2	6457	1933	1813	1813
query3	6654	217	214	214
query4	28087	17332	17507	17332
query5	4170	484	477	477
query6	277	186	188	186
query7	4592	310	293	293
query8	253	199	194	194
query9	8687	2460	2430	2430
query10	442	282	267	267
query11	10874	10180	10058	10058
query12	131	89	86	86
query13	1630	374	369	369
query14	10507	7676	7716	7676
query15	224	165	169	165
query16	7604	478	478	478
query17	1591	550	548	548
query18	1805	283	268	268
query19	189	135	134	134
query20	95	81	83	81
query21	212	95	96	95
query22	4293	3955	3974	3955
query23	33716	33196	33182	33182
query24	12211	2874	2883	2874
query25	670	371	369	369
query26	1837	153	147	147
query27	2908	267	264	264
query28	7252	1996	1986	1986
query29	1109	408	403	403
query30	295	171	151	151
query31	980	742	741	741
query32	97	53	53	53
query33	768	327	328	327
query34	903	473	464	464
query35	865	714	757	714
query36	1091	901	942	901
query37	186	75	79	75
query38	2943	2755	2776	2755
query39	858	808	817	808
query40	272	116	115	115
query41	47	44	45	44
query42	116	93	97	93
query43	523	481	479	479
query44	1179	719	704	704
query45	205	174	175	174
query46	1110	717	732	717
query47	1846	1759	1750	1750
query48	361	293	297	293
query49	1220	418	411	411
query50	806	410	406	406
query51	6845	6664	6647	6647
query52	105	88	86	86
query53	255	178	171	171
query54	1012	443	443	443
query55	73	73	74	73
query56	313	267	275	267
query57	1151	1015	1066	1015
query58	280	269	265	265
query59	3000	2698	2757	2698
query60	315	284	298	284
query61	101	94	123	94
query62	810	637	656	637
query63	212	177	178	177
query64	10572	2220	1720	1720
query65	3160	3124	3102	3102
query66	1348	345	334	334
query67	15417	14900	14659	14659
query68	4574	532	545	532
query69	435	294	310	294
query70	1164	1117	1129	1117
query71	401	271	293	271
query72	6851	5808	5846	5808
query73	749	330	325	325
query74	6152	5630	5640	5630
query75	3495	2687	2661	2661
query76	2830	971	863	863
query77	454	317	313	313
query78	11057	10114	9025	9025
query79	2379	507	514	507
query80	1761	488	476	476
query81	570	220	222	220
query82	593	135	135	135
query83	296	176	173	173
query84	272	80	80	80
query85	1348	315	305	305
query86	487	326	303	303
query87	3271	3123	3054	3054
query88	3966	2465	2450	2450
query89	401	288	278	278
query90	1887	196	194	194
query91	127	99	100	99
query92	65	47	48	47
query93	1733	523	512	512
query94	929	298	298	298
query95	370	274	270	270
query96	596	279	275	275
query97	3190	3033	3029	3029
query98	231	208	198	198
query99	1611	1239	1328	1239
Total cold run time: 285262 ms
Total hot run time: 172537 ms

@doris-robot
Copy link
Copy Markdown

doris-robot commented Jul 26, 2024

ClickBench: Total hot run time: 31.14 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 762e81cafb9c10349402fce13acbabe09b1438b0, data reload: false

query1	0.04	0.03	0.03
query2	0.07	0.04	0.04
query3	0.23	0.05	0.06
query4	1.67	0.08	0.07
query5	0.51	0.47	0.48
query6	1.13	0.73	0.72
query7	0.02	0.01	0.02
query8	0.05	0.04	0.05
query9	0.56	0.50	0.49
query10	0.54	0.54	0.53
query11	0.15	0.11	0.11
query12	0.16	0.12	0.13
query13	0.59	0.59	0.59
query14	0.76	0.77	0.78
query15	0.86	0.82	0.81
query16	0.37	0.37	0.36
query17	1.03	1.04	1.06
query18	0.22	0.21	0.21
query19	1.77	1.69	1.70
query20	0.02	0.01	0.01
query21	15.42	0.78	0.65
query22	4.35	6.57	2.44
query23	18.30	1.46	1.34
query24	2.11	0.22	0.24
query25	0.16	0.08	0.10
query26	0.29	0.21	0.21
query27	0.46	0.24	0.23
query28	13.23	1.03	1.00
query29	12.60	3.29	3.22
query30	0.28	0.05	0.06
query31	2.87	0.40	0.39
query32	3.26	0.48	0.48
query33	2.88	2.91	2.89
query34	17.00	4.37	4.38
query35	4.41	4.35	4.40
query36	0.65	0.47	0.48
query37	0.19	0.15	0.16
query38	0.16	0.15	0.15
query39	0.04	0.04	0.04
query40	0.15	0.12	0.12
query41	0.10	0.05	0.04
query42	0.06	0.05	0.05
query43	0.05	0.04	0.04
Total cold run time: 109.77 s
Total hot run time: 31.14 s

@Gabriel39
Copy link
Copy Markdown
Contributor Author

Gabriel39 commented Jul 29, 2024

run buildall

@Gabriel39
Copy link
Copy Markdown
Contributor Author

Gabriel39 commented Jul 29, 2024

run buildall

@doris-robot
Copy link
Copy Markdown

doris-robot commented Jul 29, 2024

TPC-H: Total hot run time: 39149 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c5673ec2bb45fed296a350cbc5c901859cc37add, data reload: false

------ Round 1 ----------------------------------
q1	17633	4622	4279	4279
q2	2025	201	198	198
q3	10473	1155	1028	1028
q4	10144	703	736	703
q5	7532	2723	2793	2723
q6	222	138	139	138
q7	957	608	591	591
q8	9211	1945	1900	1900
q9	8876	6571	6545	6545
q10	8781	3745	3782	3745
q11	462	247	246	246
q12	489	225	219	219
q13	17896	2985	2992	2985
q14	279	243	242	242
q15	521	481	486	481
q16	501	402	379	379
q17	970	631	709	631
q18	8068	7508	7327	7327
q19	4514	1073	959	959
q20	689	343	337	337
q21	5023	3204	3368	3204
q22	346	294	289	289
Total cold run time: 115612 ms
Total hot run time: 39149 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4480	4301	4287	4287
q2	367	264	270	264
q3	3000	2798	2863	2798
q4	1988	1745	1669	1669
q5	5661	5578	5525	5525
q6	222	142	134	134
q7	2161	1850	1882	1850
q8	3308	3442	3434	3434
q9	8910	8905	9040	8905
q10	4172	3898	3724	3724
q11	589	499	499	499
q12	806	675	636	636
q13	16256	3197	3217	3197
q14	337	300	286	286
q15	528	475	480	475
q16	489	429	423	423
q17	1799	1532	1518	1518
q18	8098	8068	7862	7862
q19	1720	1589	1589	1589
q20	2083	1872	1832	1832
q21	5090	4800	4725	4725
q22	612	495	519	495
Total cold run time: 72676 ms
Total hot run time: 56127 ms

@Gabriel39
Copy link
Copy Markdown
Contributor Author

Gabriel39 commented Jul 29, 2024

run buildall

github-actions[bot]
github-actions bot reviewed Jul 29, 2024
View reviewed changes
Copy link
Copy Markdown
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

Comment thread be/src/exec/schema_scanner/schema_active_queries_scanner.cpp Outdated
@doris-robot
Copy link
Copy Markdown

doris-robot commented Jul 29, 2024

TPC-H: Total hot run time: 39602 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c9a3c36c4bebf6af597c8e89c5da49b87ff25299, data reload: false

------ Round 1 ----------------------------------
q1	17640	4448	4334	4334
q2	2006	198	194	194
q3	10511	1178	1112	1112
q4	10136	738	752	738
q5	7548	2784	2651	2651
q6	224	140	144	140
q7	975	600	602	600
q8	9218	1940	1921	1921
q9	8967	6617	6599	6599
q10	8831	3828	3824	3824
q11	459	241	240	240
q12	484	226	223	223
q13	18861	2990	2976	2976
q14	285	236	242	236
q15	534	485	494	485
q16	524	394	381	381
q17	986	589	735	589
q18	8070	7528	7546	7528
q19	4370	927	1017	927
q20	652	341	327	327
q21	5165	3298	3966	3298
q22	350	279	288	279
Total cold run time: 116796 ms
Total hot run time: 39602 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4541	4285	4268	4268
q2	380	277	274	274
q3	2993	2841	2855	2841
q4	1998	1698	1721	1698
q5	5803	5676	5608	5608
q6	234	139	145	139
q7	2328	1955	1909	1909
q8	3339	3454	3519	3454
q9	8730	8772	8872	8772
q10	4094	3821	3811	3811
q11	593	505	491	491
q12	835	623	634	623
q13	17041	3160	3206	3160
q14	305	292	288	288
q15	540	490	501	490
q16	517	459	437	437
q17	1880	1516	1507	1507
q18	8144	8043	7758	7758
q19	1758	1489	1611	1489
q20	2209	1884	1856	1856
q21	5144	5009	4770	4770
q22	616	531	522	522
Total cold run time: 74022 ms
Total hot run time: 56165 ms

@doris-robot
Copy link
Copy Markdown

doris-robot commented Jul 29, 2024

TPC-DS: Total hot run time: 174031 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c9a3c36c4bebf6af597c8e89c5da49b87ff25299, data reload: false

query1	923	381	384	381
query2	6451	2007	1831	1831
query3	6634	208	224	208
query4	28470	17603	17317	17317
query5	3653	483	493	483
query6	292	171	170	170
query7	4574	289	289	289
query8	243	201	196	196
query9	8492	2478	2448	2448
query10	429	289	261	261
query11	11535	10060	9963	9963
query12	122	87	85	85
query13	1614	382	359	359
query14	9761	7816	7995	7816
query15	224	168	167	167
query16	7122	468	427	427
query17	1154	573	538	538
query18	1693	290	310	290
query19	194	142	144	142
query20	90	84	86	84
query21	204	98	96	96
query22	4258	3923	3927	3923
query23	34500	33783	33533	33533
query24	7248	3004	2881	2881
query25	602	388	407	388
query26	697	160	153	153
query27	2022	279	290	279
query28	4000	2120	2108	2108
query29	772	455	428	428
query30	249	158	160	158
query31	989	770	790	770
query32	102	52	56	52
query33	550	348	352	348
query34	877	501	514	501
query35	858	789	783	783
query36	1059	954	977	954
query37	146	81	87	81
query38	3000	2884	2835	2835
query39	898	880	842	842
query40	204	114	113	113
query41	48	44	45	44
query42	117	100	100	100
query43	512	465	479	465
query44	1099	748	728	728
query45	204	182	175	175
query46	1095	765	754	754
query47	1882	1758	1748	1748
query48	368	298	293	293
query49	817	422	424	422
query50	799	421	412	412
query51	6784	6792	6674	6674
query52	104	93	92	92
query53	269	193	201	193
query54	572	449	479	449
query55	76	73	73	73
query56	290	270	277	270
query57	1109	1073	1053	1053
query58	258	258	268	258
query59	2895	2924	2677	2677
query60	308	278	288	278
query61	96	97	93	93
query62	790	650	634	634
query63	214	186	185	185
query64	9159	2310	1795	1795
query65	3199	3125	3119	3119
query66	796	348	371	348
query67	15267	14877	14829	14829
query68	4533	563	556	556
query69	438	306	326	306
query70	1108	1129	1137	1129
query71	380	287	274	274
query72	7118	5893	5886	5886
query73	752	332	336	332
query74	6180	5728	5608	5608
query75	3361	2693	2657	2657
query76	2143	898	927	898
query77	466	323	313	313
query78	9939	8978	10873	8978
query79	1924	530	524	524
query80	1630	493	488	488
query81	590	218	220	218
query82	468	142	136	136
query83	278	178	173	173
query84	250	81	80	80
query85	1197	323	300	300
query86	478	300	321	300
query87	3291	3038	3075	3038
query88	3934	2468	2492	2468
query89	398	281	283	281
query90	1735	266	203	203
query91	132	105	101	101
query92	66	59	51	51
query93	1208	545	538	538
query94	789	288	290	288
query95	347	267	269	267
query96	594	285	274	274
query97	3221	3056	3064	3056
query98	235	198	198	198
query99	1551	1285	1270	1270
Total cold run time: 266766 ms
Total hot run time: 174031 ms

@doris-robot
Copy link
Copy Markdown

doris-robot commented Jul 29, 2024

ClickBench: Total hot run time: 30.61 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit c9a3c36c4bebf6af597c8e89c5da49b87ff25299, data reload: false

query1	0.04	0.04	0.03
query2	0.08	0.04	0.03
query3	0.22	0.05	0.04
query4	1.68	0.07	0.07
query5	0.51	0.47	0.47
query6	1.13	0.73	0.73
query7	0.02	0.02	0.01
query8	0.05	0.05	0.04
query9	0.55	0.48	0.50
query10	0.54	0.54	0.56
query11	0.16	0.11	0.12
query12	0.14	0.12	0.12
query13	0.60	0.58	0.58
query14	0.76	0.78	0.78
query15	0.86	0.82	0.83
query16	0.37	0.38	0.36
query17	1.03	1.02	1.05
query18	0.22	0.22	0.22
query19	1.79	1.71	1.74
query20	0.01	0.01	0.01
query21	15.39	0.76	0.64
query22	3.80	7.52	1.86
query23	18.25	1.42	1.24
query24	2.13	0.24	0.23
query25	0.16	0.08	0.09
query26	0.31	0.22	0.22
query27	0.44	0.24	0.24
query28	13.25	1.02	1.00
query29	12.63	3.35	3.31
query30	0.25	0.06	0.05
query31	2.87	0.39	0.38
query32	3.28	0.47	0.47
query33	2.85	2.95	2.96
query34	17.00	4.45	4.35
query35	4.43	4.36	4.41
query36	0.66	0.46	0.47
query37	0.18	0.16	0.16
query38	0.15	0.15	0.15
query39	0.05	0.04	0.04
query40	0.15	0.13	0.13
query41	0.09	0.04	0.04
query42	0.06	0.04	0.05
query43	0.04	0.04	0.05
Total cold run time: 109.18 s
Total hot run time: 30.61 s

@Gabriel39
Copy link
Copy Markdown
Contributor Author

Gabriel39 commented Jul 29, 2024

run buildall

@doris-robot
Copy link
Copy Markdown

doris-robot commented Jul 29, 2024

TPC-H: Total hot run time: 41432 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 608062cc155af143b15e7c82f8eb0d52bd951ebf, data reload: false

------ Round 1 ----------------------------------
q1	17705	4067	4032	4032
q2	2020	207	209	207
q3	10444	1296	1344	1296
q4	10173	809	991	809
q5	7660	2903	2963	2903
q6	221	137	134	134
q7	1037	610	604	604
q8	9444	1882	1911	1882
q9	8478	6609	6605	6605
q10	8789	3853	3809	3809
q11	432	252	247	247
q12	416	222	223	222
q13	17751	2933	2926	2926
q14	281	246	247	246
q15	532	481	479	479
q16	522	393	388	388
q17	962	942	919	919
q18	7971	7266	7192	7192
q19	1462	1222	1227	1222
q20	573	328	331	328
q21	5247	4695	4698	4695
q22	359	287	294	287
Total cold run time: 112479 ms
Total hot run time: 41432 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4063	4021	4013	4013
q2	326	220	213	213
q3	2972	2980	3110	2980
q4	1973	1994	1977	1977
q5	5611	5493	5425	5425
q6	222	127	127	127
q7	2158	1772	1807	1772
q8	3318	3386	3346	3346
q9	8670	8608	8777	8608
q10	3949	4044	3961	3961
q11	547	448	453	448
q12	782	631	616	616
q13	16331	3089	3155	3089
q14	307	266	280	266
q15	534	506	489	489
q16	496	416	417	416
q17	1778	1733	1743	1733
q18	8171	7858	7892	7858
q19	1736	1691	1715	1691
q20	2069	1889	1845	1845
q21	5737	5504	5265	5265
q22	549	473	499	473
Total cold run time: 72299 ms
Total hot run time: 56611 ms

@doris-robot
Copy link
Copy Markdown

doris-robot commented Jul 29, 2024

TPC-DS: Total hot run time: 170005 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 608062cc155af143b15e7c82f8eb0d52bd951ebf, data reload: false

query1	914	367	364	364
query2	6461	1724	1694	1694
query3	6651	216	216	216
query4	20197	17574	17349	17349
query5	3644	513	518	513
query6	290	176	200	176
query7	4608	324	293	293
query8	262	200	199	199
query9	8530	2423	2395	2395
query10	430	284	266	266
query11	10479	10164	9898	9898
query12	119	90	88	88
query13	1636	390	373	373
query14	10371	7798	7030	7030
query15	219	161	161	161
query16	6723	438	404	404
query17	936	558	538	538
query18	1898	280	295	280
query19	191	141	140	140
query20	93	85	86	85
query21	199	98	97	97
query22	4250	3982	3994	3982
query23	33966	33907	33661	33661
query24	10292	3110	3118	3110
query25	689	420	413	413
query26	1769	150	153	150
query27	3079	298	284	284
query28	7387	2034	2021	2021
query29	1189	458	429	429
query30	247	158	156	156
query31	957	794	783	783
query32	107	57	56	56
query33	715	328	345	328
query34	927	476	490	476
query35	872	774	751	751
query36	1010	876	883	876
query37	217	79	80	79
query38	2869	2809	2777	2777
query39	872	831	826	826
query40	278	114	111	111
query41	49	44	46	44
query42	139	101	102	101
query43	474	434	426	426
query44	1180	712	748	712
query45	209	176	178	176
query46	1087	825	771	771
query47	1814	1713	1677	1677
query48	373	296	289	289
query49	976	414	459	414
query50	902	436	431	431
query51	6836	6723	6626	6626
query52	107	91	95	91
query53	251	179	179	179
query54	630	454	461	454
query55	72	71	74	71
query56	284	250	257	250
query57	1120	1049	1030	1030
query58	255	267	264	264
query59	2888	2563	2325	2325
query60	297	270	295	270
query61	111	122	99	99
query62	888	667	643	643
query63	215	176	188	176
query64	5740	1955	1889	1889
query65	3171	3087	3087	3087
query66	1313	330	339	330
query67	15312	15298	15052	15052
query68	4434	578	581	578
query69	565	339	296	296
query70	1082	1039	1059	1039
query71	458	276	287	276
query72	7134	2738	2497	2497
query73	777	330	339	330
query74	6006	5638	5654	5638
query75	3470	2714	2737	2714
query76	2591	1330	1411	1330
query77	553	303	306	303
query78	9388	8936	8883	8883
query79	1323	547	526	526
query80	818	515	499	499
query81	544	227	222	222
query82	580	138	132	132
query83	208	168	166	166
query84	268	77	138	77
query85	1099	321	305	305
query86	455	306	308	306
query87	3367	3107	3160	3107
query88	3974	2479	2469	2469
query89	384	295	290	290
query90	1863	191	188	188
query91	130	101	107	101
query92	58	50	49	49
query93	1970	623	627	623
query94	802	306	303	303
query95	382	272	275	272
query96	611	276	282	276
query97	3187	3064	3016	3016
query98	226	209	200	200
query99	1634	1318	1289	1289
Total cold run time: 263577 ms
Total hot run time: 170005 ms

@doris-robot
Copy link
Copy Markdown

doris-robot commented Jul 29, 2024

ClickBench: Total hot run time: 30.16 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 608062cc155af143b15e7c82f8eb0d52bd951ebf, data reload: false

query1	0.05	0.03	0.04
query2	0.07	0.03	0.04
query3	0.23	0.05	0.05
query4	1.69	0.08	0.07
query5	0.49	0.47	0.49
query6	1.14	0.72	0.71
query7	0.02	0.01	0.01
query8	0.05	0.04	0.05
query9	0.57	0.51	0.51
query10	0.57	0.58	0.57
query11	0.15	0.12	0.12
query12	0.15	0.13	0.13
query13	0.61	0.61	0.61
query14	0.78	0.79	0.80
query15	0.90	0.86	0.88
query16	0.35	0.34	0.36
query17	1.03	1.01	1.02
query18	0.22	0.21	0.21
query19	1.82	1.82	1.77
query20	0.01	0.01	0.01
query21	15.39	0.77	0.66
query22	4.50	8.05	1.21
query23	17.80	1.40	1.42
query24	2.25	0.23	0.22
query25	0.19	0.09	0.09
query26	0.32	0.22	0.22
query27	0.46	0.24	0.24
query28	13.16	0.99	0.97
query29	12.58	3.31	3.33
query30	0.26	0.06	0.05
query31	2.88	0.41	0.41
query32	3.27	0.49	0.48
query33	2.99	2.98	2.96
query34	15.47	4.23	4.27
query35	4.32	4.30	4.31
query36	0.69	0.47	0.49
query37	0.18	0.16	0.16
query38	0.16	0.17	0.16
query39	0.05	0.03	0.04
query40	0.16	0.12	0.12
query41	0.10	0.05	0.05
query42	0.06	0.04	0.05
query43	0.05	0.04	0.04
Total cold run time: 108.19 s
Total hot run time: 30.16 s

@Gabriel39
Copy link
Copy Markdown
Contributor Author

Gabriel39 commented Jul 29, 2024

run buildall

@doris-robot
Copy link
Copy Markdown

doris-robot commented Jul 29, 2024

TPC-H: Total hot run time: 42501 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ab5f75953736b074e07eb486c038079bfdd4ab53, data reload: false

------ Round 1 ----------------------------------
q1	17705	4241	4216	4216
q2	2020	218	197	197
q3	10434	1326	1356	1326
q4	10158	818	991	818
q5	7668	3125	3126	3125
q6	239	138	137	137
q7	1092	607	620	607
q8	9454	1869	2002	1869
q9	8602	6841	6835	6835
q10	8695	3907	3890	3890
q11	444	265	252	252
q12	447	230	224	224
q13	17751	2925	2920	2920
q14	270	245	240	240
q15	531	486	494	486
q16	530	388	382	382
q17	1011	949	957	949
q18	8030	7479	7311	7311
q19	1432	1345	1356	1345
q20	588	325	345	325
q21	5348	4809	4760	4760
q22	374	287	290	287
Total cold run time: 112823 ms
Total hot run time: 42501 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4209	4179	4194	4179
q2	352	219	218	218
q3	3000	3160	3201	3160
q4	2092	2052	2054	2052
q5	5653	5563	5451	5451
q6	236	132	129	129
q7	2252	1859	1834	1834
q8	3555	3602	3636	3602
q9	8722	8753	8802	8753
q10	4001	4142	4069	4069
q11	583	472	476	472
q12	791	606	604	604
q13	16304	3136	3142	3136
q14	309	274	263	263
q15	523	489	480	480
q16	467	429	420	420
q17	1962	1879	1864	1864
q18	8224	7775	7785	7775
q19	1890	1913	1925	1913
q20	2075	1886	1839	1839
q21	5858	5513	5403	5403
q22	534	470	465	465
Total cold run time: 73592 ms
Total hot run time: 58081 ms

@doris-robot
Copy link
Copy Markdown

doris-robot commented Jul 29, 2024

TPC-DS: Total hot run time: 169598 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ab5f75953736b074e07eb486c038079bfdd4ab53, data reload: false

query1	907	382	371	371
query2	6441	1717	1717	1717
query3	6673	216	225	216
query4	20128	17605	17262	17262
query5	3600	514	509	509
query6	272	172	159	159
query7	4603	304	287	287
query8	253	194	190	190
query9	8514	2387	2383	2383
query10	416	272	292	272
query11	10530	10135	9857	9857
query12	120	92	88	88
query13	1635	399	374	374
query14	9545	7059	7068	7059
query15	214	164	165	164
query16	6976	443	403	403
query17	959	575	562	562
query18	1923	295	290	290
query19	199	153	144	144
query20	93	85	87	85
query21	208	105	107	105
query22	4285	4010	3879	3879
query23	33624	33525	33503	33503
query24	10460	3160	3060	3060
query25	711	484	383	383
query26	1841	154	152	152
query27	3084	288	295	288
query28	7643	2048	2051	2048
query29	1314	468	447	447
query30	234	151	152	151
query31	968	761	750	750
query32	99	54	58	54
query33	664	319	320	319
query34	929	501	499	499
query35	876	778	763	763
query36	1010	887	895	887
query37	282	81	79	79
query38	2943	2900	2834	2834
query39	923	798	816	798
query40	263	113	110	110
query41	44	42	43	42
query42	120	98	101	98
query43	472	415	409	409
query44	1175	718	726	718
query45	200	179	179	179
query46	1098	798	780	780
query47	1797	1719	1717	1717
query48	363	293	288	288
query49	929	412	403	403
query50	910	426	431	426
query51	6733	6685	6635	6635
query52	107	89	90	89
query53	256	176	183	176
query54	598	452	449	449
query55	76	77	77	77
query56	272	254	248	248
query57	1109	1056	1027	1027
query58	262	258	262	258
query59	2611	2455	2410	2410
query60	294	271	281	271
query61	97	91	94	91
query62	879	655	662	655
query63	211	185	182	182
query64	5650	1889	1892	1889
query65	3139	3072	3113	3072
query66	1301	330	333	330
query67	15168	14807	15011	14807
query68	4420	574	585	574
query69	664	388	333	333
query70	1107	1068	1107	1068
query71	441	313	278	278
query72	7624	2697	2545	2545
query73	791	332	327	327
query74	6054	5656	5702	5656
query75	3391	2731	2729	2729
query76	2794	1320	1403	1320
query77	554	301	308	301
query78	9416	8849	8943	8849
query79	2676	534	527	527
query80	980	516	502	502
query81	582	225	220	220
query82	911	133	131	131
query83	250	164	166	164
query84	266	82	80	80
query85	1492	309	317	309
query86	492	317	282	282
query87	3250	3145	3129	3129
query88	3915	2460	2457	2457
query89	387	290	288	288
query90	1792	199	195	195
query91	126	103	99	99
query92	61	52	48	48
query93	2109	625	616	616
query94	815	299	314	299
query95	385	271	264	264
query96	605	279	281	279
query97	3217	3040	3039	3039
query98	224	199	195	195
query99	1630	1315	1268	1268
Total cold run time: 265654 ms
Total hot run time: 169598 ms

@doris-robot
Copy link
Copy Markdown

doris-robot commented Jul 29, 2024

ClickBench: Total hot run time: 30.03 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit ab5f75953736b074e07eb486c038079bfdd4ab53, data reload: false

query1	0.04	0.04	0.03
query2	0.07	0.04	0.04
query3	0.22	0.05	0.05
query4	1.69	0.06	0.06
query5	0.48	0.47	0.48
query6	1.13	0.72	0.72
query7	0.02	0.01	0.02
query8	0.05	0.04	0.04
query9	0.57	0.50	0.51
query10	0.56	0.56	0.56
query11	0.15	0.11	0.11
query12	0.15	0.12	0.12
query13	0.61	0.60	0.59
query14	0.77	0.79	0.79
query15	0.91	0.86	0.86
query16	0.35	0.35	0.34
query17	0.97	0.99	1.00
query18	0.22	0.21	0.21
query19	1.86	1.80	1.74
query20	0.01	0.01	0.01
query21	15.39	0.76	0.66
query22	3.80	8.22	1.43
query23	18.11	1.30	1.25
query24	2.26	0.23	0.21
query25	0.18	0.08	0.08
query26	0.32	0.20	0.20
query27	0.46	0.23	0.24
query28	13.17	1.00	0.97
query29	12.51	3.30	3.27
query30	0.25	0.06	0.07
query31	2.87	0.40	0.40
query32	3.24	0.48	0.48
query33	2.95	2.95	3.01
query34	15.45	4.27	4.27
query35	4.30	4.26	4.29
query36	0.68	0.50	0.47
query37	0.19	0.16	0.16
query38	0.16	0.15	0.14
query39	0.04	0.04	0.04
query40	0.17	0.14	0.13
query41	0.10	0.05	0.05
query42	0.06	0.04	0.05
query43	0.04	0.04	0.05
Total cold run time: 107.53 s
Total hot run time: 30.03 s

@Gabriel39
Copy link
Copy Markdown
Contributor Author

Gabriel39 commented Jul 30, 2024

run buildall

@doris-robot
Copy link
Copy Markdown

doris-robot commented Jul 30, 2024

TPC-H: Total hot run time: 41443 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 293fee242408579a03ed977e3c4c9aaaffc6ae40, data reload: false

------ Round 1 ----------------------------------
q1	17619	4074	4033	4033
q2	2032	213	203	203
q3	10447	1372	1339	1339
q4	10163	803	884	803
q5	7636	2951	2947	2947
q6	220	137	135	135
q7	1023	612	612	612
q8	9416	1891	1908	1891
q9	8479	6579	6575	6575
q10	8777	3804	3808	3804
q11	431	246	245	245
q12	450	223	221	221
q13	17763	2934	2923	2923
q14	264	240	245	240
q15	523	480	479	479
q16	520	385	377	377
q17	962	910	899	899
q18	7963	7349	7276	7276
q19	1441	1215	1210	1210
q20	546	327	336	327
q21	5341	4644	4631	4631
q22	353	273	279	273
Total cold run time: 112369 ms
Total hot run time: 41443 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4095	4022	4004	4004
q2	324	227	226	226
q3	3001	2976	3088	2976
q4	1962	1972	1960	1960
q5	5556	5451	5406	5406
q6	217	128	127	127
q7	2091	1723	1790	1723
q8	3307	3359	3326	3326
q9	8625	8596	8658	8596
q10	3957	3974	4010	3974
q11	541	440	441	440
q12	716	572	568	568
q13	16534	3119	3101	3101
q14	301	270	269	269
q15	525	491	482	482
q16	452	418	415	415
q17	1722	1721	1717	1717
q18	8091	7697	7643	7643
q19	1707	1689	1698	1689
q20	2011	1891	1814	1814
q21	5762	5379	5282	5282
q22	526	459	446	446
Total cold run time: 72023 ms
Total hot run time: 56184 ms

@doris-robot
Copy link
Copy Markdown

doris-robot commented Jul 30, 2024

TPC-DS: Total hot run time: 169308 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 293fee242408579a03ed977e3c4c9aaaffc6ae40, data reload: false

query1	909	367	381	367
query2	6458	1827	1797	1797
query3	6644	213	224	213
query4	19374	17333	17237	17237
query5	3644	517	507	507
query6	270	169	173	169
query7	4610	292	284	284
query8	259	205	198	198
query9	8504	2378	2359	2359
query10	442	274	270	270
query11	10520	9881	10132	9881
query12	127	89	88	88
query13	1640	371	366	366
query14	9061	6962	8009	6962
query15	199	166	168	166
query16	6886	472	499	472
query17	979	584	554	554
query18	2042	294	284	284
query19	196	154	143	143
query20	90	88	89	88
query21	206	106	105	105
query22	4324	4017	4092	4017
query23	33479	33848	33379	33379
query24	10227	3106	3062	3062
query25	738	443	424	424
query26	1701	152	155	152
query27	2893	292	293	292
query28	7447	2010	1979	1979
query29	1297	467	433	433
query30	240	158	162	158
query31	958	797	779	779
query32	103	60	66	60
query33	687	316	337	316
query34	943	494	506	494
query35	916	792	761	761
query36	1037	898	906	898
query37	220	90	88	88
query38	3056	2776	2748	2748
query39	851	814	813	813
query40	249	113	114	113
query41	45	45	47	45
query42	127	104	105	104
query43	486	431	412	412
query44	1202	735	765	735
query45	211	179	174	174
query46	1067	802	763	763
query47	1782	1701	1701	1701
query48	354	301	286	286
query49	929	429	412	412
query50	903	433	433	433
query51	6831	6687	6705	6687
query52	98	91	89	89
query53	254	178	183	178
query54	636	438	437	437
query55	72	75	75	75
query56	275	258	255	255
query57	1154	1053	1053	1053
query58	287	273	306	273
query59	2644	2667	2516	2516
query60	302	268	265	265
query61	98	98	95	95
query62	893	672	667	667
query63	216	178	182	178
query64	5649	1880	1921	1880
query65	3166	3083	3087	3083
query66	1303	336	340	336
query67	15446	14916	14650	14650
query68	4678	562	554	554
query69	590	307	296	296
query70	1171	1060	1072	1060
query71	463	270	275	270
query72	7916	2721	2497	2497
query73	772	325	324	324
query74	6018	5569	5702	5569
query75	4068	2716	2744	2716
query76	3059	1290	1397	1290
query77	683	304	317	304
query78	9607	8862	8987	8862
query79	2952	554	544	544
query80	1601	501	499	499
query81	555	218	225	218
query82	836	132	135	132
query83	356	172	190	172
query84	283	82	81	81
query85	2015	317	317	317
query86	434	308	289	289
query87	3269	3095	3089	3089
query88	4036	2471	2404	2404
query89	405	280	286	280
query90	2050	193	197	193
query91	132	102	98	98
query92	67	49	49	49
query93	3104	600	599	599
query94	981	287	295	287
query95	381	280	271	271
query96	611	283	278	278
query97	3166	3020	3056	3020
query98	216	203	195	195
query99	1614	1308	1293	1293
Total cold run time: 268945 ms
Total hot run time: 169308 ms

@doris-robot
Copy link
Copy Markdown

doris-robot commented Jul 30, 2024

ClickBench: Total hot run time: 30.42 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 293fee242408579a03ed977e3c4c9aaaffc6ae40, data reload: false

query1	0.04	0.04	0.03
query2	0.07	0.04	0.04
query3	0.23	0.06	0.05
query4	1.67	0.06	0.06
query5	0.48	0.48	0.47
query6	1.16	0.71	0.71
query7	0.02	0.01	0.01
query8	0.05	0.05	0.04
query9	0.57	0.50	0.50
query10	0.55	0.56	0.55
query11	0.16	0.12	0.12
query12	0.15	0.13	0.12
query13	0.62	0.61	0.59
query14	0.77	0.79	0.79
query15	0.91	0.86	0.86
query16	0.35	0.36	0.36
query17	0.96	0.98	1.01
query18	0.22	0.21	0.22
query19	1.82	1.70	1.76
query20	0.01	0.01	0.01
query21	15.39	0.76	0.64
query22	4.16	6.58	1.88
query23	18.04	1.45	1.30
query24	2.24	0.22	0.22
query25	0.18	0.08	0.09
query26	0.32	0.21	0.21
query27	0.46	0.23	0.23
query28	13.15	1.00	0.97
query29	12.55	3.31	3.30
query30	0.25	0.06	0.05
query31	2.86	0.41	0.40
query32	3.24	0.49	0.48
query33	2.94	2.92	2.86
query34	15.39	4.25	4.26
query35	4.31	4.30	4.29
query36	0.68	0.47	0.48
query37	0.19	0.17	0.16
query38	0.16	0.15	0.14
query39	0.04	0.03	0.04
query40	0.16	0.14	0.14
query41	0.09	0.04	0.04
query42	0.05	0.05	0.05
query43	0.05	0.04	0.03
Total cold run time: 107.71 s
Total hot run time: 30.42 s

@Gabriel39
Copy link
Copy Markdown
Contributor Author

Gabriel39 commented Jul 31, 2024

run buildall

@doris-robot
Copy link
Copy Markdown

doris-robot commented Jul 31, 2024

TPC-H: Total hot run time: 41837 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a311b112c7e311f6f22ef6d1a89d709a1619308a, data reload: false

------ Round 1 ----------------------------------
q1	17666	4140	4183	4140
q2	2012	199	199	199
q3	10469	1332	1368	1332
q4	10158	840	926	840
q5	7584	2981	2987	2981
q6	219	137	139	137
q7	1039	622	613	613
q8	9437	1875	1979	1875
q9	8590	6642	6650	6642
q10	8751	3867	3863	3863
q11	433	252	254	252
q12	426	234	237	234
q13	17766	2932	2962	2932
q14	273	244	244	244
q15	521	482	494	482
q16	527	399	390	390
q17	980	904	899	899
q18	7995	7336	7201	7201
q19	1980	1229	1224	1224
q20	567	340	327	327
q21	5354	4880	4750	4750
q22	355	280	299	280
Total cold run time: 113102 ms
Total hot run time: 41837 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4130	4047	4011	4011
q2	335	224	222	222
q3	3020	3038	3166	3038
q4	2025	2015	1946	1946
q5	5640	5453	5499	5453
q6	219	130	131	130
q7	2163	1770	1795	1770
q8	3316	3416	3387	3387
q9	8681	8729	8755	8729
q10	4018	4069	3984	3984
q11	545	460	458	458
q12	783	604	591	591
q13	16329	3174	3150	3150
q14	299	274	268	268
q15	547	508	491	491
q16	486	419	422	419
q17	1754	1728	1733	1728
q18	8155	7760	7667	7667
q19	1743	1743	1726	1726
q20	2045	1849	1849	1849
q21	5846	5450	5357	5357
q22	526	438	475	438
Total cold run time: 72605 ms
Total hot run time: 56812 ms

@doris-robot
Copy link
Copy Markdown

doris-robot commented Jul 31, 2024

TPC-DS: Total hot run time: 172158 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a311b112c7e311f6f22ef6d1a89d709a1619308a, data reload: false

query1	914	369	377	369
query2	6420	1729	1680	1680
query3	6657	211	221	211
query4	20285	17245	17142	17142
query5	3695	536	529	529
query6	271	188	177	177
query7	4590	299	293	293
query8	252	191	189	189
query9	8510	2395	2375	2375
query10	435	307	281	281
query11	10453	10079	10085	10079
query12	123	94	93	93
query13	1629	380	384	380
query14	10388	8648	8624	8624
query15	202	167	173	167
query16	6790	437	444	437
query17	967	595	580	580
query18	1931	301	295	295
query19	201	151	153	151
query20	96	88	87	87
query21	207	97	101	97
query22	4297	4131	4158	4131
query23	34059	33546	33498	33498
query24	10342	3198	3069	3069
query25	692	436	414	414
query26	1736	153	154	153
query27	3014	281	295	281
query28	7307	2042	2036	2036
query29	1262	443	439	439
query30	239	159	155	155
query31	968	769	794	769
query32	105	61	66	61
query33	701	354	366	354
query34	934	507	531	507
query35	879	787	772	772
query36	1034	896	850	850
query37	201	81	81	81
query38	2947	2834	2757	2757
query39	887	829	824	824
query40	257	115	116	115
query41	44	43	45	43
query42	121	100	99	99
query43	456	409	430	409
query44	1234	743	741	741
query45	212	180	181	180
query46	1105	831	807	807
query47	1789	1724	1691	1691
query48	375	298	293	293
query49	934	438	431	431
query50	923	446	453	446
query51	6931	6783	6670	6670
query52	103	91	90	90
query53	261	182	180	180
query54	662	460	464	460
query55	76	77	74	74
query56	284	262	267	262
query57	1152	1030	1053	1030
query58	270	274	314	274
query59	2553	2295	2398	2295
query60	315	282	298	282
query61	100	97	94	94
query62	897	657	657	657
query63	218	187	193	187
query64	5584	1945	1908	1908
query65	3186	3111	3106	3106
query66	1315	340	326	326
query67	15219	14937	14794	14794
query68	6019	569	580	569
query69	670	393	319	319
query70	1133	1075	1072	1072
query71	469	279	282	279
query72	7619	2710	2531	2531
query73	927	333	331	331
query74	5939	5686	5680	5680
query75	3975	2743	2761	2743
query76	4105	1418	1440	1418
query77	690	324	319	319
query78	9440	9040	8915	8915
query79	1337	551	555	551
query80	791	529	529	529
query81	533	226	232	226
query82	597	134	136	134
query83	196	181	173	173
query84	273	80	82	80
query85	1250	321	365	321
query86	472	290	291	290
query87	3316	3149	3157	3149
query88	3654	2421	2429	2421
query89	376	294	290	290
query90	1840	199	195	195
query91	130	105	105	105
query92	64	51	51	51
query93	1772	622	637	622
query94	725	297	317	297
query95	395	269	275	269
query96	611	282	278	278
query97	3364	3127	3143	3127
query98	231	205	208	205
query99	1633	1338	1344	1338
Total cold run time: 267742 ms
Total hot run time: 172158 ms

@doris-robot
Copy link
Copy Markdown

doris-robot commented Jul 31, 2024

ClickBench: Total hot run time: 30.14 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit a311b112c7e311f6f22ef6d1a89d709a1619308a, data reload: false

query1	0.04	0.03	0.04
query2	0.07	0.04	0.04
query3	0.22	0.05	0.05
query4	1.68	0.07	0.07
query5	0.48	0.49	0.49
query6	1.14	0.72	0.72
query7	0.02	0.02	0.02
query8	0.05	0.04	0.04
query9	0.56	0.51	0.52
query10	0.56	0.56	0.58
query11	0.15	0.12	0.11
query12	0.15	0.13	0.13
query13	0.63	0.60	0.59
query14	0.77	0.80	0.79
query15	0.90	0.86	0.87
query16	0.36	0.36	0.36
query17	0.99	1.00	0.98
query18	0.24	0.21	0.21
query19	1.88	1.74	1.78
query20	0.01	0.01	0.01
query21	15.42	0.81	0.67
query22	4.05	7.86	1.29
query23	17.90	1.34	1.34
query24	2.28	0.23	0.22
query25	0.18	0.08	0.09
query26	0.32	0.22	0.21
query27	0.45	0.24	0.23
query28	13.20	1.01	0.96
query29	12.55	3.34	3.34
query30	0.25	0.05	0.07
query31	2.87	0.41	0.41
query32	3.22	0.49	0.49
query33	2.94	3.00	2.91
query34	15.46	4.26	4.23
query35	4.32	4.29	4.29
query36	0.67	0.50	0.49
query37	0.18	0.18	0.16
query38	0.16	0.16	0.15
query39	0.04	0.04	0.03
query40	0.16	0.13	0.13
query41	0.10	0.05	0.05
query42	0.06	0.05	0.06
query43	0.06	0.05	0.05
Total cold run time: 107.74 s
Total hot run time: 30.14 s

yiguolei
yiguolei approved these changes Jul 31, 2024
View reviewed changes
Copy link
Copy Markdown
Contributor

@yiguolei yiguolei left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Jul 31, 2024

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jul 31, 2024
@Gabriel39 Gabriel39 merged commit 4c22f2c into apache:master Jul 31, 2024
@Gabriel39 Gabriel39 mentioned this pull request Jul 31, 2024
Merged
@wangbo wangbo mentioned this pull request Jul 31, 2024
Closed
hello-stephen pushed a commit that referenced this pull request Jul 31, 2024
@Gabriel39
[fix](compile) Resolve compile failure (#38593)
4b4985d 
## Proposed changes

Resolve conflicts by merging #38436 and #38403

<!--Describe your changes.-->
Gabriel39 added a commit to Gabriel39/incubator-doris that referenced this pull request Aug 1, 2024
@Gabriel39
[Improvement](schema scan) Use async scanner for schema scanners (apa…
6e1f5dc 
…che#38403)
dataroaring pushed a commit that referenced this pull request Aug 2, 2024
@Gabriel39 @dataroaring
[Improvement](schema scan) Use async scanner for schema scanners (#38403
e679879 
)
feiniaofeiafei pushed a commit to feiniaofeiafei/doris that referenced this pull request Aug 9, 2024
@Gabriel39 @feiniaofeiafei
[Improvement](schema scan) Use async scanner for schema scanners (apa…
706c680 
…che#38403)
feiniaofeiafei pushed a commit to feiniaofeiafei/doris that referenced this pull request Aug 9, 2024
@Gabriel39 @feiniaofeiafei
[fix](compile) Resolve compile failure (apache#38593)
7f40d07 
## Proposed changes

Resolve conflicts by merging apache#38436 and apache#38403

<!--Describe your changes.-->
dataroaring pushed a commit that referenced this pull request Aug 12, 2024
@Gabriel39 @dataroaring
[fix](compile) Resolve compile failure (#38593)
bc3fe70 
## Proposed changes

Resolve conflicts by merging #38436 and #38403

<!--Describe your changes.-->
dataroaring pushed a commit that referenced this pull request Aug 16, 2024
@Gabriel39 @dataroaring
[fix](compile) Resolve compile failure (#38593)
01d7099 
## Proposed changes

Resolve conflicts by merging #38436 and #38403

<!--Describe your changes.-->
@gavinchou gavinchou mentioned this pull request Aug 19, 2024
Closed
@yiguolei yiguolei mentioned this pull request Sep 5, 2024
Closed
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@github-actions github-actions[bot] github-actions[bot] left review comments

@BiteTheDDDDt BiteTheDDDDt BiteTheDDDDt approved these changes

@yiguolei yiguolei yiguolei approved these changes

@zhangstar333 zhangstar333 zhangstar333 left review comments

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/2.1.6-merged dev/3.0.1-merged p0_b reviewed usercase Important user case type label

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@Gabriel39 @doris-robot @BiteTheDDDDt @yiguolei @zhangstar333 @dataroaring