Skip to content

[fix](parquet) disable parquet page index by default#38691

Merged
yiguolei merged 1 commit intoapache:masterfrom
morningman:disable_page_index
Aug 8, 2024
Merged

[fix](parquet) disable parquet page index by default#38691
yiguolei merged 1 commit intoapache:masterfrom
morningman:disable_page_index

Conversation

@morningman
Copy link
Copy Markdown
Contributor

@morningman morningman commented Aug 1, 2024

there are some issue with page index parsing.
disable it temporarily

@doris-robot
Copy link
Copy Markdown

doris-robot commented Aug 1, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@github-actions github-actions bot added the doing label Aug 1, 2024
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Aug 1, 2024

PR approved by anyone and no changes requested.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Aug 1, 2024

clang-tidy review says "All clean, LGTM! 👍"

@morningman
Copy link
Copy Markdown
Contributor Author

morningman commented Aug 1, 2024

run buildall

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Aug 1, 2024

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link
Copy Markdown

doris-robot commented Aug 1, 2024

TPC-H: Total hot run time: 41519 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c5df8e340903666783fd70e33988f90ed3109454, data reload: false

------ Round 1 ----------------------------------
q1	15666	4134	4066	4066
q2	426	210	211	210
q3	2726	1177	1324	1177
q4	5841	824	1044	824
q5	4335	3013	2976	2976
q6	224	141	140	140
q7	1026	616	601	601
q8	5316	1845	1955	1845
q9	6609	6566	6644	6566
q10	6111	3869	3815	3815
q11	389	246	249	246
q12	405	233	224	224
q13	16775	2940	2951	2940
q14	273	240	246	240
q15	532	481	498	481
q16	498	404	392	392
q17	947	932	904	904
q18	8137	7398	7335	7335
q19	1375	1214	1223	1214
q20	543	324	351	324
q21	5289	4723	4773	4723
q22	354	276	299	276
Total cold run time: 83797 ms
Total hot run time: 41519 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4026	4019	3993	3993
q2	326	226	222	222
q3	3025	2990	3027	2990
q4	1887	1853	1845	1845
q5	5297	5259	5220	5220
q6	217	132	131	131
q7	2070	1721	1681	1681
q8	3203	3245	3220	3220
q9	8338	8281	8292	8281
q10	3736	3845	3865	3845
q11	552	456	454	454
q12	722	543	562	543
q13	12690	2907	2932	2907
q14	297	256	255	255
q15	518	481	479	479
q16	443	395	404	395
q17	1730	1725	1701	1701
q18	7773	7307	7245	7245
q19	1656	1654	1674	1654
q20	1988	1779	1769	1769
q21	5343	5200	5221	5200
q22	532	465	455	455
Total cold run time: 66369 ms
Total hot run time: 54485 ms

@doris-robot
Copy link
Copy Markdown

doris-robot commented Aug 1, 2024

TPC-DS: Total hot run time: 168677 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c5df8e340903666783fd70e33988f90ed3109454, data reload: false

query1	913	376	371	371
query2	6485	1748	1706	1706
query3	6678	223	229	223
query4	18990	17493	17587	17493
query5	4277	529	523	523
query6	283	175	170	170
query7	4592	299	286	286
query8	244	184	197	184
query9	8529	2393	2370	2370
query10	439	283	264	264
query11	10489	10068	10090	10068
query12	142	94	93	93
query13	1634	392	378	378
query14	9152	6366	7536	6366
query15	212	162	165	162
query16	7090	485	452	452
query17	953	570	566	566
query18	1931	283	286	283
query19	206	151	154	151
query20	102	88	88	88
query21	211	104	99	99
query22	4319	3957	4018	3957
query23	33947	33274	32984	32984
query24	10309	3075	3102	3075
query25	749	379	374	374
query26	1766	148	151	148
query27	2953	271	280	271
query28	6911	1974	1978	1974
query29	1359	413	413	413
query30	240	153	160	153
query31	929	761	770	761
query32	97	55	58	55
query33	696	328	312	312
query34	895	480	482	480
query35	857	728	727	727
query36	1015	900	850	850
query37	296	87	84	84
query38	2834	2729	2755	2729
query39	859	805	828	805
query40	277	113	112	112
query41	49	46	44	44
query42	122	98	103	98
query43	465	419	410	410
query44	1181	715	723	715
query45	206	179	181	179
query46	1084	811	771	771
query47	1811	1698	1714	1698
query48	360	290	288	288
query49	1080	453	437	437
query50	885	429	423	423
query51	6840	6747	6725	6725
query52	110	91	88	88
query53	251	181	182	181
query54	648	467	463	463
query55	79	75	79	75
query56	279	252	268	252
query57	1147	1030	1050	1030
query58	295	268	273	268
query59	2496	2471	2409	2409
query60	304	289	276	276
query61	98	97	95	95
query62	918	680	692	680
query63	220	185	182	182
query64	5836	1963	1866	1866
query65	3179	3116	3144	3116
query66	1437	338	353	338
query67	15176	14904	14886	14886
query68	4337	566	567	566
query69	453	303	296	296
query70	1112	1070	1094	1070
query71	428	283	282	282
query72	7077	2670	2535	2535
query73	750	329	334	329
query74	6007	5636	5738	5636
query75	3391	2742	2769	2742
query76	2622	1180	1262	1180
query77	440	325	317	317
query78	9470	8928	8830	8830
query79	2281	538	534	534
query80	1173	518	508	508
query81	573	233	229	229
query82	1067	133	129	129
query83	241	168	173	168
query84	262	81	79	79
query85	1386	309	357	309
query86	454	315	282	282
query87	3268	3130	3066	3066
query88	3152	2420	2397	2397
query89	377	302	288	288
query90	1833	192	194	192
query91	127	101	101	101
query92	64	55	54	54
query93	1985	610	613	610
query94	962	299	298	298
query95	382	273	267	267
query96	602	275	284	275
query97	3209	3046	3059	3046
query98	222	209	201	201
query99	1660	1308	1311	1308
Total cold run time: 262783 ms
Total hot run time: 168677 ms

@doris-robot
Copy link
Copy Markdown

doris-robot commented Aug 1, 2024

ClickBench: Total hot run time: 30.39 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit c5df8e340903666783fd70e33988f90ed3109454, data reload: false

query1	0.04	0.04	0.03
query2	0.07	0.04	0.04
query3	0.22	0.05	0.04
query4	1.68	0.08	0.08
query5	0.49	0.49	0.49
query6	1.13	0.72	0.72
query7	0.02	0.01	0.01
query8	0.05	0.04	0.04
query9	0.58	0.51	0.51
query10	0.57	0.57	0.56
query11	0.16	0.12	0.12
query12	0.15	0.13	0.13
query13	0.62	0.59	0.60
query14	0.78	0.80	0.79
query15	0.89	0.88	0.87
query16	0.36	0.36	0.36
query17	1.02	1.03	1.00
query18	0.23	0.21	0.21
query19	1.86	1.82	1.81
query20	0.01	0.01	0.00
query21	15.40	0.74	0.66
query22	3.80	8.25	1.39
query23	17.96	1.48	1.38
query24	2.29	0.22	0.21
query25	0.19	0.09	0.08
query26	0.32	0.21	0.21
query27	0.45	0.23	0.23
query28	13.17	0.99	0.97
query29	12.65	3.35	3.31
query30	0.26	0.06	0.06
query31	2.88	0.41	0.41
query32	3.22	0.49	0.49
query33	2.93	3.03	2.99
query34	15.42	4.28	4.23
query35	4.30	4.29	4.31
query36	0.67	0.48	0.48
query37	0.19	0.16	0.16
query38	0.17	0.15	0.16
query39	0.05	0.04	0.03
query40	0.17	0.13	0.14
query41	0.10	0.05	0.05
query42	0.06	0.05	0.05
query43	0.04	0.03	0.04
Total cold run time: 107.62 s
Total hot run time: 30.39 s

@morningman
Copy link
Copy Markdown
Contributor Author

morningman commented Aug 2, 2024

run buildall

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Aug 2, 2024

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link
Copy Markdown

doris-robot commented Aug 2, 2024

TPC-H: Total hot run time: 41260 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 175e12a6be7143e973c3dfe165f920d148abfd13, data reload: false

------ Round 1 ----------------------------------
q1	17634	4067	4038	4038
q2	2019	195	189	189
q3	10500	1264	1357	1264
q4	10159	815	953	815
q5	7645	2888	2941	2888
q6	221	140	139	139
q7	1039	611	597	597
q8	9440	1921	1902	1902
q9	8544	6576	6613	6576
q10	8714	3817	3803	3803
q11	425	245	242	242
q12	421	221	220	220
q13	17765	2961	2932	2932
q14	264	236	244	236
q15	522	479	496	479
q16	528	385	376	376
q17	952	912	886	886
q18	8109	7207	7310	7207
q19	1396	1219	1207	1207
q20	565	321	328	321
q21	5377	4671	4762	4671
q22	340	287	272	272
Total cold run time: 112579 ms
Total hot run time: 41260 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4057	4000	4050	4000
q2	325	221	216	216
q3	2967	2988	3089	2988
q4	1955	1966	1962	1962
q5	5511	5447	5436	5436
q6	220	134	128	128
q7	2261	1754	1813	1754
q8	3284	3357	3314	3314
q9	8644	8641	8672	8641
q10	3966	3983	3949	3949
q11	554	445	467	445
q12	749	605	600	600
q13	16068	3134	3109	3109
q14	314	272	274	272
q15	530	476	501	476
q16	453	413	411	411
q17	1767	1714	1727	1714
q18	8204	7850	7567	7567
q19	1737	1704	1695	1695
q20	2045	1863	1872	1863
q21	5646	5444	5406	5406
q22	505	463	465	463
Total cold run time: 71762 ms
Total hot run time: 56409 ms

@doris-robot
Copy link
Copy Markdown

doris-robot commented Aug 2, 2024

TPC-DS: Total hot run time: 168957 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 175e12a6be7143e973c3dfe165f920d148abfd13, data reload: false

query1	915	377	365	365
query2	6458	1683	1696	1683
query3	6661	215	222	215
query4	20357	17398	17245	17245
query5	3653	507	513	507
query6	270	167	170	167
query7	4597	302	292	292
query8	252	189	208	189
query9	8522	2344	2360	2344
query10	436	278	267	267
query11	10592	10099	10109	10099
query12	127	89	87	87
query13	1633	392	378	378
query14	8569	6694	6250	6250
query15	222	161	165	161
query16	6698	449	444	444
query17	944	578	540	540
query18	1838	277	282	277
query19	190	140	142	140
query20	89	82	84	82
query21	202	97	97	97
query22	4275	4203	4109	4109
query23	33766	33662	33419	33419
query24	10303	3092	3124	3092
query25	703	399	398	398
query26	1848	150	150	150
query27	3005	288	293	288
query28	7585	2001	2001	2001
query29	1228	443	429	429
query30	238	155	151	151
query31	980	785	765	765
query32	104	52	54	52
query33	675	338	326	326
query34	925	496	521	496
query35	898	782	767	767
query36	1020	906	902	902
query37	287	86	83	83
query38	2970	2861	2803	2803
query39	883	804	817	804
query40	256	113	111	111
query41	47	43	44	43
query42	114	101	96	96
query43	455	411	418	411
query44	1177	734	734	734
query45	207	179	180	179
query46	1086	832	793	793
query47	1834	1729	1736	1729
query48	366	297	292	292
query49	934	411	418	411
query50	889	436	426	426
query51	6778	6632	6677	6632
query52	101	93	94	93
query53	253	180	181	180
query54	604	457	455	455
query55	78	73	75	73
query56	289	248	251	248
query57	1129	1022	1031	1022
query58	261	261	267	261
query59	2785	2481	2455	2455
query60	297	272	295	272
query61	96	89	92	89
query62	912	653	681	653
query63	220	175	181	175
query64	5617	1893	1899	1893
query65	3161	3091	3123	3091
query66	1303	342	366	342
query67	15365	15008	14782	14782
query68	4462	568	568	568
query69	693	353	301	301
query70	1109	1016	1080	1016
query71	456	286	276	276
query72	7641	2634	2510	2510
query73	779	333	327	327
query74	6052	5618	5641	5618
query75	3541	2726	2691	2691
query76	3240	1183	1286	1183
query77	548	307	314	307
query78	9353	8932	8876	8876
query79	1080	531	535	531
query80	909	500	501	500
query81	559	229	222	222
query82	440	130	127	127
query83	249	176	166	166
query84	261	77	86	77
query85	1134	309	343	309
query86	376	292	284	284
query87	3320	3080	3108	3080
query88	3025	2561	2539	2539
query89	383	298	289	289
query90	1728	193	197	193
query91	136	113	115	113
query92	58	52	53	52
query93	1332	611	611	611
query94	787	309	298	298
query95	393	271	270	270
query96	620	296	294	294
query97	3176	3083	3067	3067
query98	217	215	208	208
query99	1558	1249	1293	1249
Total cold run time: 261147 ms
Total hot run time: 168957 ms

@doris-robot
Copy link
Copy Markdown

doris-robot commented Aug 2, 2024

ClickBench: Total hot run time: 30.54 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 175e12a6be7143e973c3dfe165f920d148abfd13, data reload: false

query1	0.05	0.04	0.04
query2	0.08	0.04	0.03
query3	0.23	0.05	0.05
query4	1.71	0.06	0.06
query5	0.47	0.48	0.50
query6	1.13	0.72	0.71
query7	0.02	0.01	0.01
query8	0.05	0.04	0.04
query9	0.60	0.50	0.51
query10	0.55	0.57	0.57
query11	0.15	0.12	0.11
query12	0.14	0.12	0.13
query13	0.63	0.60	0.61
query14	0.76	0.80	0.81
query15	0.91	0.86	0.85
query16	0.35	0.36	0.34
query17	1.02	0.98	1.00
query18	0.22	0.21	0.20
query19	1.84	1.72	1.80
query20	0.02	0.01	0.01
query21	15.40	0.78	0.66
query22	4.10	6.81	1.94
query23	17.91	1.28	1.31
query24	2.25	0.22	0.22
query25	0.19	0.08	0.07
query26	0.32	0.21	0.20
query27	0.46	0.22	0.23
query28	13.21	1.00	0.97
query29	12.59	3.32	3.25
query30	0.25	0.06	0.05
query31	2.89	0.41	0.40
query32	3.22	0.48	0.47
query33	2.91	2.99	2.98
query34	15.44	4.28	4.25
query35	4.26	4.29	4.30
query36	0.68	0.47	0.48
query37	0.18	0.17	0.16
query38	0.16	0.15	0.15
query39	0.04	0.03	0.03
query40	0.16	0.13	0.13
query41	0.11	0.05	0.04
query42	0.06	0.05	0.05
query43	0.05	0.04	0.04
Total cold run time: 107.77 s
Total hot run time: 30.54 s

@morningman
Copy link
Copy Markdown
Contributor Author

morningman commented Aug 2, 2024

run buildall

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Aug 2, 2024

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link
Copy Markdown

doris-robot commented Aug 2, 2024

TPC-H: Total hot run time: 41293 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4b441dfa6f89fd3b792f14cd6542789f3e119d87, data reload: false

------ Round 1 ----------------------------------
q1	17607	4069	3997	3997
q2	2012	198	196	196
q3	10455	1313	1349	1313
q4	10158	781	895	781
q5	7647	2930	2959	2930
q6	218	133	135	133
q7	1036	627	606	606
q8	9423	1879	1929	1879
q9	8439	6580	6614	6580
q10	8759	3821	3845	3821
q11	435	243	253	243
q12	421	225	218	218
q13	17767	2936	2907	2907
q14	269	243	245	243
q15	525	487	504	487
q16	531	386	382	382
q17	951	903	918	903
q18	7909	7217	7177	7177
q19	1396	1216	1208	1208
q20	555	321	342	321
q21	5221	4693	4829	4693
q22	355	275	278	275
Total cold run time: 112089 ms
Total hot run time: 41293 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4071	3964	4011	3964
q2	326	226	220	220
q3	2979	2976	3121	2976
q4	1977	1973	1998	1973
q5	5495	5584	5452	5452
q6	235	140	135	135
q7	2112	1760	1841	1760
q8	3309	3359	3302	3302
q9	8635	8622	8755	8622
q10	3954	4033	3943	3943
q11	552	458	471	458
q12	768	631	596	596
q13	16422	3146	3108	3108
q14	326	268	284	268
q15	535	505	485	485
q16	466	412	411	411
q17	1744	1719	1721	1719
q18	8138	7694	7732	7694
q19	1783	1705	1706	1705
q20	2050	1878	1835	1835
q21	5651	5480	5284	5284
q22	505	473	484	473
Total cold run time: 72033 ms
Total hot run time: 56383 ms

@doris-robot
Copy link
Copy Markdown

doris-robot commented Aug 2, 2024

TPC-DS: Total hot run time: 169487 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4b441dfa6f89fd3b792f14cd6542789f3e119d87, data reload: false

query1	924	373	362	362
query2	6456	1709	1670	1670
query3	6663	212	225	212
query4	20267	17379	17454	17379
query5	3640	511	499	499
query6	265	171	172	171
query7	4609	304	286	286
query8	255	197	187	187
query9	8521	2349	2340	2340
query10	440	280	275	275
query11	10507	10203	10086	10086
query12	129	89	88	88
query13	1636	387	380	380
query14	8460	7549	6898	6898
query15	202	164	165	164
query16	6841	465	428	428
query17	959	575	562	562
query18	1898	291	292	291
query19	190	147	144	144
query20	92	87	95	87
query21	202	101	97	97
query22	4291	4021	3951	3951
query23	33514	33639	33845	33639
query24	9518	3091	3048	3048
query25	687	425	415	415
query26	1533	148	150	148
query27	3084	294	285	285
query28	7530	2004	1990	1990
query29	1059	426	435	426
query30	235	157	178	157
query31	949	779	778	778
query32	102	55	56	55
query33	674	355	354	354
query34	951	503	508	503
query35	865	753	757	753
query36	1061	866	868	866
query37	161	83	78	78
query38	2870	2797	2771	2771
query39	872	820	819	819
query40	248	115	112	112
query41	48	50	44	44
query42	113	100	100	100
query43	463	427	423	423
query44	1164	737	745	737
query45	210	181	179	179
query46	1114	830	807	807
query47	1816	1751	1713	1713
query48	367	294	294	294
query49	871	417	422	417
query50	889	437	437	437
query51	6853	6664	6563	6563
query52	104	86	89	86
query53	249	185	180	180
query54	631	457	452	452
query55	76	76	72	72
query56	287	255	259	255
query57	1127	1004	1064	1004
query58	263	270	331	270
query59	2477	2434	2205	2205
query60	296	274	273	273
query61	100	96	97	96
query62	880	659	686	659
query63	212	186	181	181
query64	4915	1907	1875	1875
query65	3147	3109	3089	3089
query66	982	331	328	328
query67	14997	14869	14910	14869
query68	4384	568	571	568
query69	522	320	294	294
query70	1077	1073	1071	1071
query71	413	280	279	279
query72	7751	2691	2542	2542
query73	768	329	330	329
query74	6115	5631	5594	5594
query75	3656	2734	2750	2734
query76	2594	1160	1277	1160
query77	531	310	304	304
query78	9363	9007	8833	8833
query79	1946	531	530	530
query80	1267	503	510	503
query81	570	236	225	225
query82	1062	131	129	129
query83	256	178	173	173
query84	267	81	82	81
query85	1261	330	307	307
query86	430	291	303	291
query87	3230	3077	3129	3077
query88	2993	2499	2554	2499
query89	397	285	296	285
query90	1871	192	189	189
query91	126	103	104	103
query92	60	50	51	50
query93	1798	618	614	614
query94	957	291	295	291
query95	373	265	262	262
query96	616	288	282	282
query97	3297	3063	3046	3046
query98	218	199	193	193
query99	1619	1287	1307	1287
Total cold run time: 259799 ms
Total hot run time: 169487 ms

@doris-robot
Copy link
Copy Markdown

doris-robot commented Aug 2, 2024

ClickBench: Total hot run time: 30.2 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 4b441dfa6f89fd3b792f14cd6542789f3e119d87, data reload: false

query1	0.05	0.04	0.04
query2	0.07	0.04	0.04
query3	0.22	0.05	0.04
query4	1.70	0.06	0.07
query5	0.48	0.48	0.48
query6	1.14	0.72	0.71
query7	0.02	0.01	0.02
query8	0.05	0.04	0.04
query9	0.58	0.51	0.52
query10	0.56	0.56	0.56
query11	0.16	0.11	0.11
query12	0.15	0.12	0.12
query13	0.61	0.62	0.60
query14	0.78	0.80	0.78
query15	0.90	0.86	0.85
query16	0.36	0.35	0.36
query17	1.02	1.04	0.98
query18	0.23	0.21	0.20
query19	1.81	1.76	1.74
query20	0.01	0.01	0.01
query21	15.38	0.74	0.66
query22	4.08	8.14	1.38
query23	18.09	1.40	1.39
query24	2.28	0.22	0.22
query25	0.18	0.09	0.08
query26	0.32	0.22	0.20
query27	0.46	0.23	0.23
query28	13.16	1.01	0.97
query29	12.54	3.37	3.40
query30	0.25	0.06	0.06
query31	2.87	0.42	0.39
query32	3.26	0.50	0.48
query33	2.96	2.94	2.96
query34	15.46	4.27	4.25
query35	4.30	4.30	4.29
query36	0.69	0.48	0.48
query37	0.19	0.16	0.17
query38	0.17	0.14	0.14
query39	0.04	0.04	0.03
query40	0.16	0.12	0.12
query41	0.11	0.04	0.05
query42	0.06	0.05	0.05
query43	0.05	0.04	0.04
Total cold run time: 107.96 s
Total hot run time: 30.2 s

@yiguolei yiguolei force-pushed the disable_page_index branch from 4b441df to 74dc076 Compare August 5, 2024 06:33
@yiguolei
Copy link
Copy Markdown
Contributor

yiguolei commented Aug 5, 2024

run buildall

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Aug 5, 2024

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link
Copy Markdown

doris-robot commented Aug 5, 2024

TPC-H: Total hot run time: 41506 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 74dc076f4a42ef5de4fdea0661d418b9e8c88381, data reload: false

------ Round 1 ----------------------------------
q1	17606	4176	4068	4068
q2	2029	201	211	201
q3	10463	1249	1317	1249
q4	10163	871	905	871
q5	7659	2994	2979	2979
q6	217	137	135	135
q7	1061	612	628	612
q8	9454	1862	1949	1862
q9	8592	6594	6565	6565
q10	8763	3850	3823	3823
q11	436	245	247	245
q12	432	225	231	225
q13	17769	2908	2944	2908
q14	273	248	248	248
q15	522	491	496	491
q16	537	396	378	378
q17	967	931	929	929
q18	7983	7276	7164	7164
q19	1447	1224	1220	1220
q20	562	317	335	317
q21	5322	4787	4736	4736
q22	351	288	280	280
Total cold run time: 112608 ms
Total hot run time: 41506 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4089	4043	4003	4003
q2	326	230	229	229
q3	3019	2982	3163	2982
q4	1997	2028	1960	1960
q5	5644	5452	5444	5444
q6	217	129	130	129
q7	2126	1837	1835	1835
q8	3287	3334	3354	3334
q9	8664	8666	8736	8666
q10	4012	3956	3973	3956
q11	555	472	463	463
q12	792	608	633	608
q13	14511	3101	3105	3101
q14	315	278	263	263
q15	525	487	497	487
q16	454	431	419	419
q17	1803	1749	1742	1742
q18	8325	7761	7658	7658
q19	1766	1736	1750	1736
q20	2064	1835	1834	1834
q21	5740	5348	5368	5348
q22	511	463	462	462
Total cold run time: 70742 ms
Total hot run time: 56659 ms

@doris-robot
Copy link
Copy Markdown

doris-robot commented Aug 5, 2024

TPC-DS: Total hot run time: 167939 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 74dc076f4a42ef5de4fdea0661d418b9e8c88381, data reload: false

query1	903	379	372	372
query2	6475	1633	1642	1633
query3	6662	217	231	217
query4	19898	17276	17411	17276
query5	3610	502	522	502
query6	275	167	161	161
query7	4587	293	297	293
query8	262	216	203	203
query9	8510	2390	2375	2375
query10	423	273	267	267
query11	10565	9974	9905	9905
query12	120	91	86	86
query13	1653	372	379	372
query14	9620	6099	7016	6099
query15	220	167	166	166
query16	6979	486	473	473
query17	947	567	593	567
query18	1909	277	275	275
query19	203	144	141	141
query20	88	87	82	82
query21	209	97	98	97
query22	4274	4151	3948	3948
query23	33650	33859	33411	33411
query24	9531	3025	3098	3025
query25	640	401	423	401
query26	1198	161	151	151
query27	2390	279	284	279
query28	7248	2010	2005	2005
query29	965	431	423	423
query30	240	155	152	152
query31	954	785	763	763
query32	103	54	59	54
query33	678	326	320	320
query34	933	500	496	496
query35	855	760	752	752
query36	1067	919	887	887
query37	171	85	86	85
query38	2956	2839	2828	2828
query39	861	823	800	800
query40	201	114	115	114
query41	49	44	44	44
query42	129	99	101	99
query43	484	430	405	405
query44	1202	749	749	749
query45	206	178	174	174
query46	1101	835	774	774
query47	1796	1729	1677	1677
query48	368	285	291	285
query49	833	435	427	427
query50	905	436	440	436
query51	6710	6691	6696	6691
query52	112	89	99	89
query53	250	191	186	186
query54	637	454	453	453
query55	76	81	76	76
query56	268	250	253	250
query57	1161	1014	1059	1014
query58	262	271	275	271
query59	2544	2322	2264	2264
query60	296	268	274	268
query61	100	92	100	92
query62	898	643	649	643
query63	210	214	184	184
query64	4472	1898	1882	1882
query65	3143	3086	3101	3086
query66	860	325	323	323
query67	15149	14729	14587	14587
query68	4380	577	596	577
query69	675	404	336	336
query70	1072	1037	1038	1037
query71	411	321	280	280
query72	7583	2647	2534	2534
query73	780	326	328	326
query74	6047	5563	5675	5563
query75	3372	2713	2730	2713
query76	2705	1200	1278	1200
query77	550	314	307	307
query78	9429	8985	8928	8928
query79	2207	540	541	540
query80	889	511	502	502
query81	556	227	227	227
query82	1356	135	132	132
query83	198	172	167	167
query84	269	79	79	79
query85	1423	317	305	305
query86	477	272	285	272
query87	3323	3077	3160	3077
query88	3768	2384	2431	2384
query89	384	295	290	290
query90	1784	194	195	194
query91	124	99	104	99
query92	62	49	50	49
query93	2153	619	614	614
query94	755	292	293	292
query95	386	270	268	268
query96	608	285	280	280
query97	3262	3031	3107	3031
query98	223	201	195	195
query99	1632	1256	1274	1256
Total cold run time: 260097 ms
Total hot run time: 167939 ms

@doris-robot
Copy link
Copy Markdown

doris-robot commented Aug 5, 2024

ClickBench: Total hot run time: 30.08 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 74dc076f4a42ef5de4fdea0661d418b9e8c88381, data reload: false

query1	0.04	0.04	0.04
query2	0.07	0.04	0.03
query3	0.23	0.05	0.05
query4	1.69	0.07	0.07
query5	0.48	0.48	0.48
query6	1.14	0.72	0.71
query7	0.01	0.01	0.01
query8	0.04	0.04	0.04
query9	0.56	0.50	0.51
query10	0.56	0.55	0.58
query11	0.15	0.12	0.11
query12	0.14	0.12	0.13
query13	0.63	0.60	0.61
query14	0.78	0.79	0.81
query15	0.90	0.87	0.86
query16	0.35	0.35	0.36
query17	0.99	1.00	1.01
query18	0.25	0.24	0.21
query19	1.83	1.72	1.77
query20	0.01	0.01	0.01
query21	15.39	0.77	0.66
query22	3.68	8.05	1.36
query23	17.96	1.29	1.33
query24	2.26	0.22	0.22
query25	0.18	0.08	0.08
query26	0.32	0.22	0.22
query27	0.46	0.23	0.23
query28	13.19	0.98	0.95
query29	12.55	3.27	3.24
query30	0.25	0.06	0.06
query31	2.87	0.40	0.41
query32	3.25	0.50	0.49
query33	2.92	2.99	2.98
query34	15.44	4.28	4.27
query35	4.30	4.31	4.32
query36	0.68	0.47	0.48
query37	0.18	0.16	0.16
query38	0.16	0.16	0.15
query39	0.04	0.04	0.03
query40	0.15	0.14	0.13
query41	0.10	0.05	0.04
query42	0.06	0.05	0.05
query43	0.06	0.04	0.04
Total cold run time: 107.3 s
Total hot run time: 30.08 s

morningman added a commit to morningman/doris that referenced this pull request Aug 5, 2024
@morningman
[fix](parquet) disable parquet page index by default apache#38691
5043e02
@morningman morningman mentioned this pull request Aug 5, 2024
Merged
yiguolei pushed a commit that referenced this pull request Aug 6, 2024
@morningman
[fix](parquet) disable parquet page index by default #38691 (#38901)
e9bf077 
bp #38691
@yiguolei yiguolei force-pushed the disable_page_index branch from 74dc076 to 172bdeb Compare August 7, 2024 11:33
@github-actions github-actions bot added the doing label Aug 7, 2024
@yiguolei
Copy link
Copy Markdown
Contributor

yiguolei commented Aug 7, 2024

run buildall

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Aug 7, 2024

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link
Copy Markdown

doris-robot commented Aug 7, 2024

TPC-H: Total hot run time: 41906 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 172bdeb6d18297902491556a6b62a4f56ac1f191, data reload: false

------ Round 1 ----------------------------------
q1	17855	4176	4083	4083
q2	2035	204	203	203
q3	10462	1314	1362	1314
q4	10164	809	975	809
q5	7667	3022	3016	3016
q6	225	140	141	140
q7	1050	618	617	617
q8	9457	1888	1937	1888
q9	8541	6619	6635	6619
q10	8753	3863	3816	3816
q11	434	262	260	260
q12	410	234	231	231
q13	17770	2954	2973	2954
q14	274	254	247	247
q15	526	488	498	488
q16	538	404	392	392
q17	973	962	917	917
q18	8132	7328	7293	7293
q19	1687	1245	1224	1224
q20	564	332	342	332
q21	5299	4870	4776	4776
q22	362	287	296	287
Total cold run time: 113178 ms
Total hot run time: 41906 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4098	4029	4025	4025
q2	325	222	222	222
q3	3010	3041	3151	3041
q4	1999	2051	1940	1940
q5	5649	5457	5439	5439
q6	221	132	131	131
q7	2116	1769	1839	1769
q8	3323	3369	3348	3348
q9	8733	8703	8844	8703
q10	3940	4092	3889	3889
q11	561	462	463	462
q12	770	612	613	612
q13	16473	3125	3115	3115
q14	312	289	263	263
q15	536	490	487	487
q16	467	427	407	407
q17	1764	1760	1752	1752
q18	8294	7724	7645	7645
q19	1723	1761	1767	1761
q20	2066	1843	1850	1843
q21	5704	5507	5243	5243
q22	537	477	454	454
Total cold run time: 72621 ms
Total hot run time: 56551 ms

@doris-robot
Copy link
Copy Markdown

doris-robot commented Aug 7, 2024

TPC-DS: Total hot run time: 170271 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 172bdeb6d18297902491556a6b62a4f56ac1f191, data reload: false

query1	913	373	371	371
query2	6466	1738	1757	1738
query3	6650	215	223	215
query4	20539	17471	17319	17319
query5	3655	516	520	516
query6	287	175	176	175
query7	4596	297	299	297
query8	255	187	189	187
query9	8508	2345	2334	2334
query10	414	266	265	265
query11	10572	10139	10076	10076
query12	116	89	89	89
query13	1622	390	370	370
query14	8515	7065	7589	7065
query15	209	166	169	166
query16	6890	458	463	458
query17	957	555	563	555
query18	1796	283	280	280
query19	185	142	141	141
query20	91	83	94	83
query21	205	99	98	98
query22	4200	4148	4206	4148
query23	34052	33521	33430	33430
query24	10316	3131	3097	3097
query25	699	445	400	400
query26	1738	161	157	157
query27	3026	278	281	278
query28	7503	2021	1991	1991
query29	1277	462	426	426
query30	240	161	154	154
query31	936	756	763	756
query32	98	58	60	58
query33	674	328	323	323
query34	931	507	519	507
query35	864	760	793	760
query36	1056	908	916	908
query37	299	87	85	85
query38	2954	2826	2806	2806
query39	904	819	818	818
query40	255	119	118	118
query41	47	46	45	45
query42	136	115	107	107
query43	459	442	429	429
query44	1112	742	767	742
query45	208	179	181	179
query46	1101	806	799	799
query47	1825	1732	1768	1732
query48	371	309	301	301
query49	960	438	445	438
query50	897	435	440	435
query51	6817	6768	6658	6658
query52	108	91	89	89
query53	259	181	180	180
query54	619	469	476	469
query55	84	76	76	76
query56	283	270	274	270
query57	1141	1073	1061	1061
query58	279	292	285	285
query59	2515	2572	2407	2407
query60	291	285	297	285
query61	122	118	116	116
query62	873	658	640	640
query63	213	189	185	185
query64	5795	1989	1962	1962
query65	3191	3134	3099	3099
query66	1324	342	346	342
query67	15208	14730	14719	14719
query68	4363	569	577	569
query69	533	314	322	314
query70	1099	1086	1032	1032
query71	479	290	286	286
query72	8020	2928	2541	2541
query73	760	332	328	328
query74	6054	5645	5663	5645
query75	3756	2702	2742	2702
query76	2757	1180	1292	1180
query77	662	310	311	310
query78	9726	8935	8849	8849
query79	4367	530	528	528
query80	1737	500	503	500
query81	557	230	229	229
query82	1340	136	133	133
query83	344	174	174	174
query84	273	82	85	82
query85	2046	301	291	291
query86	442	316	315	315
query87	3310	3112	3116	3112
query88	4250	2483	2490	2483
query89	444	299	288	288
query90	2048	195	195	195
query91	128	104	102	102
query92	61	54	51	51
query93	5447	611	609	609
query94	1047	275	296	275
query95	401	265	274	265
query96	643	282	285	282
query97	3269	3110	3075	3075
query98	236	200	196	196
query99	1620	1262	1275	1262
Total cold run time: 273920 ms
Total hot run time: 170271 ms

@doris-robot
Copy link
Copy Markdown

doris-robot commented Aug 7, 2024

ClickBench: Total hot run time: 29.73 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 172bdeb6d18297902491556a6b62a4f56ac1f191, data reload: false

query1	0.05	0.04	0.04
query2	0.07	0.03	0.04
query3	0.22	0.05	0.05
query4	1.68	0.07	0.07
query5	0.49	0.48	0.48
query6	1.16	0.73	0.72
query7	0.03	0.01	0.01
query8	0.05	0.04	0.04
query9	0.58	0.52	0.51
query10	0.58	0.58	0.56
query11	0.16	0.12	0.12
query12	0.15	0.12	0.12
query13	0.62	0.61	0.61
query14	0.78	0.80	0.80
query15	0.91	0.87	0.87
query16	0.35	0.35	0.35
query17	0.98	0.99	0.98
query18	0.22	0.22	0.23
query19	1.83	1.72	1.70
query20	0.01	0.01	0.01
query21	15.39	0.75	0.65
query22	4.63	8.24	1.06
query23	17.60	1.28	1.24
query24	2.26	0.22	0.22
query25	0.19	0.09	0.08
query26	0.32	0.22	0.21
query27	0.47	0.24	0.23
query28	13.16	1.00	0.97
query29	12.55	3.28	3.27
query30	0.26	0.07	0.05
query31	2.84	0.42	0.40
query32	3.23	0.48	0.48
query33	2.99	2.98	2.97
query34	15.43	4.28	4.25
query35	4.30	4.29	4.30
query36	0.67	0.48	0.48
query37	0.20	0.16	0.16
query38	0.16	0.15	0.16
query39	0.04	0.03	0.03
query40	0.16	0.12	0.12
query41	0.10	0.05	0.05
query42	0.06	0.04	0.05
query43	0.04	0.04	0.04
Total cold run time: 107.97 s
Total hot run time: 29.73 s

@yiguolei yiguolei merged commit 547ff3c into apache:master Aug 8, 2024
@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Aug 8, 2024
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Aug 8, 2024

PR approved by at least one committer and no changes requested.

dataroaring pushed a commit that referenced this pull request Aug 11, 2024
@morningman @dataroaring
[fix](parquet) disable parquet page index by default (#38691)
44f8d3b 
there are some issue with page index parsing.
disable it temporarily
wyxxxcat pushed a commit to wyxxxcat/doris that referenced this pull request Aug 14, 2024
@morningman @wyxxxcat
[fix](parquet) disable parquet page index by default (apache#38691)
74edd87 
there are some issue with page index parsing.
disable it temporarily
dataroaring pushed a commit that referenced this pull request Aug 16, 2024
@morningman @dataroaring
[fix](parquet) disable parquet page index by default (#38691)
f17241c 
there are some issue with page index parsing.
disable it temporarily
@gavinchou gavinchou mentioned this pull request Oct 13, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@yiguolei yiguolei yiguolei approved these changes

+1 more reviewer

@suxiaogang223 suxiaogang223 suxiaogang223 approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/2.1.6-merged dev/3.0.2-merged doing reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@morningman @doris-robot @yiguolei @suxiaogang223 @dataroaring