[opt] (inverted index) add inverted index file size for open file by csun5285 · Pull Request #37482 · apache/doris

csun5285 · 2024-07-08T11:17:37Z

Proposed changes

Issue Number: close #xxx

doris-robot · 2024-07-08T11:17:42Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

github-actions · 2024-07-08T11:25:00Z

clang-tidy review says "All clean, LGTM! 👍"

github-actions · 2024-07-08T12:21:17Z

clang-tidy review says "All clean, LGTM! 👍"

github-actions · 2024-07-09T02:03:43Z

clang-tidy review says "All clean, LGTM! 👍"

github-actions · 2024-07-09T10:54:22Z

clang-tidy review says "All clean, LGTM! 👍"

github-actions · 2024-07-09T12:00:10Z

clang-tidy review says "All clean, LGTM! 👍"

csun5285 · 2024-07-09T12:49:04Z

run buildall

csun5285 · 2024-07-09T13:59:44Z

run buildall

github-actions · 2024-07-09T14:06:55Z

clang-tidy review says "All clean, LGTM! 👍"

doris-robot · 2024-07-09T15:41:04Z

TPC-H: Total hot run time: 40355 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 14dd4e2fa0113b25e6ff8426789233f7548b589b, data reload: false

------ Round 1 ----------------------------------
q1	17755	4555	4403	4403
q2	2039	193	198	193
q3	10511	1266	1149	1149
q4	10248	815	788	788
q5	7491	2718	2684	2684
q6	233	139	140	139
q7	975	603	605	603
q8	9233	2147	2124	2124
q9	8799	6567	6562	6562
q10	8965	3772	3767	3767
q11	461	231	239	231
q12	449	231	234	231
q13	17772	2965	3006	2965
q14	267	220	231	220
q15	537	487	478	478
q16	521	376	375	375
q17	976	639	754	639
q18	8256	7567	7450	7450
q19	3598	1536	1446	1446
q20	650	324	311	311
q21	4930	3275	3264	3264
q22	396	333	333	333
Total cold run time: 115062 ms
Total hot run time: 40355 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4450	4320	4296	4296
q2	374	273	269	269
q3	3006	2812	2860	2812
q4	1980	1715	1714	1714
q5	5684	5535	5503	5503
q6	226	134	134	134
q7	2211	1825	1892	1825
q8	3313	3448	3480	3448
q9	8774	8730	8817	8730
q10	4119	3961	3819	3819
q11	592	491	504	491
q12	820	672	627	627
q13	16560	3144	3245	3144
q14	308	280	275	275
q15	534	481	496	481
q16	497	436	428	428
q17	1862	1546	1546	1546
q18	8281	7944	7826	7826
q19	1838	1599	1660	1599
q20	2123	1906	1867	1867
q21	5125	5066	4956	4956
q22	652	548	535	535
Total cold run time: 73329 ms
Total hot run time: 56325 ms

doris-robot · 2024-07-09T15:52:13Z

TPC-DS: Total hot run time: 172656 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 14dd4e2fa0113b25e6ff8426789233f7548b589b, data reload: false

query1	923	379	372	372
query2	6451	2470	2292	2292
query3	6632	206	213	206
query4	28581	17477	17372	17372
query5	3687	494	493	493
query6	266	195	161	161
query7	4579	284	284	284
query8	328	283	285	283
query9	8465	2421	2390	2390
query10	443	297	274	274
query11	11761	10046	10059	10046
query12	122	80	81	80
query13	1639	387	372	372
query14	10146	7520	6910	6910
query15	240	183	196	183
query16	7405	306	306	306
query17	1386	546	524	524
query18	1667	278	286	278
query19	188	151	156	151
query20	90	81	81	81
query21	215	122	122	122
query22	4366	4134	4092	4092
query23	34013	33534	33641	33534
query24	11361	2931	2917	2917
query25	628	394	411	394
query26	773	146	148	146
query27	2348	280	283	280
query28	7054	2173	2187	2173
query29	918	626	645	626
query30	264	147	156	147
query31	970	768	754	754
query32	95	64	57	57
query33	756	289	337	289
query34	1017	497	507	497
query35	691	575	557	557
query36	1111	960	955	955
query37	153	88	83	83
query38	2907	2906	2894	2894
query39	906	846	848	846
query40	227	149	129	129
query41	59	52	50	50
query42	120	102	106	102
query43	584	529	546	529
query44	1237	732	725	725
query45	191	161	160	160
query46	1067	743	721	721
query47	1845	1755	1777	1755
query48	359	297	299	297
query49	845	411	422	411
query50	776	397	396	396
query51	6938	6866	6814	6814
query52	113	96	96	96
query53	360	288	303	288
query54	927	453	454	453
query55	74	76	76	76
query56	311	276	275	275
query57	1115	1085	1074	1074
query58	246	256	261	256
query59	3503	3022	3293	3022
query60	304	278	313	278
query61	100	94	98	94
query62	795	651	663	651
query63	332	300	286	286
query64	9197	2162	1626	1626
query65	3192	3065	3103	3065
query66	778	327	326	326
query67	15383	14941	15036	14941
query68	4471	559	539	539
query69	504	343	343	343
query70	1197	1120	1170	1120
query71	421	284	292	284
query72	7062	5218	4137	4137
query73	738	332	324	324
query74	6066	5589	5617	5589
query75	3425	2714	2662	2662
query76	2652	958	1031	958
query77	483	311	300	300
query78	9474	9140	8876	8876
query79	2712	519	517	517
query80	1771	477	491	477
query81	580	220	220	220
query82	785	136	136	136
query83	307	175	169	169
query84	278	88	86	86
query85	1948	318	312	312
query86	478	319	312	312
query87	3292	3107	3110	3107
query88	4337	2383	2345	2345
query89	489	392	393	392
query90	1783	195	193	193
query91	130	102	103	102
query92	60	53	51	51
query93	3886	533	531	531
query94	1130	214	219	214
query95	398	313	327	313
query96	612	279	272	272
query97	3159	3007	3032	3007
query98	278	195	197	195
query99	1748	1256	1256	1256
Total cold run time: 283177 ms
Total hot run time: 172656 ms

doris-robot · 2024-07-09T15:57:47Z

ClickBench: Total hot run time: 30.45 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 14dd4e2fa0113b25e6ff8426789233f7548b589b, data reload: false

query1	0.05	0.03	0.03
query2	0.08	0.04	0.04
query3	0.24	0.05	0.05
query4	1.67	0.08	0.08
query5	0.48	0.47	0.48
query6	1.14	0.73	0.73
query7	0.02	0.01	0.02
query8	0.05	0.05	0.05
query9	0.56	0.50	0.48
query10	0.55	0.55	0.54
query11	0.16	0.12	0.12
query12	0.15	0.12	0.13
query13	0.60	0.59	0.58
query14	0.79	0.78	0.77
query15	0.84	0.81	0.81
query16	0.34	0.36	0.36
query17	0.98	1.00	0.94
query18	0.24	0.22	0.22
query19	1.79	1.71	1.69
query20	0.01	0.01	0.01
query21	15.41	0.79	0.66
query22	4.28	6.97	1.83
query23	18.29	1.31	1.24
query24	2.09	0.23	0.24
query25	0.17	0.09	0.08
query26	0.32	0.21	0.20
query27	0.45	0.23	0.23
query28	13.25	1.03	1.00
query29	12.56	3.27	3.28
query30	0.27	0.06	0.06
query31	2.84	0.39	0.39
query32	3.27	0.49	0.47
query33	2.90	2.94	2.85
query34	17.01	4.44	4.37
query35	4.44	4.43	4.52
query36	0.65	0.49	0.47
query37	0.20	0.17	0.15
query38	0.16	0.14	0.14
query39	0.04	0.03	0.04
query40	0.15	0.12	0.13
query41	0.10	0.05	0.05
query42	0.06	0.05	0.05
query43	0.04	0.04	0.03
Total cold run time: 109.69 s
Total hot run time: 30.45 s

csun5285 · 2024-07-31T08:29:16Z

run buildall

csun5285 · 2024-07-31T09:09:44Z

run buildall

github-actions

clang-tidy made some suggestions

doris-robot · 2024-07-31T10:51:37Z

TPC-H: Total hot run time: 41713 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 1e34d3f2f2e8fc40588dd96971b4ffb5d1411f77, data reload: false

------ Round 1 ----------------------------------
q1	17671	4561	4128	4128
q2	2035	201	200	200
q3	10588	1317	1392	1317
q4	10226	834	941	834
q5	7682	2950	2992	2950
q6	222	137	140	137
q7	1054	620	617	617
q8	9449	1800	1968	1800
q9	8599	6607	6612	6607
q10	8702	3869	3838	3838
q11	432	254	258	254
q12	422	233	232	232
q13	17767	2948	2997	2948
q14	275	256	246	246
q15	533	490	496	490
q16	499	401	390	390
q17	992	948	923	923
q18	8252	7340	7231	7231
q19	1487	1227	1223	1223
q20	560	333	340	333
q21	5325	4730	4745	4730
q22	354	290	285	285
Total cold run time: 113126 ms
Total hot run time: 41713 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4090	4009	4044	4009
q2	332	222	220	220
q3	3017	3071	3175	3071
q4	2069	2019	2017	2017
q5	5671	5506	5458	5458
q6	221	131	131	131
q7	2172	1805	1863	1805
q8	3357	3422	3391	3391
q9	8714	8771	8896	8771
q10	3986	4124	3922	3922
q11	547	474	481	474
q12	773	625	608	608
q13	15002	3119	3160	3119
q14	300	288	273	273
q15	519	503	493	493
q16	495	440	414	414
q17	1773	1771	1785	1771
q18	8235	7810	7723	7723
q19	1854	1723	1757	1723
q20	2118	1840	1842	1840
q21	5804	5535	5267	5267
q22	571	471	463	463
Total cold run time: 71620 ms
Total hot run time: 56963 ms

doris-robot · 2024-08-07T10:56:08Z

TPC-DS: Total hot run time: 168435 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d8a3b6b51e28c0fe96d11892b3c267f0618a0168, data reload: false

query1	911	378	377	377
query2	6451	1716	1654	1654
query3	6646	221	224	221
query4	20154	17580	17240	17240
query5	3644	523	506	506
query6	273	181	157	157
query7	4602	303	289	289
query8	242	200	202	200
query9	8514	2399	2396	2396
query10	458	284	263	263
query11	10510	10037	9967	9967
query12	127	88	89	88
query13	1630	389	375	375
query14	8961	6812	6025	6025
query15	204	159	166	159
query16	6979	458	487	458
query17	962	569	552	552
query18	1910	289	292	289
query19	199	155	146	146
query20	90	90	87	87
query21	206	102	110	102
query22	4118	4157	3921	3921
query23	33571	34057	33453	33453
query24	10266	3184	3119	3119
query25	742	420	402	402
query26	1757	157	162	157
query27	3043	294	292	292
query28	7565	2047	2038	2038
query29	1359	449	429	429
query30	232	162	159	159
query31	957	750	757	750
query32	101	54	57	54
query33	676	334	319	319
query34	922	509	506	506
query35	893	782	722	722
query36	1044	886	894	886
query37	295	98	89	89
query38	2922	2820	2772	2772
query39	857	815	806	806
query40	248	112	110	110
query41	48	45	44	44
query42	128	102	106	102
query43	445	425	419	419
query44	1173	728	721	721
query45	209	183	185	183
query46	1079	803	782	782
query47	1820	1758	1756	1756
query48	376	294	291	291
query49	941	417	418	417
query50	893	444	437	437
query51	6786	6795	6675	6675
query52	107	87	88	87
query53	251	180	184	180
query54	636	462	450	450
query55	75	76	74	74
query56	273	263	246	246
query57	1150	1036	1054	1036
query58	250	273	285	273
query59	2475	2339	2325	2325
query60	296	273	264	264
query61	96	89	90	89
query62	879	630	640	630
query63	217	183	177	177
query64	5612	1882	1863	1863
query65	3180	3098	3126	3098
query66	1308	330	332	330
query67	15439	14792	14953	14792
query68	4483	549	575	549
query69	466	305	298	298
query70	1104	1073	1061	1061
query71	433	282	275	275
query72	7525	2667	2540	2540
query73	762	333	329	329
query74	6054	5602	5646	5602
query75	3367	2683	2736	2683
query76	2434	1187	1285	1187
query77	467	322	331	322
query78	9456	8995	8849	8849
query79	1660	533	526	526
query80	952	516	517	516
query81	570	229	230	229
query82	521	131	133	131
query83	261	171	165	165
query84	271	82	79	79
query85	1210	321	357	321
query86	432	320	292	292
query87	3330	3117	3107	3107
query88	3062	2486	2492	2486
query89	390	283	292	283
query90	1727	192	191	191
query91	126	101	103	101
query92	64	51	55	51
query93	1995	632	626	626
query94	844	303	306	303
query95	375	269	276	269
query96	609	295	281	281
query97	3234	3040	3090	3040
query98	215	200	200	200
query99	1597	1287	1267	1267
Total cold run time: 261379 ms
Total hot run time: 168435 ms

doris-robot · 2024-08-07T11:01:40Z

ClickBench: Total hot run time: 30.25 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit d8a3b6b51e28c0fe96d11892b3c267f0618a0168, data reload: false

query1	0.04	0.04	0.04
query2	0.07	0.04	0.04
query3	0.23	0.05	0.05
query4	1.68	0.07	0.08
query5	0.50	0.49	0.48
query6	1.15	0.74	0.72
query7	0.02	0.01	0.01
query8	0.05	0.05	0.05
query9	0.56	0.52	0.55
query10	0.56	0.56	0.57
query11	0.16	0.11	0.11
query12	0.15	0.12	0.13
query13	0.63	0.60	0.59
query14	0.77	0.80	0.80
query15	0.91	0.87	0.87
query16	0.35	0.36	0.36
query17	1.04	1.03	1.00
query18	0.22	0.21	0.21
query19	1.86	1.72	1.77
query20	0.01	0.01	0.01
query21	15.45	0.76	0.65
query22	3.73	8.47	1.49
query23	18.18	1.38	1.26
query24	2.23	0.23	0.22
query25	0.18	0.08	0.08
query26	0.32	0.20	0.22
query27	0.46	0.24	0.23
query28	13.16	0.99	0.97
query29	12.57	3.31	3.27
query30	0.25	0.07	0.06
query31	2.85	0.41	0.42
query32	3.23	0.50	0.49
query33	2.97	2.99	2.95
query34	15.45	4.30	4.30
query35	4.35	4.28	4.25
query36	0.68	0.51	0.48
query37	0.18	0.16	0.16
query38	0.16	0.15	0.15
query39	0.04	0.04	0.04
query40	0.17	0.13	0.14
query41	0.09	0.05	0.05
query42	0.05	0.04	0.05
query43	0.05	0.04	0.04
Total cold run time: 107.76 s
Total hot run time: 30.25 s

csun5285 · 2024-08-13T01:58:01Z

run buildall

doris-robot · 2024-08-13T03:37:20Z

TPC-H: Total hot run time: 39514 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 298c058a1574a85547d95d7d3455b40054c2ae93, data reload: false

------ Round 1 ----------------------------------
q1	18121	5000	4270	4270
q2	2007	204	203	203
q3	11300	993	1154	993
q4	10496	764	686	686
q5	7502	2573	2513	2513
q6	224	140	139	139
q7	969	605	611	605
q8	9228	1938	1955	1938
q9	8740	6548	6602	6548
q10	7044	2291	2198	2198
q11	459	243	248	243
q12	396	246	228	228
q13	17989	3005	2981	2981
q14	292	249	236	236
q15	516	488	489	488
q16	502	400	385	385
q17	992	684	722	684
q18	8154	7464	7498	7464
q19	6864	1088	919	919
q20	700	353	344	344
q21	5468	4445	4593	4445
q22	1108	1018	1004	1004
Total cold run time: 119071 ms
Total hot run time: 39514 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4450	4249	4238	4238
q2	384	277	279	277
q3	2904	2833	2829	2829
q4	1999	1868	1810	1810
q5	5476	5507	5530	5507
q6	235	140	137	137
q7	2199	1798	1800	1798
q8	3322	3463	3449	3449
q9	8839	8945	8940	8940
q10	3431	3177	3290	3177
q11	615	516	501	501
q12	850	640	651	640
q13	17423	3175	3174	3174
q14	338	291	295	291
q15	541	499	481	481
q16	511	421	433	421
q17	1840	1556	1539	1539
q18	8189	8135	7930	7930
q19	1762	1651	1581	1581
q20	2128	1925	1855	1855
q21	5561	5118	5394	5118
q22	1147	1061	1020	1020
Total cold run time: 74144 ms
Total hot run time: 56713 ms

doris-robot · 2024-08-13T03:50:18Z

TPC-DS: Total hot run time: 202800 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 298c058a1574a85547d95d7d3455b40054c2ae93, data reload: false

query1	1683	1464	1427	1427
query2	6339	1959	1914	1914
query3	14006	8551	312	312
query4	33816	23277	23008	23008
query5	4484	485	480	480
query6	260	161	151	151
query7	3940	291	297	291
query8	247	218	203	203
query9	8410	2468	2431	2431
query10	553	454	451	451
query11	17156	15062	14895	14895
query12	143	96	93	93
query13	1541	363	355	355
query14	9140	7743	7769	7743
query15	289	212	240	212
query16	7674	470	546	470
query17	1486	579	557	557
query18	1977	295	297	295
query19	217	142	149	142
query20	124	110	112	110
query21	217	106	110	106
query22	4562	4288	4249	4249
query23	34326	33597	33555	33555
query24	10465	2607	2626	2607
query25	597	393	388	388
query26	706	154	155	154
query27	2163	281	281	281
query28	6109	2042	2027	2027
query29	814	446	402	402
query30	255	148	151	148
query31	969	754	691	691
query32	94	52	55	52
query33	629	279	270	270
query34	840	468	471	468
query35	944	821	808	808
query36	1065	930	905	905
query37	130	82	88	82
query38	4291	4168	4104	4104
query39	1403	1359	1390	1359
query40	199	117	116	116
query41	45	44	44	44
query42	113	94	99	94
query43	524	460	462	460
query44	1048	719	720	719
query45	243	206	210	206
query46	1077	744	741	741
query47	1820	1737	1785	1737
query48	369	288	296	288
query49	834	410	422	410
query50	801	398	400	398
query51	6804	6680	6734	6680
query52	109	86	92	86
query53	254	181	180	180
query54	915	450	436	436
query55	74	73	79	73
query56	263	244	238	238
query57	1147	1086	1082	1082
query58	219	228	222	222
query59	3019	2673	2898	2673
query60	305	327	263	263
query61	97	94	93	93
query62	794	643	644	643
query63	211	186	183	183
query64	9165	2251	1764	1764
query65	3365	3217	3165	3165
query66	762	341	328	328
query67	15242	14954	14796	14796
query68	5978	549	544	544
query69	430	429	390	390
query70	1164	1045	1151	1045
query71	486	276	268	268
query72	19438	16934	16856	16856
query73	791	333	325	325
query74	9311	8898	8701	8701
query75	3652	2644	2696	2644
query76	3302	1046	956	956
query77	645	317	332	317
query78	9779	9238	8945	8945
query79	4064	520	511	511
query80	2273	489	541	489
query81	600	227	233	227
query82	813	135	134	134
query83	259	149	151	149
query84	278	76	78	76
query85	1278	277	271	271
query86	386	285	301	285
query87	4746	4541	4441	4441
query88	4610	2472	2485	2472
query89	563	293	285	285
query90	1952	197	195	195
query91	124	96	94	94
query92	59	51	48	48
query93	6168	546	535	535
query94	773	306	306	306
query95	354	259	313	259
query96	614	275	272	272
query97	3198	3021	3033	3021
query98	221	202	194	194
query99	1681	1231	1253	1231
Total cold run time: 323872 ms
Total hot run time: 202800 ms

doris-robot · 2024-08-13T03:55:51Z

ClickBench: Total hot run time: 31.46 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 298c058a1574a85547d95d7d3455b40054c2ae93, data reload: false

query1	0.04	0.04	0.04
query2	0.07	0.04	0.04
query3	0.22	0.06	0.05
query4	1.66	0.09	0.08
query5	0.50	0.51	0.49
query6	1.12	0.74	0.73
query7	0.02	0.01	0.01
query8	0.05	0.05	0.04
query9	0.55	0.49	0.49
query10	0.55	0.54	0.54
query11	0.15	0.11	0.11
query12	0.14	0.12	0.12
query13	0.63	0.59	0.59
query14	0.77	0.77	0.78
query15	0.83	0.82	0.82
query16	0.37	0.36	0.38
query17	1.04	1.01	1.00
query18	0.22	0.22	0.23
query19	1.83	1.78	1.77
query20	0.01	0.01	0.01
query21	15.42	0.74	0.65
query22	4.10	6.25	2.72
query23	18.20	1.36	1.19
query24	2.07	0.21	0.23
query25	0.16	0.08	0.08
query26	0.31	0.21	0.22
query27	0.45	0.23	0.23
query28	13.32	1.02	1.00
query29	12.65	3.28	3.28
query30	0.23	0.06	0.05
query31	2.97	0.40	0.37
query32	3.22	0.48	0.48
query33	2.90	2.90	2.92
query34	16.99	4.37	4.36
query35	4.45	4.41	4.38
query36	0.66	0.50	0.48
query37	0.20	0.16	0.16
query38	0.17	0.15	0.14
query39	0.04	0.04	0.05
query40	0.15	0.12	0.12
query41	0.09	0.05	0.04
query42	0.06	0.05	0.05
query43	0.05	0.04	0.04
Total cold run time: 109.63 s
Total hot run time: 31.46 s

add other codes remove unused add test fix add for v1 fix add exsits for write fix log fix fix be ut format fix log

csun5285 · 2024-08-14T02:50:47Z

run buildall

airborne12

LGTM

github-actions · 2024-08-14T03:30:04Z

PR approved by at least one committer and no changes requested.

github-actions · 2024-08-14T03:30:06Z

PR approved by anyone and no changes requested.

doris-robot · 2024-08-14T04:06:28Z

TPC-H: Total hot run time: 40184 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 03ce88db6ce65b8af6bcb221ba63ecb2e642c2ca, data reload: false

------ Round 1 ----------------------------------
q1	17618	4506	4361	4361
q2	2017	195	174	174
q3	10678	1253	1203	1203
q4	10705	794	760	760
q5	7827	2841	2820	2820
q6	230	140	143	140
q7	995	607	614	607
q8	9399	2079	2118	2079
q9	9335	6723	6739	6723
q10	7465	2245	2185	2185
q11	454	243	249	243
q12	396	229	218	218
q13	18032	3003	2983	2983
q14	289	237	240	237
q15	544	468	505	468
q16	497	406	387	387
q17	999	638	709	638
q18	8047	7420	7327	7327
q19	1563	1014	909	909
q20	689	318	325	318
q21	5349	4404	4503	4404
q22	1109	1023	1000	1000
Total cold run time: 114237 ms
Total hot run time: 40184 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4427	4241	4240	4240
q2	386	280	267	267
q3	2868	2660	2651	2651
q4	1951	1752	1770	1752
q5	5714	5646	5576	5576
q6	226	163	135	135
q7	2080	1684	1765	1684
q8	3302	3502	3424	3424
q9	8846	8838	8718	8718
q10	3460	3263	3265	3263
q11	609	506	498	498
q12	813	612	614	612
q13	16012	3111	3178	3111
q14	323	291	278	278
q15	511	502	494	494
q16	470	431	449	431
q17	1881	1556	1517	1517
q18	8214	7901	7814	7814
q19	5873	1620	1586	1586
q20	2115	1896	1835	1835
q21	5500	5027	5246	5027
q22	1121	1054	1042	1042
Total cold run time: 76702 ms
Total hot run time: 55955 ms

doris-robot · 2024-08-14T04:18:59Z

TPC-DS: Total hot run time: 188837 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 03ce88db6ce65b8af6bcb221ba63ecb2e642c2ca, data reload: false

query1	1246	884	863	863
query2	6277	1895	1785	1785
query3	10583	3863	3770	3770
query4	58985	25119	23109	23109
query5	5863	501	518	501
query6	497	164	167	164
query7	6525	300	298	298
query8	319	202	208	202
query9	9301	2402	2364	2364
query10	514	271	267	267
query11	16782	14987	15132	14987
query12	181	113	109	109
query13	1657	403	372	372
query14	12108	6437	7762	6437
query15	240	159	166	159
query16	7669	538	500	500
query17	1116	593	562	562
query18	2026	301	310	301
query19	267	152	147	147
query20	118	113	111	111
query21	215	105	99	99
query22	4579	4289	4182	4182
query23	34222	33396	33128	33128
query24	5584	2873	2828	2828
query25	534	382	371	371
query26	687	165	166	165
query27	1799	271	275	271
query28	3850	2021	2011	2011
query29	684	403	403	403
query30	235	146	145	145
query31	921	728	730	728
query32	89	55	54	54
query33	446	276	270	270
query34	860	468	480	468
query35	811	743	736	736
query36	1031	930	916	916
query37	133	76	79	76
query38	4001	3771	3777	3771
query39	1426	1357	1411	1357
query40	197	118	117	117
query41	47	43	43	43
query42	119	96	99	96
query43	503	455	464	455
query44	1094	739	733	733
query45	188	164	159	159
query46	1085	749	762	749
query47	1852	1744	1755	1744
query48	367	293	304	293
query49	748	411	417	411
query50	804	411	407	407
query51	6771	6742	6695	6695
query52	112	86	88	86
query53	256	185	180	180
query54	544	439	455	439
query55	79	77	75	75
query56	272	251	248	248
query57	1104	1073	1062	1062
query58	225	227	233	227
query59	2897	2718	2921	2718
query60	303	269	279	269
query61	113	114	117	114
query62	742	626	647	626
query63	211	183	181	181
query64	3303	1834	1786	1786
query65	3233	3176	3149	3149
query66	713	349	345	345
query67	15133	14778	14699	14699
query68	5378	560	553	553
query69	431	276	295	276
query70	1190	1148	1133	1133
query71	466	274	280	274
query72	7069	2396	2211	2211
query73	819	334	329	329
query74	9170	8649	8812	8649
query75	3382	2710	2653	2653
query76	2721	999	1122	999
query77	566	309	305	305
query78	10876	9058	10324	9058
query79	9023	541	544	541
query80	2543	495	503	495
query81	594	222	219	219
query82	1324	134	129	129
query83	299	148	143	143
query84	280	79	76	76
query85	1820	277	267	267
query86	383	281	293	281
query87	4545	4206	4233	4206
query88	4775	2558	2475	2475
query89	433	290	291	290
query90	1637	203	196	196
query91	122	97	95	95
query92	60	52	52	52
query93	6014	536	557	536
query94	714	297	286	286
query95	337	308	266	266
query96	628	280	273	273
query97	3178	2990	3061	2990
query98	220	207	211	207
query99	1499	1296	1281	1281
Total cold run time: 329892 ms
Total hot run time: 188837 ms

doris-robot · 2024-08-14T04:24:36Z

ClickBench: Total hot run time: 31.43 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 03ce88db6ce65b8af6bcb221ba63ecb2e642c2ca, data reload: false

query1	0.05	0.04	0.03
query2	0.08	0.04	0.04
query3	0.23	0.06	0.06
query4	1.65	0.10	0.09
query5	0.53	0.48	0.49
query6	1.13	0.71	0.73
query7	0.02	0.01	0.02
query8	0.05	0.04	0.04
query9	0.54	0.50	0.49
query10	0.53	0.53	0.53
query11	0.16	0.12	0.11
query12	0.14	0.12	0.12
query13	0.59	0.60	0.59
query14	0.76	0.79	0.77
query15	0.86	0.83	0.82
query16	0.37	0.35	0.37
query17	0.96	1.00	0.99
query18	0.23	0.23	0.21
query19	1.94	1.78	1.72
query20	0.01	0.01	0.01
query21	15.39	0.76	0.67
query22	4.97	5.54	2.58
query23	18.35	1.39	1.30
query24	2.10	0.22	0.22
query25	0.15	0.09	0.08
query26	0.29	0.22	0.21
query27	0.46	0.23	0.22
query28	13.30	1.03	1.01
query29	12.60	3.34	3.31
query30	0.24	0.05	0.05
query31	2.90	0.40	0.38
query32	3.24	0.48	0.48
query33	2.94	2.94	2.94
query34	17.05	4.33	4.40
query35	4.42	4.42	4.41
query36	0.66	0.47	0.47
query37	0.19	0.15	0.15
query38	0.15	0.16	0.15
query39	0.05	0.04	0.05
query40	0.16	0.12	0.12
query41	0.08	0.05	0.05
query42	0.06	0.05	0.05
query43	0.05	0.04	0.04
Total cold run time: 110.63 s
Total hot run time: 31.43 s

zzzxl1993

LGTM

qidaye

LGTM

…7482)

github-actions bot added the meta-change label Jul 8, 2024

csun5285 force-pushed the fix_file_size branch from 5843156 to b362a6c Compare July 8, 2024 12:13

csun5285 force-pushed the fix_file_size branch from c4d3802 to bea6a4e Compare July 9, 2024 11:52

csun5285 force-pushed the fix_file_size branch from bea6a4e to 14dd4e2 Compare July 9, 2024 13:59

airborne12 reviewed Jul 11, 2024

View reviewed changes

Comment thread be/src/olap/rowset/segment_v2/inverted_index_file_reader.h

csun5285 marked this pull request as draft July 16, 2024 04:32

xiaokang added usercase Important user case type label 2.1.0-conflict dev/2.0.x dev/3.0.x doing labels Jul 31, 2024

csun5285 force-pushed the fix_file_size branch from 14dd4e2 to 79a4dac Compare July 31, 2024 08:18

csun5285 marked this pull request as ready for review July 31, 2024 08:19

csun5285 force-pushed the fix_file_size branch from 79a4dac to 1e34d3f Compare July 31, 2024 09:09

github-actions bot reviewed Jul 31, 2024

View reviewed changes

Comment thread be/src/olap/rowset/beta_rowset_writer.h

Comment thread be/src/olap/rowset/segment_v2/inverted_index_file_writer.h

csun5285 force-pushed the fix_file_size branch from d8a3b6b to 298c058 Compare August 12, 2024 07:10

airborne12 reviewed Aug 13, 2024

View reviewed changes

Comment thread be/src/olap/rowset/segment_v2/inverted_index_compound_reader.cpp

csun5285 added 2 commits August 14, 2024 09:46

add file size

7ea8ee3

add other codes remove unused add test fix add for v1 fix add exsits for write fix log fix fix be ut format fix log

add debug point

03ce88d

csun5285 force-pushed the fix_file_size branch from 298c058 to 03ce88d Compare August 14, 2024 02:50

airborne12 approved these changes Aug 14, 2024

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Aug 14, 2024

github-actions bot added the reviewed label Aug 14, 2024

xiaokang removed the 2.1.0-conflict label Aug 14, 2024

zzzxl1993 approved these changes Aug 15, 2024

View reviewed changes

qidaye approved these changes Aug 16, 2024

View reviewed changes

qidaye merged commit 397b077 into apache:master Aug 16, 2024

dataroaring pushed a commit that referenced this pull request Aug 17, 2024

[opt] (inverted index) add inverted index file size for open file (#3…

1535ae5

…7482)

dataroaring added dev/3.0.2-merged and removed dev/3.0.x labels Aug 17, 2024

xiaokang removed dev/2.0.x doing labels Aug 20, 2024

gavinchou mentioned this pull request Oct 13, 2024

Release Note 3.0.2 #41558

Closed

Conversation

csun5285 commented Jul 8, 2024

Proposed changes

Uh oh!

doris-robot commented Jul 8, 2024

Uh oh!

github-actions bot commented Jul 8, 2024

Uh oh!

github-actions bot commented Jul 8, 2024

Uh oh!

github-actions bot commented Jul 9, 2024

Uh oh!

github-actions bot commented Jul 9, 2024

Uh oh!

github-actions bot commented Jul 9, 2024

Uh oh!

csun5285 commented Jul 9, 2024

Uh oh!

csun5285 commented Jul 9, 2024

Uh oh!

github-actions bot commented Jul 9, 2024

Uh oh!

doris-robot commented Jul 9, 2024

Uh oh!

doris-robot commented Jul 9, 2024

Uh oh!

doris-robot commented Jul 9, 2024

Uh oh!

Uh oh!

csun5285 commented Jul 31, 2024

Uh oh!

csun5285 commented Jul 31, 2024

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

doris-robot commented Jul 31, 2024

Uh oh!

doris-robot commented Aug 7, 2024

Uh oh!

doris-robot commented Aug 7, 2024

Uh oh!

csun5285 commented Aug 13, 2024

Uh oh!

doris-robot commented Aug 13, 2024

Uh oh!

doris-robot commented Aug 13, 2024

Uh oh!

doris-robot commented Aug 13, 2024

Uh oh!

Uh oh!

csun5285 commented Aug 14, 2024

Uh oh!

airborne12 left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Aug 14, 2024

Uh oh!

github-actions bot commented Aug 14, 2024

Uh oh!

doris-robot commented Aug 14, 2024

Uh oh!

doris-robot commented Aug 14, 2024

Uh oh!

doris-robot commented Aug 14, 2024

Uh oh!

zzzxl1993 left a comment

Choose a reason for hiding this comment

Uh oh!

qidaye left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development