Skip to content

[fix](auth)support check priv when tvf use resource#36928

Merged
morningman merged 6 commits intoapache:masterfrom
zddr:check_resource_auth
Jul 2, 2024
Merged

[fix](auth)support check priv when tvf use resource#36928
morningman merged 6 commits intoapache:masterfrom
zddr:check_resource_auth

Conversation

@zddr
Copy link
Copy Markdown
Contributor

@zddr zddr commented Jun 27, 2024

current, follow sql not check priv, we need check usage priv of test_resource

SELECT * FROM S3 (
    "uri" = "xxx",
    "format" = "orc",
    "resource" = "test_resource",
    "use_path_style" = "true"
);

@doris-robot
Copy link
Copy Markdown

doris-robot commented Jun 27, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

924060929
924060929 previously approved these changes Jun 27, 2024
View reviewed changes
@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jun 27, 2024
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Jun 27, 2024

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Jun 27, 2024

PR approved by anyone and no changes requested.

@zddr
Copy link
Copy Markdown
Contributor Author

zddr commented Jun 27, 2024

run buildall

@doris-robot
Copy link
Copy Markdown

doris-robot commented Jun 27, 2024

TPC-H: Total hot run time: 40271 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 3203547c11b531d78bf3d45fd4582f297b55210b, data reload: false

------ Round 1 ----------------------------------
q1	17609	4517	4341	4341
q2	2022	191	202	191
q3	10833	1243	1130	1130
q4	10573	762	812	762
q5	7830	2726	2671	2671
q6	227	138	139	138
q7	956	616	626	616
q8	9636	2137	2107	2107
q9	9158	6588	6557	6557
q10	8993	3742	3703	3703
q11	451	241	238	238
q12	530	229	239	229
q13	17775	2979	2992	2979
q14	269	221	219	219
q15	528	484	494	484
q16	525	381	374	374
q17	971	710	712	710
q18	8104	7391	7378	7378
q19	5091	1554	1524	1524
q20	659	304	326	304
q21	4993	3274	4010	3274
q22	410	351	342	342
Total cold run time: 118143 ms
Total hot run time: 40271 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4427	4254	4289	4254
q2	371	267	256	256
q3	2951	2776	2686	2686
q4	1889	1614	1626	1614
q5	5275	5233	5268	5233
q6	221	133	130	130
q7	2161	1735	1668	1668
q8	3237	3360	3349	3349
q9	8394	8354	8311	8311
q10	3878	3681	3680	3680
q11	582	479	478	478
q12	780	615	614	614
q13	17488	3013	3003	3003
q14	302	267	270	267
q15	510	473	476	473
q16	480	411	429	411
q17	1782	1466	1472	1466
q18	7652	7537	7534	7534
q19	1760	1572	1650	1572
q20	1994	1791	1763	1763
q21	4917	4676	4799	4676
q22	644	566	561	561
Total cold run time: 71695 ms
Total hot run time: 53999 ms

@doris-robot
Copy link
Copy Markdown

doris-robot commented Jun 27, 2024

TPC-DS: Total hot run time: 172579 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 3203547c11b531d78bf3d45fd4582f297b55210b, data reload: false

query1	929	385	378	378
query2	6457	2423	2314	2314
query3	6645	209	216	209
query4	18877	17565	17451	17451
query5	4192	467	481	467
query6	259	165	176	165
query7	4589	291	293	291
query8	308	306	289	289
query9	8497	2305	2326	2305
query10	606	305	286	286
query11	10650	10149	10010	10010
query12	138	87	83	83
query13	1644	368	369	368
query14	10290	6848	6533	6533
query15	231	186	188	186
query16	8063	269	272	269
query17	1902	555	519	519
query18	2060	303	286	286
query19	194	151	153	151
query20	93	82	93	82
query21	215	136	128	128
query22	4504	4298	4182	4182
query23	33879	32991	33185	32991
query24	11653	2825	2831	2825
query25	658	359	364	359
query26	1794	157	152	152
query27	2990	315	306	306
query28	7642	2008	1998	1998
query29	1063	598	610	598
query30	285	156	147	147
query31	960	728	744	728
query32	98	51	57	51
query33	771	282	309	282
query34	1015	459	470	459
query35	729	621	621	621
query36	1115	919	935	919
query37	207	72	70	70
query38	2917	2755	2762	2755
query39	855	798	814	798
query40	280	124	124	124
query41	55	53	51	51
query42	122	96	99	96
query43	601	567	556	556
query44	1205	725	732	725
query45	198	167	161	161
query46	1085	704	705	704
query47	1852	1789	1799	1789
query48	368	307	287	287
query49	1204	405	406	405
query50	784	386	378	378
query51	6968	6753	6808	6753
query52	113	91	95	91
query53	363	292	281	281
query54	999	442	444	442
query55	75	71	74	71
query56	310	260	267	260
query57	1159	1035	1080	1035
query58	263	242	245	242
query59	3528	3330	3232	3232
query60	292	266	275	266
query61	91	92	94	92
query62	645	451	440	440
query63	318	289	283	283
query64	9847	2325	1719	1719
query65	3152	3075	3078	3075
query66	1367	344	327	327
query67	15666	14966	15016	14966
query68	4640	540	539	539
query69	527	420	344	344
query70	1101	1135	1124	1124
query71	386	273	275	273
query72	6998	5391	5296	5296
query73	755	321	326	321
query74	5908	5489	5529	5489
query75	3529	2662	2702	2662
query76	2832	1008	929	929
query77	425	295	304	295
query78	10335	9755	9854	9755
query79	2635	520	508	508
query80	868	462	462	462
query81	579	229	223	223
query82	786	103	102	102
query83	319	167	166	166
query84	237	90	88	88
query85	1564	271	266	266
query86	494	316	329	316
query87	3300	3152	3094	3094
query88	3981	2441	2451	2441
query89	478	380	381	380
query90	1774	190	187	187
query91	127	100	98	98
query92	59	51	49	49
query93	1745	490	509	490
query94	1224	188	187	187
query95	403	305	306	305
query96	602	273	264	264
query97	3260	3018	3064	3018
query98	232	209	202	202
query99	1090	844	855	844
Total cold run time: 276077 ms
Total hot run time: 172579 ms

@doris-robot
Copy link
Copy Markdown

doris-robot commented Jun 27, 2024

ClickBench: Total hot run time: 30.42 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 3203547c11b531d78bf3d45fd4582f297b55210b, data reload: false

query1	0.04	0.03	0.03
query2	0.07	0.04	0.04
query3	0.23	0.05	0.05
query4	1.68	0.06	0.07
query5	0.49	0.48	0.47
query6	1.13	0.73	0.74
query7	0.02	0.02	0.01
query8	0.06	0.05	0.05
query9	0.55	0.50	0.48
query10	0.54	0.53	0.54
query11	0.15	0.12	0.12
query12	0.14	0.12	0.13
query13	0.60	0.59	0.61
query14	0.77	0.80	0.76
query15	0.85	0.82	0.82
query16	0.37	0.36	0.35
query17	1.03	1.02	0.94
query18	0.22	0.23	0.23
query19	1.79	1.75	1.78
query20	0.01	0.01	0.00
query21	15.46	0.76	0.67
query22	4.17	7.17	1.79
query23	18.25	1.44	1.23
query24	2.08	0.22	0.21
query25	0.14	0.09	0.09
query26	0.27	0.17	0.18
query27	0.07	0.08	0.07
query28	13.26	1.02	1.00
query29	12.57	3.27	3.28
query30	0.25	0.06	0.06
query31	2.86	0.39	0.39
query32	3.28	0.47	0.48
query33	2.92	2.90	2.96
query34	17.22	4.45	4.42
query35	4.47	4.49	4.57
query36	0.65	0.46	0.47
query37	0.18	0.16	0.15
query38	0.17	0.14	0.14
query39	0.04	0.03	0.03
query40	0.18	0.15	0.17
query41	0.10	0.05	0.05
query42	0.06	0.05	0.04
query43	0.05	0.04	0.04
Total cold run time: 109.44 s
Total hot run time: 30.42 s

@zddr zddr requested a review from 924060929 June 27, 2024 09:55
@zddr
Copy link
Copy Markdown
Contributor Author

zddr commented Jun 27, 2024

run buildall

@github-actions github-actions bot removed the approved Indicates a PR has been approved by one committer. label Jun 27, 2024
@doris-robot
Copy link
Copy Markdown

doris-robot commented Jun 27, 2024

TPC-H: Total hot run time: 39494 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 9a574485ec352fcf051dc844d58836678ea250e8, data reload: false

------ Round 1 ----------------------------------
q1	17618	4314	4215	4215
q2	2017	189	198	189
q3	10437	1201	1042	1042
q4	10188	777	825	777
q5	7519	2618	2627	2618
q6	216	132	132	132
q7	956	595	591	591
q8	9224	2051	2020	2020
q9	8779	6476	6456	6456
q10	9048	3722	3721	3721
q11	475	240	234	234
q12	452	228	228	228
q13	17761	2962	3008	2962
q14	261	225	215	215
q15	515	474	464	464
q16	498	372	370	370
q17	949	677	653	653
q18	8047	7462	7362	7362
q19	6111	1510	1459	1459
q20	635	312	327	312
q21	4953	3135	3348	3135
q22	389	342	339	339
Total cold run time: 117048 ms
Total hot run time: 39494 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4382	4220	4198	4198
q2	380	264	269	264
q3	2972	2785	2851	2785
q4	2007	1823	1653	1653
q5	5607	5516	5456	5456
q6	229	135	124	124
q7	2187	1868	1911	1868
q8	3230	3370	3390	3370
q9	8690	8683	8829	8683
q10	4067	3913	3671	3671
q11	594	503	524	503
q12	811	678	657	657
q13	17140	3174	3205	3174
q14	303	274	277	274
q15	530	478	485	478
q16	488	440	438	438
q17	1789	1517	1530	1517
q18	8189	7966	7771	7771
q19	1763	1462	1510	1462
q20	2202	1869	1862	1862
q21	5215	5070	4846	4846
q22	629	533	557	533
Total cold run time: 73404 ms
Total hot run time: 55587 ms

@doris-robot
Copy link
Copy Markdown

doris-robot commented Jun 27, 2024

TPC-DS: Total hot run time: 174244 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 9a574485ec352fcf051dc844d58836678ea250e8, data reload: false

query1	931	392	374	374
query2	6444	2464	2366	2366
query3	6626	211	213	211
query4	20789	17565	17384	17384
query5	3675	473	462	462
query6	257	174	162	162
query7	4588	310	302	302
query8	305	284	285	284
query9	8438	2323	2310	2310
query10	564	308	274	274
query11	10626	10309	10150	10150
query12	118	83	80	80
query13	1638	363	356	356
query14	10103	6796	7641	6796
query15	280	208	189	189
query16	8000	265	266	265
query17	1905	528	522	522
query18	2084	270	263	263
query19	190	144	148	144
query20	84	83	80	80
query21	207	134	122	122
query22	4575	4274	4053	4053
query23	33898	33609	33779	33609
query24	10711	2898	2975	2898
query25	576	362	382	362
query26	704	158	152	152
query27	2278	326	327	326
query28	6090	2095	2068	2068
query29	871	645	612	612
query30	256	152	156	152
query31	987	764	782	764
query32	94	53	57	53
query33	735	304	289	289
query34	881	485	484	484
query35	787	640	667	640
query36	1156	1007	984	984
query37	157	81	75	75
query38	3009	2791	2927	2791
query39	890	854	828	828
query40	206	125	137	125
query41	55	55	51	51
query42	108	93	97	93
query43	614	575	576	575
query44	1144	746	721	721
query45	207	161	167	161
query46	1078	713	694	694
query47	1913	1808	1793	1793
query48	384	306	294	294
query49	834	398	416	398
query50	767	380	382	380
query51	6865	6758	6817	6758
query52	98	100	89	89
query53	356	292	291	291
query54	900	446	440	440
query55	74	73	72	72
query56	304	255	255	255
query57	1153	1087	1058	1058
query58	231	244	227	227
query59	3411	3416	3204	3204
query60	301	269	287	269
query61	90	87	89	87
query62	604	449	442	442
query63	315	283	281	281
query64	8483	2228	1815	1815
query65	3145	3052	3083	3052
query66	748	318	329	318
query67	15568	14997	15014	14997
query68	4508	535	532	532
query69	612	446	394	394
query70	1195	1116	1084	1084
query71	454	277	273	273
query72	7092	5812	5438	5438
query73	758	325	337	325
query74	5978	5533	5513	5513
query75	3518	2675	2703	2675
query76	2989	929	950	929
query77	625	299	296	296
query78	10516	9867	9946	9867
query79	2338	525	523	523
query80	1000	494	469	469
query81	597	215	221	215
query82	801	105	103	103
query83	267	174	170	170
query84	236	91	85	85
query85	1688	281	332	281
query86	473	315	308	308
query87	3331	3067	3080	3067
query88	4277	2456	2443	2443
query89	481	368	374	368
query90	1760	191	190	190
query91	127	97	100	97
query92	55	51	47	47
query93	2504	491	496	491
query94	1100	186	185	185
query95	391	312	313	312
query96	603	277	277	277
query97	3261	3079	3060	3060
query98	226	190	190	190
query99	1330	861	865	861
Total cold run time: 271554 ms
Total hot run time: 174244 ms

@doris-robot
Copy link
Copy Markdown

doris-robot commented Jun 27, 2024

ClickBench: Total hot run time: 30.64 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 9a574485ec352fcf051dc844d58836678ea250e8, data reload: false

query1	0.04	0.03	0.04
query2	0.08	0.04	0.04
query3	0.22	0.05	0.05
query4	1.68	0.07	0.08
query5	0.50	0.49	0.48
query6	1.14	0.72	0.72
query7	0.03	0.02	0.01
query8	0.05	0.05	0.05
query9	0.56	0.52	0.49
query10	0.56	0.53	0.53
query11	0.15	0.11	0.12
query12	0.15	0.12	0.12
query13	0.60	0.59	0.60
query14	0.81	0.78	0.77
query15	0.83	0.81	0.83
query16	0.36	0.35	0.35
query17	0.98	0.99	1.03
query18	0.23	0.24	0.25
query19	1.89	1.81	1.82
query20	0.01	0.01	0.01
query21	15.43	0.76	0.66
query22	4.18	6.97	1.95
query23	18.29	1.43	1.26
query24	2.09	0.24	0.22
query25	0.15	0.08	0.09
query26	0.26	0.17	0.17
query27	0.08	0.07	0.08
query28	13.19	1.02	1.00
query29	12.59	3.29	3.25
query30	0.25	0.06	0.06
query31	2.86	0.39	0.38
query32	3.28	0.47	0.47
query33	2.89	2.90	2.93
query34	17.08	4.39	4.40
query35	4.49	4.50	4.47
query36	0.65	0.48	0.46
query37	0.17	0.15	0.16
query38	0.15	0.14	0.16
query39	0.05	0.03	0.03
query40	0.18	0.15	0.14
query41	0.08	0.04	0.04
query42	0.05	0.05	0.05
query43	0.04	0.04	0.03
Total cold run time: 109.35 s
Total hot run time: 30.64 s

@zddr
Copy link
Copy Markdown
Contributor Author

zddr commented Jun 28, 2024

run buildall

@doris-robot
Copy link
Copy Markdown

doris-robot commented Jun 28, 2024

TPC-H: Total hot run time: 39817 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 1478d30d0c38ccbc90d90e53680f3bcf69c48817, data reload: false

------ Round 1 ----------------------------------
q1	17798	4435	4377	4377
q2	2555	198	199	198
q3	12066	1150	1121	1121
q4	10561	847	751	751
q5	7846	2642	2625	2625
q6	231	139	139	139
q7	971	633	611	611
q8	9551	2049	2026	2026
q9	9001	6464	6450	6450
q10	8840	3684	3683	3683
q11	462	238	237	237
q12	406	233	227	227
q13	19049	2990	2991	2990
q14	268	234	220	220
q15	519	482	482	482
q16	512	375	372	372
q17	971	623	727	623
q18	7900	7466	7333	7333
q19	7466	1483	1486	1483
q20	654	327	350	327
q21	4868	3207	3927	3207
q22	391	335	335	335
Total cold run time: 122886 ms
Total hot run time: 39817 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4392	4241	4209	4209
q2	373	253	261	253
q3	2996	2779	2708	2708
q4	1849	1619	1570	1570
q5	5250	5293	5231	5231
q6	216	125	126	125
q7	2112	1738	1746	1738
q8	3163	3317	3292	3292
q9	8249	8248	8223	8223
q10	3842	3658	3669	3658
q11	594	471	485	471
q12	777	653	572	572
q13	17257	2981	2984	2981
q14	294	261	280	261
q15	519	473	469	469
q16	479	406	428	406
q17	1761	1480	1473	1473
q18	7576	7639	7450	7450
q19	1724	1613	1558	1558
q20	2031	1783	1766	1766
q21	6552	4669	4696	4669
q22	641	538	545	538
Total cold run time: 72647 ms
Total hot run time: 53621 ms

@doris-robot
Copy link
Copy Markdown

doris-robot commented Jun 28, 2024

TPC-DS: Total hot run time: 170051 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 1478d30d0c38ccbc90d90e53680f3bcf69c48817, data reload: false

query1	927	390	379	379
query2	6467	2430	2355	2355
query3	6654	212	219	212
query4	20601	17511	17267	17267
query5	4247	481	470	470
query6	269	165	179	165
query7	4589	293	298	293
query8	304	285	276	276
query9	8499	2453	2437	2437
query10	616	292	288	288
query11	10522	10183	10138	10138
query12	135	87	82	82
query13	1631	379	359	359
query14	9720	7555	6761	6761
query15	241	194	184	184
query16	7815	257	261	257
query17	1920	533	523	523
query18	1961	272	268	268
query19	193	157	174	157
query20	88	84	81	81
query21	213	132	126	126
query22	4264	4180	3987	3987
query23	33553	32991	33130	32991
query24	12047	2840	2744	2744
query25	662	380	365	365
query26	1783	155	149	149
query27	3040	317	306	306
query28	7852	2124	2117	2117
query29	1172	631	628	628
query30	280	160	154	154
query31	948	744	778	744
query32	94	54	62	54
query33	815	305	298	298
query34	960	467	482	467
query35	747	634	606	606
query36	1067	953	959	953
query37	292	74	73	73
query38	2871	2757	2760	2757
query39	861	802	773	773
query40	285	128	128	128
query41	58	53	54	53
query42	118	104	102	102
query43	585	550	564	550
query44	1293	739	737	737
query45	192	162	157	157
query46	1064	716	787	716
query47	1881	1769	1766	1766
query48	367	303	301	301
query49	1183	412	400	400
query50	773	386	384	384
query51	6804	6782	6714	6714
query52	106	96	91	91
query53	357	295	296	295
query54	978	444	440	440
query55	78	75	74	74
query56	275	261	268	261
query57	1138	1045	1058	1045
query58	245	247	262	247
query59	3514	3282	3318	3282
query60	292	276	281	276
query61	97	93	94	93
query62	673	439	441	439
query63	328	297	285	285
query64	9846	2273	1745	1745
query65	3172	3081	3109	3081
query66	1379	335	327	327
query67	15509	14845	15113	14845
query68	4557	548	559	548
query69	455	301	326	301
query70	1141	1087	1201	1087
query71	403	275	271	271
query72	7158	5727	2740	2740
query73	737	325	321	321
query74	5926	5529	5474	5474
query75	3442	2704	2662	2662
query76	2775	1045	936	936
query77	490	310	323	310
query78	10355	9823	9646	9646
query79	2316	529	529	529
query80	1255	465	477	465
query81	586	222	222	222
query82	767	102	107	102
query83	251	165	170	165
query84	241	88	86	86
query85	2066	276	266	266
query86	506	328	311	311
query87	3273	3089	3129	3089
query88	3756	2376	2367	2367
query89	473	388	390	388
query90	1821	192	189	189
query91	127	99	106	99
query92	65	50	54	50
query93	2323	522	516	516
query94	1291	190	188	188
query95	396	320	310	310
query96	595	273	270	270
query97	3269	3046	3032	3032
query98	220	226	197	197
query99	1174	853	836	836
Total cold run time: 277720 ms
Total hot run time: 170051 ms

@doris-robot
Copy link
Copy Markdown

doris-robot commented Jun 28, 2024

ClickBench: Total hot run time: 30.28 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 1478d30d0c38ccbc90d90e53680f3bcf69c48817, data reload: false

query1	0.04	0.04	0.03
query2	0.08	0.04	0.05
query3	0.22	0.05	0.05
query4	1.67	0.09	0.09
query5	0.51	0.48	0.48
query6	1.14	0.73	0.72
query7	0.02	0.01	0.01
query8	0.05	0.04	0.04
query9	0.54	0.50	0.49
query10	0.55	0.56	0.56
query11	0.16	0.12	0.12
query12	0.16	0.13	0.12
query13	0.59	0.59	0.61
query14	0.81	0.77	0.77
query15	0.84	0.82	0.82
query16	0.37	0.36	0.35
query17	0.96	0.96	0.94
query18	0.21	0.22	0.25
query19	1.75	1.70	1.68
query20	0.02	0.01	0.01
query21	15.48	0.75	0.65
query22	4.22	7.87	1.72
query23	18.28	1.46	1.23
query24	2.24	0.23	0.22
query25	0.15	0.09	0.09
query26	0.26	0.17	0.18
query27	0.08	0.08	0.07
query28	13.21	1.04	1.00
query29	12.60	3.36	3.32
query30	0.24	0.06	0.05
query31	2.89	0.40	0.39
query32	3.24	0.47	0.47
query33	2.85	2.89	2.89
query34	17.05	4.46	4.40
query35	4.51	4.42	4.50
query36	0.65	0.46	0.47
query37	0.19	0.15	0.16
query38	0.15	0.15	0.14
query39	0.04	0.04	0.04
query40	0.17	0.16	0.15
query41	0.09	0.04	0.05
query42	0.05	0.04	0.04
query43	0.04	0.04	0.04
Total cold run time: 109.37 s
Total hot run time: 30.28 s

@zddr
Copy link
Copy Markdown
Contributor Author

zddr commented Jun 28, 2024

run buildall

@doris-robot
Copy link
Copy Markdown

doris-robot commented Jun 28, 2024

TPC-H: Total hot run time: 40444 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f0e2024f91614ad5b9dee9990b3c31b6de80aed9, data reload: false

------ Round 1 ----------------------------------
q1	17612	4368	4282	4282
q2	2012	194	190	190
q3	10452	1256	1175	1175
q4	10189	790	857	790
q5	7482	2642	2628	2628
q6	219	135	135	135
q7	955	606	606	606
q8	9219	2061	2032	2032
q9	9091	6503	6422	6422
q10	8952	3686	3702	3686
q11	441	237	244	237
q12	483	233	237	233
q13	17814	2966	3013	2966
q14	268	233	224	224
q15	519	484	487	484
q16	513	373	375	373
q17	978	632	651	632
q18	7974	7398	7380	7380
q19	7544	1550	1427	1427
q20	672	330	345	330
q21	4913	3867	3987	3867
q22	400	347	345	345
Total cold run time: 118702 ms
Total hot run time: 40444 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4379	4285	4199	4199
q2	370	277	265	265
q3	3041	2887	2988	2887
q4	1928	1751	1676	1676
q5	5515	5586	5483	5483
q6	225	126	131	126
q7	2266	1830	1835	1830
q8	3323	3429	3388	3388
q9	8670	8693	8770	8693
q10	4174	3722	3782	3722
q11	601	502	527	502
q12	823	658	663	658
q13	16234	3189	3167	3167
q14	307	291	288	288
q15	540	474	480	474
q16	487	453	434	434
q17	1848	1496	1513	1496
q18	8174	8023	7859	7859
q19	1866	1589	1599	1589
q20	3183	1851	1836	1836
q21	9334	4926	4911	4911
q22	614	570	561	561
Total cold run time: 77902 ms
Total hot run time: 56044 ms

@doris-robot
Copy link
Copy Markdown

doris-robot commented Jun 28, 2024

TPC-DS: Total hot run time: 175626 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f0e2024f91614ad5b9dee9990b3c31b6de80aed9, data reload: false

query1	942	382	394	382
query2	7652	2302	2312	2302
query3	6634	210	217	210
query4	18932	18022	17586	17586
query5	3739	505	484	484
query6	257	185	169	169
query7	4595	298	296	296
query8	310	298	283	283
query9	8485	2468	2452	2452
query10	564	321	279	279
query11	10755	10279	10089	10089
query12	126	88	87	87
query13	1670	384	372	372
query14	9687	7550	7857	7550
query15	276	194	194	194
query16	7531	278	277	277
query17	1954	569	544	544
query18	1959	281	283	281
query19	204	159	156	156
query20	88	85	85	85
query21	221	125	123	123
query22	4666	4286	4254	4254
query23	33994	33646	33569	33569
query24	11026	2957	2980	2957
query25	628	403	420	403
query26	730	165	160	160
query27	2245	323	339	323
query28	6337	2214	2204	2204
query29	889	654	671	654
query30	273	163	157	157
query31	962	776	787	776
query32	97	54	59	54
query33	796	306	310	306
query34	1040	495	493	493
query35	759	672	657	657
query36	1141	987	971	971
query37	149	77	81	77
query38	2976	2848	2820	2820
query39	924	843	832	832
query40	226	134	132	132
query41	56	54	53	53
query42	128	103	103	103
query43	588	573	546	546
query44	1288	769	747	747
query45	200	170	168	168
query46	1081	729	709	709
query47	1853	1864	1803	1803
query48	390	297	297	297
query49	842	400	413	400
query50	775	403	398	398
query51	6908	6749	6763	6749
query52	100	98	93	93
query53	364	299	314	299
query54	940	445	445	445
query55	73	73	73	73
query56	281	262	261	261
query57	1180	1069	1065	1065
query58	237	233	238	233
query59	3356	3034	3262	3034
query60	286	279	276	276
query61	93	93	98	93
query62	614	430	455	430
query63	320	296	297	296
query64	8840	2262	1730	1730
query65	3190	3129	3097	3097
query66	739	339	330	330
query67	15609	14923	14919	14919
query68	8522	565	567	565
query69	707	448	409	409
query70	1238	1049	1075	1049
query71	533	271	271	271
query72	8367	5757	5409	5409
query73	1977	330	330	330
query74	5903	5605	5580	5580
query75	5072	2621	2671	2621
query76	4632	1023	997	997
query77	767	304	303	303
query78	10498	9974	9805	9805
query79	7859	517	535	517
query80	1581	475	464	464
query81	552	219	221	219
query82	774	108	102	102
query83	308	171	166	166
query84	270	84	83	83
query85	1245	293	315	293
query86	405	302	332	302
query87	3306	3125	3097	3097
query88	4472	2412	2357	2357
query89	489	381	382	381
query90	1903	189	187	187
query91	128	98	101	98
query92	62	54	48	48
query93	5742	520	518	518
query94	1182	192	187	187
query95	406	312	309	309
query96	602	264	269	264
query97	3230	3048	3016	3016
query98	234	200	204	200
query99	1183	832	870	832
Total cold run time: 290919 ms
Total hot run time: 175626 ms

@doris-robot
Copy link
Copy Markdown

doris-robot commented Jun 28, 2024

ClickBench: Total hot run time: 30.25 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit f0e2024f91614ad5b9dee9990b3c31b6de80aed9, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.04	0.04
query3	0.22	0.05	0.04
query4	1.68	0.10	0.11
query5	0.50	0.50	0.49
query6	1.14	0.73	0.73
query7	0.02	0.02	0.01
query8	0.05	0.04	0.05
query9	0.55	0.51	0.50
query10	0.54	0.54	0.54
query11	0.16	0.12	0.12
query12	0.15	0.12	0.12
query13	0.60	0.58	0.60
query14	0.80	0.77	0.80
query15	0.85	0.81	0.81
query16	0.36	0.36	0.36
query17	1.06	0.99	0.97
query18	0.25	0.23	0.25
query19	1.82	1.72	1.67
query20	0.01	0.01	0.01
query21	15.46	0.74	0.65
query22	3.92	7.84	1.72
query23	18.29	1.41	1.25
query24	2.19	0.22	0.21
query25	0.17	0.09	0.08
query26	0.26	0.17	0.18
query27	0.08	0.08	0.08
query28	13.20	1.01	0.99
query29	12.58	3.28	3.28
query30	0.25	0.06	0.06
query31	2.89	0.38	0.39
query32	3.26	0.47	0.47
query33	2.93	2.86	2.87
query34	16.98	4.40	4.45
query35	4.47	4.44	4.51
query36	0.65	0.48	0.46
query37	0.18	0.15	0.14
query38	0.16	0.15	0.15
query39	0.04	0.03	0.03
query40	0.15	0.14	0.15
query41	0.09	0.05	0.05
query42	0.05	0.04	0.05
query43	0.05	0.04	0.04
Total cold run time: 109.18 s
Total hot run time: 30.25 s

@zddr
Copy link
Copy Markdown
Contributor Author

zddr commented Jul 2, 2024

run buildall

@doris-robot
Copy link
Copy Markdown

doris-robot commented Jul 2, 2024

TPC-H: Total hot run time: 39830 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 2144f9057584b043ecdf16b18865f473202ed932, data reload: false

------ Round 1 ----------------------------------
q1	17623	4311	4280	4280
q2	2003	195	187	187
q3	10461	1184	1123	1123
q4	10194	861	818	818
q5	7478	2650	2631	2631
q6	215	137	135	135
q7	956	593	607	593
q8	9228	2064	2054	2054
q9	8715	6462	6478	6462
q10	9007	3688	3705	3688
q11	456	244	237	237
q12	397	233	230	230
q13	17939	2960	2953	2953
q14	268	245	219	219
q15	529	482	490	482
q16	524	373	375	373
q17	964	634	719	634
q18	8034	7439	7367	7367
q19	6803	1506	1469	1469
q20	672	317	329	317
q21	4961	3898	3243	3243
q22	393	335	342	335
Total cold run time: 117820 ms
Total hot run time: 39830 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4362	4220	4248	4220
q2	379	257	272	257
q3	2997	2879	2924	2879
q4	1964	1735	1691	1691
q5	5654	5479	5452	5452
q6	229	131	132	131
q7	2226	1867	1842	1842
q8	3221	3391	3405	3391
q9	8749	8669	8812	8669
q10	4101	3864	3728	3728
q11	601	524	510	510
q12	816	654	641	641
q13	16040	3203	3189	3189
q14	299	278	274	274
q15	540	473	481	473
q16	476	421	432	421
q17	1830	1530	1536	1530
q18	8174	7963	7922	7922
q19	1853	1730	1707	1707
q20	2151	1894	1875	1875
q21	5029	5048	4859	4859
q22	623	573	570	570
Total cold run time: 72314 ms
Total hot run time: 56231 ms

@doris-robot
Copy link
Copy Markdown

doris-robot commented Jul 2, 2024

TPC-DS: Total hot run time: 174902 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 2144f9057584b043ecdf16b18865f473202ed932, data reload: false

query1	907	385	370	370
query2	6459	2385	2458	2385
query3	6634	214	223	214
query4	19245	17291	17426	17291
query5	3700	503	478	478
query6	265	161	162	161
query7	4585	302	295	295
query8	327	293	300	293
query9	8456	2379	2344	2344
query10	570	289	276	276
query11	10681	10214	10051	10051
query12	120	87	85	85
query13	1641	362	371	362
query14	10212	7696	7695	7695
query15	256	188	187	187
query16	7911	271	277	271
query17	1880	572	547	547
query18	2003	284	281	281
query19	194	160	159	159
query20	95	81	84	81
query21	223	132	128	128
query22	4377	4141	4087	4087
query23	33920	33531	33557	33531
query24	10843	2861	2890	2861
query25	602	432	394	394
query26	701	160	166	160
query27	2231	336	324	324
query28	6009	2151	2112	2112
query29	904	651	647	647
query30	243	168	162	162
query31	967	790	773	773
query32	96	57	63	57
query33	665	302	310	302
query34	868	477	483	477
query35	758	656	650	650
query36	1130	995	999	995
query37	144	83	74	74
query38	2973	2858	2875	2858
query39	897	855	840	840
query40	214	135	133	133
query41	57	53	54	53
query42	119	102	103	102
query43	589	562	535	535
query44	1077	726	744	726
query45	200	172	166	166
query46	1071	740	718	718
query47	1874	1785	1775	1775
query48	368	294	301	294
query49	949	422	413	413
query50	768	383	380	380
query51	6824	6768	6775	6768
query52	106	90	90	90
query53	365	292	286	286
query54	882	460	448	448
query55	74	75	72	72
query56	291	267	258	258
query57	1138	1079	1053	1053
query58	243	231	243	231
query59	3496	3180	3145	3145
query60	321	280	306	280
query61	92	116	92	92
query62	592	446	426	426
query63	307	282	279	279
query64	8503	2302	1789	1789
query65	3184	3125	3109	3109
query66	731	327	321	321
query67	15665	15106	15056	15056
query68	8776	549	543	543
query69	747	457	385	385
query70	1371	1076	1126	1076
query71	550	276	278	276
query72	9219	5491	5354	5354
query73	2279	327	330	327
query74	5915	5459	5554	5459
query75	5473	2684	2712	2684
query76	5391	1009	914	914
query77	777	295	288	288
query78	10618	9914	9739	9739
query79	8217	516	508	508
query80	969	505	482	482
query81	546	226	221	221
query82	224	101	105	101
query83	332	173	170	170
query84	278	83	82	82
query85	881	273	266	266
query86	349	315	316	315
query87	3327	3107	3088	3088
query88	4566	2352	2391	2352
query89	514	386	386	386
query90	2010	189	192	189
query91	165	98	102	98
query92	58	49	51	49
query93	6028	510	511	510
query94	1322	190	190	190
query95	403	309	310	309
query96	607	263	265	263
query97	3181	2996	3061	2996
query98	237	197	193	193
query99	1115	852	833	833
Total cold run time: 291237 ms
Total hot run time: 174902 ms

@doris-robot
Copy link
Copy Markdown

doris-robot commented Jul 2, 2024

ClickBench: Total hot run time: 30.7 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 2144f9057584b043ecdf16b18865f473202ed932, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.04	0.04
query3	0.22	0.05	0.05
query4	1.67	0.10	0.09
query5	0.51	0.48	0.49
query6	1.12	0.72	0.72
query7	0.02	0.01	0.02
query8	0.05	0.04	0.05
query9	0.54	0.49	0.49
query10	0.54	0.55	0.55
query11	0.15	0.12	0.12
query12	0.16	0.12	0.12
query13	0.60	0.61	0.59
query14	0.79	0.77	0.77
query15	0.85	0.80	0.81
query16	0.38	0.37	0.34
query17	1.01	1.01	1.01
query18	0.24	0.27	0.25
query19	1.73	1.74	1.68
query20	0.01	0.00	0.00
query21	15.44	0.78	0.66
query22	3.96	7.01	2.02
query23	18.26	1.38	1.26
query24	2.25	0.22	0.23
query25	0.16	0.09	0.09
query26	0.25	0.19	0.18
query27	0.09	0.08	0.08
query28	13.14	1.03	1.02
query29	12.59	3.31	3.27
query30	0.27	0.06	0.05
query31	2.86	0.38	0.39
query32	3.29	0.47	0.47
query33	2.90	2.94	2.86
query34	17.02	4.42	4.43
query35	4.53	4.47	4.48
query36	0.64	0.47	0.50
query37	0.18	0.16	0.16
query38	0.17	0.15	0.14
query39	0.05	0.03	0.04
query40	0.18	0.15	0.14
query41	0.09	0.05	0.05
query42	0.05	0.04	0.06
query43	0.04	0.05	0.04
Total cold run time: 109.12 s
Total hot run time: 30.7 s

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jul 2, 2024
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Jul 2, 2024

PR approved by at least one committer and no changes requested.

@morningman morningman merged commit af31222 into apache:master Jul 2, 2024
zddr added a commit to zddr/incubator-doris that referenced this pull request Jul 2, 2024
@zddr
[fix](auth)support check priv when tvf use resource (apache#36928)
31d2298 
current, follow sql not check priv, we need check `usage priv` of
`test_resource`
```
SELECT * FROM S3 (
    "uri" = "xxx",
    "format" = "orc",
    "resource" = "test_resource",
    "use_path_style" = "true"
);
````
@zddr zddr mentioned this pull request Jul 2, 2024
Merged
morningman pushed a commit that referenced this pull request Jul 2, 2024
@zddr
[fix](auth)support check priv when tvf use resource (#36928) (#37132)
0a1abf1 
pick #36928
dataroaring pushed a commit that referenced this pull request Jul 17, 2024
@zddr @dataroaring
[fix](auth)support check priv when tvf use resource (#36928)
4b7f081 
current, follow sql not check priv, we need check `usage priv` of
`test_resource`
```
SELECT * FROM S3 (
    "uri" = "xxx",
    "format" = "orc",
    "resource" = "test_resource",
    "use_path_style" = "true"
);
````
@gavinchou gavinchou mentioned this pull request Aug 19, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@morningman morningman morningman approved these changes

@924060929 924060929 924060929 approved these changes

+1 more reviewer

@BePPPower BePPPower BePPPower approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/2.1.5-merged dev/3.0.1-merged kind/behavior-changed reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

7 participants

@zddr @doris-robot @morningman @924060929 @BePPPower @dataroaring @morrySnow