[fix](local shuffle) Fix hash shuffle local exchanger by Gabriel39 · Pull Request #40036 · apache/doris

Gabriel39 · 2024-08-28T07:18:05Z

Proposed changes

Local Exchange and Exchange in this plan must have same HASH algorithm on multiple BEs

doris-robot · 2024-08-28T07:18:10Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

Gabriel39 · 2024-08-28T07:18:32Z

run buildall

github-actions · 2024-08-28T07:25:13Z

clang-tidy review says "All clean, LGTM! 👍"

Gabriel39 · 2024-08-28T07:46:48Z

run buildall

doris-robot · 2024-08-28T07:46:54Z

TPC-H: Total hot run time: 37723 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 9ce6ef89cb91e667199d0c452bfccf21778c08b0, data reload: false

------ Round 1 ----------------------------------
q1	17627	4408	4300	4300
q2	2019	185	177	177
q3	11771	959	1150	959
q4	10517	774	742	742
q5	7766	2797	2826	2797
q6	224	140	142	140
q7	966	614	599	599
q8	9521	2061	2053	2053
q9	7477	6502	6538	6502
q10	7010	2221	2279	2221
q11	473	236	256	236
q12	390	225	227	225
q13	18742	3006	3071	3006
q14	276	245	233	233
q15	520	492	489	489
q16	599	524	507	507
q17	987	629	730	629
q18	7313	6816	6857	6816
q19	1386	1077	885	885
q20	701	340	340	340
q21	3992	3115	2846	2846
q22	1109	1042	1021	1021
Total cold run time: 111386 ms
Total hot run time: 37723 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4339	4296	4276	4276
q2	372	270	271	270
q3	2856	2660	2642	2642
q4	1946	1616	1666	1616
q5	5617	5669	5718	5669
q6	230	134	138	134
q7	2242	1845	1781	1781
q8	3322	3418	3419	3418
q9	8747	8795	8803	8795
q10	3519	3370	3370	3370
q11	608	514	515	514
q12	817	682	672	672
q13	15531	3209	3249	3209
q14	324	287	297	287
q15	537	525	498	498
q16	606	576	582	576
q17	1829	1551	1572	1551
q18	8103	7789	7853	7789
q19	1720	1570	1643	1570
q20	2139	1898	1900	1898
q21	5797	5507	5430	5430
q22	1160	1060	1052	1052
Total cold run time: 72361 ms
Total hot run time: 57017 ms

github-actions · 2024-08-28T07:53:02Z

clang-tidy review says "All clean, LGTM! 👍"

doris-robot · 2024-08-28T08:04:57Z

TPC-H: Total hot run time: 38144 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 55a83055e08a67efb00113f7becb07de8ae0fcb3, data reload: false

------ Round 1 ----------------------------------
q1	15206	4633	4316	4316
q2	1240	180	183	180
q3	6598	1122	1057	1057
q4	8074	738	787	738
q5	6387	2866	2767	2767
q6	230	138	137	137
q7	977	621	610	610
q8	7622	2065	2073	2065
q9	6637	6557	6538	6538
q10	2537	2254	2195	2195
q11	369	255	256	255
q12	404	236	230	230
q13	7201	3057	3048	3048
q14	279	236	237	236
q15	535	480	475	475
q16	562	520	522	520
q17	971	702	662	662
q18	7458	6793	6912	6793
q19	1244	992	993	992
q20	614	341	346	341
q21	3943	2984	2999	2984
q22	1126	1025	1005	1005
Total cold run time: 80214 ms
Total hot run time: 38144 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4397	4342	4285	4285
q2	388	275	299	275
q3	2931	2703	2712	2703
q4	1905	1678	1677	1677
q5	5373	5409	5419	5409
q6	224	135	135	135
q7	2089	1762	1757	1757
q8	3219	3377	3397	3377
q9	8456	8455	8435	8435
q10	3419	3259	3211	3211
q11	625	506	519	506
q12	803	614	643	614
q13	6269	3102	3058	3058
q14	312	289	283	283
q15	523	480	481	480
q16	589	567	582	567
q17	1792	1516	1494	1494
q18	7797	7637	7521	7521
q19	1692	1516	1525	1516
q20	2079	1828	1851	1828
q21	5594	5241	5312	5241
q22	1123	1067	1024	1024
Total cold run time: 61599 ms
Total hot run time: 55396 ms

Gabriel39 · 2024-08-28T08:11:58Z

run buildall

github-actions · 2024-08-28T08:19:38Z

clang-tidy review says "All clean, LGTM! 👍"

doris-robot · 2024-08-28T09:03:57Z

TPC-H: Total hot run time: 37923 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 40faeb9e8312766ad58c9468faabdf665f8ced33, data reload: false

------ Round 1 ----------------------------------
q1	17619	4452	4292	4292
q2	2018	184	180	180
q3	11831	994	1083	994
q4	10529	791	740	740
q5	7777	2872	2800	2800
q6	229	140	138	138
q7	946	607	597	597
q8	9323	2074	2126	2074
q9	7265	6541	6546	6541
q10	6988	2191	2237	2191
q11	469	238	244	238
q12	394	220	221	220
q13	17778	3011	3039	3011
q14	287	242	239	239
q15	524	483	483	483
q16	583	498	514	498
q17	994	698	791	698
q18	7495	6894	6799	6799
q19	1379	1001	995	995
q20	685	339	347	339
q21	3909	2907	2850	2850
q22	1122	1006	1016	1006
Total cold run time: 110144 ms
Total hot run time: 37923 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4330	4276	4245	4245
q2	374	277	274	274
q3	2886	2626	2652	2626
q4	1923	1696	1659	1659
q5	5600	5698	5744	5698
q6	237	140	139	139
q7	2275	1958	1829	1829
q8	3352	3416	3444	3416
q9	8776	8809	8774	8774
q10	3577	3419	3358	3358
q11	598	495	499	495
q12	837	703	684	684
q13	15397	3238	3176	3176
q14	326	284	285	284
q15	523	488	503	488
q16	627	606	571	571
q17	1863	1525	1568	1525
q18	8171	7861	7771	7771
q19	1698	1617	1477	1477
q20	2189	1885	1869	1869
q21	5768	5487	5577	5487
q22	1117	1030	1036	1030
Total cold run time: 72444 ms
Total hot run time: 56875 ms

Gabriel39 · 2024-08-28T10:43:01Z

run buildall

github-actions · 2024-08-28T10:49:36Z

clang-tidy review says "All clean, LGTM! 👍"

doris-robot · 2024-08-28T11:01:31Z

TPC-H: Total hot run time: 38558 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e8162c68bbe2093d6beb7754cbd7b3e01d155aa5, data reload: false

------ Round 1 ----------------------------------
q1	17877	4514	4415	4415
q2	2592	182	188	182
q3	10880	1152	1093	1093
q4	10559	741	808	741
q5	8099	2910	2864	2864
q6	233	145	148	145
q7	1006	620	632	620
q8	9589	2068	2076	2068
q9	7032	6537	6524	6524
q10	6995	2240	2301	2240
q11	479	242	246	242
q12	408	227	232	227
q13	17757	3034	3042	3034
q14	285	237	228	228
q15	527	492	496	492
q16	575	502	512	502
q17	984	766	695	695
q18	7358	6886	6958	6886
q19	1412	1041	963	963
q20	710	345	336	336
q21	3941	3034	3116	3034
q22	1106	1027	1033	1027
Total cold run time: 110404 ms
Total hot run time: 38558 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4395	4389	4274	4274
q2	384	272	268	268
q3	2895	2695	2676	2676
q4	1936	1641	1659	1641
q5	5403	5353	5395	5353
q6	219	131	133	131
q7	2126	1768	1721	1721
q8	3197	3418	3361	3361
q9	8428	8481	8428	8428
q10	3442	3244	3191	3191
q11	610	495	498	495
q12	824	619	622	619
q13	10744	3055	3035	3035
q14	302	298	268	268
q15	517	496	481	481
q16	631	554	571	554
q17	1803	1495	1501	1495
q18	7793	7411	7579	7411
q19	1726	1513	1470	1470
q20	2078	1830	1825	1825
q21	5403	5397	5188	5188
q22	1102	1034	1042	1034
Total cold run time: 65958 ms
Total hot run time: 54919 ms

doris-robot · 2024-08-28T11:13:07Z

TPC-DS: Total hot run time: 188696 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e8162c68bbe2093d6beb7754cbd7b3e01d155aa5, data reload: false

query1	908	379	368	368
query2	6451	1982	1914	1914
query3	6654	214	225	214
query4	26783	23222	23255	23222
query5	4148	526	508	508
query6	255	169	163	163
query7	4587	322	312	312
query8	262	218	220	218
query9	8548	2516	2510	2510
query10	449	282	284	282
query11	17755	15065	15019	15019
query12	152	108	104	104
query13	1639	399	366	366
query14	9804	7346	7387	7346
query15	268	166	175	166
query16	8049	471	427	427
query17	1577	572	564	564
query18	2042	291	283	283
query19	344	147	147	147
query20	115	107	112	107
query21	212	101	104	101
query22	4431	4223	4088	4088
query23	33909	33215	33349	33215
query24	11181	2941	2895	2895
query25	619	377	386	377
query26	1147	163	160	160
query27	2302	284	284	284
query28	7343	2140	2123	2123
query29	794	428	420	420
query30	312	158	151	151
query31	991	771	759	759
query32	103	56	59	56
query33	754	294	292	292
query34	966	505	514	505
query35	863	745	742	742
query36	1116	958	951	951
query37	160	93	95	93
query38	4058	3903	3927	3903
query39	1446	1388	1391	1388
query40	201	116	118	116
query41	47	50	48	48
query42	116	101	101	101
query43	520	482	479	479
query44	1250	768	766	766
query45	199	164	172	164
query46	1116	799	758	758
query47	1856	1824	1791	1791
query48	380	314	303	303
query49	1088	440	434	434
query50	825	409	418	409
query51	7329	7093	7038	7038
query52	98	87	89	87
query53	260	191	183	183
query54	943	474	465	465
query55	85	80	80	80
query56	316	255	250	250
query57	1155	1058	1037	1037
query58	243	217	231	217
query59	2990	2891	2816	2816
query60	300	270	269	269
query61	105	100	118	100
query62	851	629	663	629
query63	220	191	186	186
query64	4176	745	663	663
query65	3248	3139	3140	3139
query66	1405	342	338	338
query67	15437	15295	15072	15072
query68	3558	586	589	586
query69	397	278	289	278
query70	1194	1147	1183	1147
query71	356	280	280	280
query72	6437	3987	4066	3987
query73	750	336	339	336
query74	9127	8868	8774	8774
query75	3431	2707	2683	2683
query76	2230	1052	1009	1009
query77	499	331	328	328
query78	9892	9147	9169	9147
query79	1042	553	560	553
query80	695	520	535	520
query81	456	232	238	232
query82	247	146	149	146
query83	174	157	159	157
query84	229	82	78	78
query85	686	296	293	293
query86	302	313	294	294
query87	4427	4307	4278	4278
query88	2930	2370	2378	2370
query89	389	294	289	289
query90	1910	202	200	200
query91	127	114	114	114
query92	68	115	51	51
query93	1045	542	547	542
query94	778	293	295	293
query95	360	271	271	271
query96	588	268	278	268
query97	3195	3051	3089	3051
query98	226	210	203	203
query99	1473	1257	1308	1257
Total cold run time: 279474 ms
Total hot run time: 188696 ms

doris-robot · 2024-08-28T11:18:38Z

ClickBench: Total hot run time: 32.75 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e8162c68bbe2093d6beb7754cbd7b3e01d155aa5, data reload: false

query1	0.04	0.04	0.04
query2	0.08	0.04	0.04
query3	0.22	0.05	0.06
query4	1.68	0.08	0.09
query5	0.51	0.50	0.50
query6	1.12	0.73	0.73
query7	0.02	0.01	0.02
query8	0.05	0.04	0.05
query9	0.55	0.49	0.48
query10	0.56	0.54	0.53
query11	0.15	0.12	0.11
query12	0.15	0.12	0.12
query13	0.62	0.59	0.59
query14	2.04	2.10	2.02
query15	0.84	0.83	0.83
query16	0.37	0.40	0.37
query17	1.03	0.96	1.05
query18	0.21	0.21	0.21
query19	1.89	1.75	1.74
query20	0.01	0.01	0.01
query21	15.42	0.67	0.67
query22	3.64	6.64	2.64
query23	18.29	1.38	1.31
query24	2.17	0.23	0.22
query25	0.15	0.08	0.08
query26	0.27	0.17	0.17
query27	0.08	0.07	0.08
query28	13.20	1.04	1.01
query29	12.62	3.38	3.30
query30	0.24	0.06	0.05
query31	2.90	0.40	0.40
query32	3.23	0.48	0.48
query33	3.00	3.02	3.03
query34	17.08	4.41	4.37
query35	4.48	4.42	4.42
query36	0.66	0.46	0.49
query37	0.18	0.16	0.16
query38	0.15	0.15	0.15
query39	0.04	0.04	0.04
query40	0.16	0.14	0.13
query41	0.10	0.05	0.05
query42	0.06	0.05	0.06
query43	0.04	0.04	0.04
Total cold run time: 110.3 s
Total hot run time: 32.75 s

Gabriel39 · 2024-08-28T12:46:27Z

run buildall

github-actions · 2024-08-28T12:50:51Z

clang-tidy review says "All clean, LGTM! 👍"

github-actions · 2024-08-28T12:53:03Z

clang-tidy review says "All clean, LGTM! 👍"

Gabriel39 · 2024-08-28T13:51:06Z

run buildall

github-actions · 2024-08-28T13:58:35Z

clang-tidy review says "All clean, LGTM! 👍"

doris-robot · 2024-08-28T14:17:54Z

TPC-H: Total hot run time: 38982 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ac1b45520df8a87876e51d0b9621bb1bd6064777, data reload: false

------ Round 1 ----------------------------------
q1	17611	4795	4885	4795
q2	2022	175	186	175
q3	10547	1201	1131	1131
q4	10265	706	785	706
q5	8607	2904	2843	2843
q6	233	142	153	142
q7	990	641	607	607
q8	9635	2133	2101	2101
q9	7269	6664	6698	6664
q10	7365	2249	2248	2248
q11	475	242	248	242
q12	393	225	221	221
q13	17777	3008	3030	3008
q14	279	240	236	236
q15	526	483	494	483
q16	616	523	547	523
q17	986	708	720	708
q18	7394	6747	6838	6747
q19	1390	1089	1073	1073
q20	721	325	338	325
q21	4058	3043	2982	2982
q22	1133	1032	1022	1022
Total cold run time: 110292 ms
Total hot run time: 38982 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4399	4302	4255	4255
q2	374	290	269	269
q3	2887	2709	2662	2662
q4	1917	1696	1631	1631
q5	5422	5403	5429	5403
q6	223	130	128	128
q7	2124	1736	1766	1736
q8	3213	3342	3364	3342
q9	8478	8464	8514	8464
q10	3453	3195	3200	3195
q11	596	495	477	477
q12	798	659	647	647
q13	11005	3046	3038	3038
q14	301	271	277	271
q15	524	482	492	482
q16	624	564	557	557
q17	1778	1485	1493	1485
q18	7767	7368	7489	7368
q19	1665	1363	1532	1363
q20	2069	1825	1776	1776
q21	5520	5247	5343	5247
q22	1127	1033	1044	1033
Total cold run time: 66264 ms
Total hot run time: 54829 ms

doris-robot · 2024-08-28T14:29:46Z

TPC-DS: Total hot run time: 189608 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ac1b45520df8a87876e51d0b9621bb1bd6064777, data reload: false

query1	929	374	367	367
query2	6480	2061	2027	2027
query3	6643	212	249	212
query4	29263	23141	23203	23141
query5	4190	505	502	502
query6	256	165	187	165
query7	4576	297	297	297
query8	260	225	216	216
query9	8432	2518	2506	2506
query10	432	258	260	258
query11	16525	15002	15057	15002
query12	157	104	97	97
query13	1630	373	379	373
query14	9457	7383	6924	6924
query15	256	173	178	173
query16	8128	506	510	506
query17	1650	575	557	557
query18	2122	314	301	301
query19	336	157	152	152
query20	118	114	116	114
query21	211	108	104	104
query22	4642	4313	4428	4313
query23	34146	34946	33791	33791
query24	11598	2941	2948	2941
query25	557	413	418	413
query26	1189	159	162	159
query27	2793	290	280	280
query28	7610	2164	2127	2127
query29	669	432	424	424
query30	305	160	153	153
query31	1002	789	794	789
query32	102	63	60	60
query33	781	312	300	300
query34	939	490	483	483
query35	893	732	719	719
query36	1112	918	906	906
query37	213	106	98	98
query38	3977	3867	3831	3831
query39	1444	1491	1410	1410
query40	286	119	119	119
query41	54	49	49	49
query42	121	103	100	100
query43	511	484	491	484
query44	1263	766	762	762
query45	196	168	170	168
query46	1121	769	761	761
query47	1868	1801	1788	1788
query48	380	301	302	301
query49	1165	450	426	426
query50	811	409	415	409
query51	7165	7090	7035	7035
query52	101	91	91	91
query53	257	190	185	185
query54	929	473	470	470
query55	81	82	80	80
query56	280	259	255	255
query57	1190	1078	1062	1062
query58	254	229	231	229
query59	2989	2811	2899	2811
query60	310	282	284	282
query61	126	122	118	118
query62	838	693	677	677
query63	218	185	185	185
query64	5225	835	654	654
query65	3211	3188	3167	3167
query66	1259	336	341	336
query67	15681	15464	15221	15221
query68	4842	555	572	555
query69	511	310	270	270
query70	1158	1121	1140	1121
query71	446	277	274	274
query72	7178	4083	3997	3997
query73	770	335	360	335
query74	9135	8739	8834	8739
query75	3984	2677	2674	2674
query76	3183	1072	955	955
query77	691	318	321	318
query78	10413	10798	9639	9639
query79	1744	540	558	540
query80	912	512	495	495
query81	594	239	241	239
query82	306	156	146	146
query83	342	153	155	153
query84	288	82	74	74
query85	722	302	278	278
query86	452	271	286	271
query87	4478	4281	4314	4281
query88	3280	2412	2330	2330
query89	398	293	292	292
query90	1926	204	196	196
query91	126	103	104	103
query92	64	64	55	55
query93	1062	535	531	531
query94	898	286	324	286
query95	360	261	264	261
query96	597	273	270	270
query97	3242	3061	3059	3059
query98	224	206	204	204
query99	1551	1264	1251	1251
Total cold run time: 289790 ms
Total hot run time: 189608 ms

doris-robot · 2024-08-28T14:35:22Z

ClickBench: Total hot run time: 32.43 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit ac1b45520df8a87876e51d0b9621bb1bd6064777, data reload: false

query1	0.04	0.04	0.04
query2	0.08	0.04	0.04
query3	0.23	0.06	0.05
query4	1.67	0.08	0.08
query5	0.50	0.50	0.49
query6	1.14	0.73	0.72
query7	0.02	0.02	0.02
query8	0.05	0.04	0.04
query9	0.54	0.49	0.49
query10	0.54	0.53	0.55
query11	0.15	0.12	0.11
query12	0.15	0.12	0.12
query13	0.60	0.58	0.58
query14	2.09	2.09	2.04
query15	0.89	0.83	0.81
query16	0.36	0.37	0.38
query17	0.96	0.96	1.01
query18	0.20	0.20	0.20
query19	1.79	1.73	1.73
query20	0.01	0.01	0.01
query21	15.40	0.67	0.67
query22	4.57	6.32	2.33
query23	18.25	1.36	1.25
query24	2.14	0.21	0.22
query25	0.14	0.08	0.08
query26	0.27	0.18	0.17
query27	0.08	0.08	0.07
query28	13.27	1.02	1.00
query29	12.62	3.35	3.38
query30	0.24	0.06	0.06
query31	2.87	0.40	0.38
query32	3.25	0.49	0.47
query33	2.96	3.02	3.01
query34	17.18	4.39	4.47
query35	4.51	4.49	4.50
query36	0.66	0.52	0.47
query37	0.19	0.16	0.16
query38	0.16	0.16	0.15
query39	0.05	0.04	0.04
query40	0.15	0.12	0.12
query41	0.09	0.05	0.05
query42	0.06	0.04	0.04
query43	0.04	0.04	0.04
Total cold run time: 111.16 s
Total hot run time: 32.43 s

Gabriel39 · 2024-08-29T02:55:45Z

run buildall

github-actions · 2024-08-29T02:58:29Z

PR approved by at least one committer and no changes requested.

github-actions · 2024-08-29T02:58:31Z

PR approved by anyone and no changes requested.

github-actions · 2024-08-29T03:02:33Z

clang-tidy review says "All clean, LGTM! 👍"

doris-robot · 2024-08-29T03:32:34Z

TPC-H: Total hot run time: 38629 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 726f15a8214097abc4aa4cff0a267b71bfc72e09, data reload: false

------ Round 1 ----------------------------------
q1	17891	4967	4395	4395
q2	2683	181	174	174
q3	11575	1120	1081	1081
q4	10586	803	715	715
q5	7774	2898	2893	2893
q6	235	139	138	138
q7	984	639	638	638
q8	9648	2054	2078	2054
q9	7184	6545	6589	6545
q10	7002	2220	2225	2220
q11	454	235	249	235
q12	398	229	229	229
q13	17778	3014	3056	3014
q14	283	246	233	233
q15	529	485	492	485
q16	578	518	514	514
q17	995	713	713	713
q18	7327	6813	6956	6813
q19	1376	1196	1051	1051
q20	692	341	333	333
q21	4019	3122	3169	3122
q22	1115	1060	1034	1034
Total cold run time: 111106 ms
Total hot run time: 38629 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4327	4296	4263	4263
q2	385	279	268	268
q3	2888	2596	2666	2596
q4	1944	1697	1693	1693
q5	5437	5395	5382	5382
q6	221	129	131	129
q7	2104	1723	1759	1723
q8	3183	3363	3341	3341
q9	8444	8395	8386	8386
q10	3442	3242	3196	3196
q11	623	500	497	497
q12	812	613	610	610
q13	12179	3059	3026	3026
q14	315	288	284	284
q15	546	475	488	475
q16	628	577	543	543
q17	1800	1496	1502	1496
q18	7771	7342	7362	7342
q19	1667	1597	1501	1501
q20	2055	1824	1817	1817
q21	5494	5180	5304	5180
q22	1115	1062	1018	1018
Total cold run time: 67380 ms
Total hot run time: 54766 ms

doris-robot · 2024-08-29T03:44:18Z

TPC-DS: Total hot run time: 189049 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 726f15a8214097abc4aa4cff0a267b71bfc72e09, data reload: false

query1	904	375	364	364
query2	6455	2023	1854	1854
query3	6646	214	244	214
query4	34208	23326	23507	23326
query5	4166	501	502	501
query6	253	171	162	162
query7	4581	300	304	300
query8	250	199	209	199
query9	8532	2479	2489	2479
query10	443	273	269	269
query11	17633	14949	15289	14949
query12	147	97	98	97
query13	1632	385	384	384
query14	9735	7699	7359	7359
query15	273	172	181	172
query16	7403	447	473	447
query17	1577	583	559	559
query18	1727	316	285	285
query19	275	148	144	144
query20	125	113	111	111
query21	208	103	103	103
query22	4395	4293	4243	4243
query23	33768	33370	33507	33370
query24	11145	2922	2865	2865
query25	624	380	385	380
query26	1170	172	160	160
query27	2292	283	279	279
query28	7064	2123	2097	2097
query29	782	421	404	404
query30	306	155	148	148
query31	984	802	808	802
query32	96	57	59	57
query33	785	286	285	285
query34	968	474	499	474
query35	880	739	716	716
query36	1097	958	955	955
query37	166	92	99	92
query38	3974	3812	3859	3812
query39	1458	1406	1394	1394
query40	207	119	118	118
query41	48	53	46	46
query42	118	96	102	96
query43	508	477	470	470
query44	1306	769	751	751
query45	198	168	172	168
query46	1114	725	770	725
query47	1877	1820	1780	1780
query48	371	305	298	298
query49	1075	472	448	448
query50	821	450	424	424
query51	7144	7081	6942	6942
query52	100	87	92	87
query53	255	189	178	178
query54	924	462	455	455
query55	81	75	80	75
query56	263	252	247	247
query57	1188	1077	1052	1052
query58	237	220	244	220
query59	3009	2827	2638	2638
query60	306	280	267	267
query61	100	102	103	102
query62	871	648	676	648
query63	225	186	183	183
query64	4483	686	661	661
query65	3294	3204	3213	3204
query66	1390	334	331	331
query67	15502	15541	15360	15360
query68	3176	606	600	600
query69	397	283	281	281
query70	1194	1066	1125	1066
query71	337	274	278	274
query72	6414	4120	4036	4036
query73	754	331	336	331
query74	9138	8773	8847	8773
query75	3411	2675	2710	2675
query76	1877	1006	984	984
query77	501	327	325	325
query78	10524	9908	9635	9635
query79	1079	555	557	555
query80	698	518	515	515
query81	502	240	234	234
query82	249	150	144	144
query83	174	156	153	153
query84	238	73	79	73
query85	880	289	284	284
query86	318	290	298	290
query87	4398	4148	4172	4148
query88	2956	2364	2362	2362
query89	401	297	368	297
query90	1955	201	198	198
query91	129	105	106	105
query92	64	55	52	52
query93	1035	571	552	552
query94	723	290	292	290
query95	361	267	271	267
query96	598	271	283	271
query97	3193	3043	3070	3043
query98	232	207	207	207
query99	1523	1244	1253	1244
Total cold run time: 285571 ms
Total hot run time: 189049 ms

doris-robot · 2024-08-29T03:49:51Z

ClickBench: Total hot run time: 32.35 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 726f15a8214097abc4aa4cff0a267b71bfc72e09, data reload: false

query1	0.05	0.04	0.04
query2	0.08	0.04	0.04
query3	0.22	0.05	0.05
query4	1.68	0.07	0.06
query5	0.51	0.50	0.49
query6	1.13	0.74	0.73
query7	0.02	0.02	0.01
query8	0.05	0.05	0.04
query9	0.55	0.50	0.49
query10	0.55	0.55	0.54
query11	0.16	0.11	0.12
query12	0.15	0.13	0.12
query13	0.63	0.59	0.58
query14	2.05	2.02	2.07
query15	0.84	0.82	0.82
query16	0.37	0.39	0.38
query17	1.03	1.02	0.99
query18	0.21	0.20	0.20
query19	1.85	1.76	1.76
query20	0.01	0.01	0.01
query21	15.41	0.68	0.67
query22	4.39	7.00	2.24
query23	18.28	1.40	1.23
query24	2.08	0.23	0.23
query25	0.16	0.07	0.07
query26	0.27	0.18	0.17
query27	0.08	0.09	0.07
query28	13.22	1.04	1.00
query29	12.63	3.32	3.35
query30	0.24	0.05	0.05
query31	2.88	0.41	0.38
query32	3.25	0.49	0.48
query33	3.00	3.02	2.97
query34	17.20	4.43	4.43
query35	4.55	4.50	4.50
query36	0.65	0.47	0.47
query37	0.18	0.16	0.15
query38	0.16	0.15	0.15
query39	0.05	0.03	0.03
query40	0.15	0.13	0.12
query41	0.09	0.05	0.05
query42	0.06	0.05	0.05
query43	0.04	0.05	0.04
Total cold run time: 111.16 s
Total hot run time: 32.35 s

yiguolei · 2024-08-29T06:05:28Z

run buildall

github-actions · 2024-08-29T06:16:36Z

clang-tidy review says "All clean, LGTM! 👍"

doris-robot · 2024-08-29T06:45:56Z

TPC-H: Total hot run time: 38487 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d4a9a7f2e1ffcb3b9bc7dd1d65fa1ac897ed2395, data reload: false

------ Round 1 ----------------------------------
q1	17601	4510	4357	4357
q2	2008	184	173	173
q3	11645	1042	1126	1042
q4	10512	854	775	775
q5	7767	2833	2851	2833
q6	231	143	148	143
q7	975	613	614	613
q8	9487	2080	2093	2080
q9	7254	6563	6591	6563
q10	7013	2200	2212	2200
q11	440	249	239	239
q12	405	223	236	223
q13	18362	3046	3031	3031
q14	286	234	232	232
q15	524	489	498	489
q16	597	515	509	509
q17	997	647	702	647
q18	7328	6979	6951	6951
q19	1379	1129	1039	1039
q20	674	339	338	338
q21	3887	2995	3081	2995
q22	1134	1021	1015	1015
Total cold run time: 110506 ms
Total hot run time: 38487 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4385	4285	4301	4285
q2	388	276	283	276
q3	2881	2670	2661	2661
q4	1963	1693	1667	1667
q5	5626	5669	5688	5669
q6	232	142	136	136
q7	2254	1834	1778	1778
q8	3330	3411	3483	3411
q9	8861	8838	8794	8794
q10	3593	3367	3350	3350
q11	620	515	518	515
q12	825	661	667	661
q13	13044	3193	3197	3193
q14	324	285	291	285
q15	539	500	487	487
q16	633	573	588	573
q17	1853	1568	1555	1555
q18	8153	7907	7800	7800
q19	1742	1605	1538	1538
q20	2129	1956	1898	1898
q21	5790	5559	5542	5542
q22	1133	1057	1021	1021
Total cold run time: 70298 ms
Total hot run time: 57095 ms

doris-robot · 2024-08-29T06:58:06Z

TPC-DS: Total hot run time: 192621 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d4a9a7f2e1ffcb3b9bc7dd1d65fa1ac897ed2395, data reload: false

query1	1259	865	874	865
query2	6324	1871	1911	1871
query3	10609	3875	3802	3802
query4	59855	23839	23069	23069
query5	5438	499	494	494
query6	399	165	166	165
query7	5771	297	293	293
query8	282	222	197	197
query9	9184	2478	2482	2478
query10	484	268	247	247
query11	18178	14977	15213	14977
query12	166	110	109	109
query13	1588	390	368	368
query14	11231	7341	7309	7309
query15	246	172	176	172
query16	7539	473	485	473
query17	1110	570	566	566
query18	1941	295	290	290
query19	287	155	149	149
query20	122	112	119	112
query21	211	106	102	102
query22	4760	4575	4432	4432
query23	34350	33676	33873	33676
query24	5971	2885	2885	2885
query25	530	372	383	372
query26	696	155	160	155
query27	1801	273	282	273
query28	3824	2103	2091	2091
query29	684	400	403	400
query30	243	154	149	149
query31	932	746	795	746
query32	83	52	58	52
query33	435	293	295	293
query34	884	480	479	479
query35	850	714	742	714
query36	1083	942	933	933
query37	151	90	92	90
query38	3933	3827	3991	3827
query39	1445	1392	1409	1392
query40	198	113	116	113
query41	47	49	44	44
query42	113	95	96	95
query43	512	451	471	451
query44	1083	742	742	742
query45	201	168	165	165
query46	1083	751	737	737
query47	1948	1852	1808	1808
query48	368	291	295	291
query49	757	413	426	413
query50	823	422	425	422
query51	7233	7043	7026	7026
query52	102	87	88	87
query53	253	177	179	177
query54	558	452	445	445
query55	78	76	74	74
query56	263	245	246	245
query57	1178	1048	1099	1048
query58	223	237	227	227
query59	2960	2795	2920	2795
query60	295	264	263	263
query61	102	99	96	96
query62	740	639	674	639
query63	212	181	187	181
query64	2816	669	685	669
query65	3211	3164	3140	3140
query66	613	344	353	344
query67	15412	15379	15245	15245
query68	2932	584	597	584
query69	386	282	274	274
query70	1174	1121	1121	1121
query71	330	280	271	271
query72	5306	4066	4018	4018
query73	751	327	339	327
query74	9091	8747	8775	8747
query75	3357	2697	2710	2697
query76	1392	928	963	928
query77	546	314	314	314
query78	9775	8971	9131	8971
query79	1021	537	531	531
query80	695	504	504	504
query81	461	236	236	236
query82	248	148	156	148
query83	232	146	146	146
query84	246	77	74	74
query85	671	285	285	285
query86	309	305	309	305
query87	4320	4244	4296	4244
query88	3238	2336	2323	2323
query89	383	281	285	281
query90	1965	194	191	191
query91	122	98	99	98
query92	59	51	52	51
query93	1043	539	544	539
query94	651	305	297	297
query95	330	263	266	263
query96	593	270	269	269
query97	3248	3111	3067	3067
query98	217	199	200	199
query99	1531	1272	1269	1269
Total cold run time: 304884 ms
Total hot run time: 192621 ms

doris-robot · 2024-08-29T07:03:43Z

ClickBench: Total hot run time: 32.16 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit d4a9a7f2e1ffcb3b9bc7dd1d65fa1ac897ed2395, data reload: false

query1	0.04	0.04	0.04
query2	0.08	0.04	0.04
query3	0.23	0.04	0.05
query4	1.67	0.08	0.08
query5	0.52	0.49	0.49
query6	1.14	0.73	0.72
query7	0.02	0.02	0.01
query8	0.05	0.04	0.04
query9	0.55	0.48	0.50
query10	0.54	0.55	0.53
query11	0.15	0.12	0.11
query12	0.15	0.12	0.13
query13	0.62	0.59	0.58
query14	2.10	2.06	2.08
query15	0.85	0.82	0.82
query16	0.40	0.36	0.37
query17	1.05	1.06	1.10
query18	0.21	0.20	0.20
query19	1.96	1.84	1.81
query20	0.01	0.00	0.01
query21	15.73	0.68	0.67
query22	4.26	6.78	2.06
query23	18.26	1.42	1.25
query24	2.07	0.25	0.23
query25	0.15	0.08	0.08
query26	0.27	0.18	0.18
query27	0.08	0.08	0.08
query28	13.24	1.02	1.00
query29	12.59	3.31	3.26
query30	0.25	0.05	0.06
query31	2.88	0.39	0.40
query32	3.24	0.49	0.48
query33	2.96	3.03	3.00
query34	17.11	4.36	4.33
query35	4.47	4.46	4.41
query36	0.67	0.48	0.47
query37	0.20	0.16	0.16
query38	0.16	0.16	0.16
query39	0.05	0.04	0.04
query40	0.16	0.13	0.13
query41	0.10	0.05	0.05
query42	0.06	0.05	0.05
query43	0.05	0.04	0.04
Total cold run time: 111.35 s
Total hot run time: 32.16 s

## Proposed changes pick #40036

Gabriel39 mentioned this pull request Aug 28, 2024

[cherry-pick](branch-2.1) Improve local shuffle strategy #40038

Closed

yiguolei reviewed Aug 28, 2024

View reviewed changes

Comment thread be/src/pipeline/pipeline_fragment_context.cpp

yiguolei reviewed Aug 29, 2024

View reviewed changes

Comment thread be/src/pipeline/exec/analytic_sink_operator.h

Gabriel39 force-pushed the fix_0828_1 branch from ac1b455 to 726f15a Compare August 29, 2024 02:54

yiguolei approved these changes Aug 29, 2024

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Aug 29, 2024

github-actions bot added the reviewed label Aug 29, 2024

update

d4a9a7f

yiguolei force-pushed the fix_0828_1 branch from 726f15a to d4a9a7f Compare August 29, 2024 06:05

Gabriel39 mentioned this pull request Aug 29, 2024

[fix](local shuffle) Fix hash shuffle local exchanger #40101

Merged

Mryange approved these changes Aug 29, 2024

View reviewed changes

Gabriel39 merged commit 56fca53 into apache:master Aug 29, 2024

yiguolei pushed a commit that referenced this pull request Aug 29, 2024

[fix](local shuffle) Fix hash shuffle local exchanger (#40101)

841565a

## Proposed changes pick #40036

dataroaring added dev/3.0.x-conflict dev/3.0.2-merged labels Sep 3, 2024

dataroaring pushed a commit that referenced this pull request Sep 3, 2024

[fix](local shuffle) Fix hash shuffle local exchanger (#40036)

d117ed7

Gabriel39 removed the dev/3.0.x-conflict label Sep 14, 2024

wm1581066 added the dev/2.1.6-merged label Sep 25, 2024

Conversation

Gabriel39 commented Aug 28, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Proposed changes

Uh oh!

doris-robot commented Aug 28, 2024

Uh oh!

Gabriel39 commented Aug 28, 2024

Uh oh!

github-actions bot commented Aug 28, 2024

Uh oh!

Uh oh!

Gabriel39 commented Aug 28, 2024

Uh oh!

doris-robot commented Aug 28, 2024

Uh oh!

github-actions bot commented Aug 28, 2024

Uh oh!

doris-robot commented Aug 28, 2024

Uh oh!

Gabriel39 commented Aug 28, 2024

Uh oh!

github-actions bot commented Aug 28, 2024

Uh oh!

doris-robot commented Aug 28, 2024

Uh oh!

Gabriel39 commented Aug 28, 2024

Uh oh!

github-actions bot commented Aug 28, 2024

Uh oh!

doris-robot commented Aug 28, 2024

Uh oh!

doris-robot commented Aug 28, 2024

Uh oh!

doris-robot commented Aug 28, 2024

Uh oh!

Gabriel39 commented Aug 28, 2024

Uh oh!

github-actions bot commented Aug 28, 2024

Uh oh!

github-actions bot commented Aug 28, 2024

Uh oh!

Gabriel39 commented Aug 28, 2024

Uh oh!

github-actions bot commented Aug 28, 2024

Uh oh!

doris-robot commented Aug 28, 2024

Uh oh!

doris-robot commented Aug 28, 2024

Uh oh!

doris-robot commented Aug 28, 2024

Uh oh!

Uh oh!

Gabriel39 commented Aug 29, 2024

Uh oh!

github-actions bot commented Aug 29, 2024

Uh oh!

github-actions bot commented Aug 29, 2024

Uh oh!

github-actions bot commented Aug 29, 2024

Uh oh!

doris-robot commented Aug 29, 2024

Uh oh!

doris-robot commented Aug 29, 2024

Uh oh!

doris-robot commented Aug 29, 2024

Uh oh!

yiguolei commented Aug 29, 2024

Uh oh!

github-actions bot commented Aug 29, 2024

Uh oh!

doris-robot commented Aug 29, 2024

Uh oh!

doris-robot commented Aug 29, 2024

Uh oh!

doris-robot commented Aug 29, 2024

Uh oh!

Reviewers

Assignees

Labels

Gabriel39 commented Aug 28, 2024 •

edited

Loading