Skip to content

[fix](cloud) fix alter consume offset can not work in cloud mode#39159

Merged
liaoxin01 merged 1 commit intoapache:masterfrom
sollhui:cloud_fix_alter_offset
Aug 26, 2024
Merged

[fix](cloud) fix alter consume offset can not work in cloud mode#39159
liaoxin01 merged 1 commit intoapache:masterfrom
sollhui:cloud_fix_alter_offset

Conversation

@sollhui
Copy link
Copy Markdown
Contributor

@sollhui sollhui commented Aug 9, 2024

In cloud mode, if update consume offset, it will not update progress in meta service, which causes alter consume offset can not work for progress will regularly update with persistent progress values in meta service.

@doris-robot
Copy link
Copy Markdown

doris-robot commented Aug 9, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@sollhui
Copy link
Copy Markdown
Contributor Author

sollhui commented Aug 9, 2024

run buildall

@sollhui sollhui marked this pull request as draft August 9, 2024 09:23
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Aug 9, 2024

clang-tidy review says "All clean, LGTM! 👍"

@sollhui
Copy link
Copy Markdown
Contributor Author

sollhui commented Aug 9, 2024

run buildall

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Aug 9, 2024

clang-tidy review says "All clean, LGTM! 👍"

@sollhui sollhui marked this pull request as ready for review August 9, 2024 12:40
@sollhui
Copy link
Copy Markdown
Contributor Author

sollhui commented Aug 9, 2024

run buildall

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Aug 9, 2024

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link
Copy Markdown

doris-robot commented Aug 9, 2024

TPC-H: Total hot run time: 39299 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 3f663fe08624f2b95a5c1660d18687534c91cdad, data reload: false

------ Round 1 ----------------------------------
q1	19305	5175	4280	4280
q2	2033	175	180	175
q3	10565	1121	1026	1026
q4	10271	693	633	633
q5	7501	2508	2466	2466
q6	226	141	138	138
q7	965	595	579	579
q8	9210	1893	1925	1893
q9	8644	6523	6539	6523
q10	7058	2337	2122	2122
q11	458	243	245	243
q12	389	220	221	220
q13	17764	2990	2977	2977
q14	279	242	240	240
q15	518	473	500	473
q16	508	384	380	380
q17	965	683	731	683
q18	8021	7454	7395	7395
q19	4382	1073	1098	1073
q20	680	335	338	335
q21	6211	4445	4577	4445
q22	1098	1006	1000	1000
Total cold run time: 117051 ms
Total hot run time: 39299 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4641	4243	4262	4243
q2	366	274	280	274
q3	2833	2642	2799	2642
q4	1962	1746	1655	1655
q5	5577	5448	5380	5380
q6	228	136	130	130
q7	2108	1812	1767	1767
q8	3278	3450	3460	3450
q9	8733	8752	8794	8752
q10	3477	3303	3180	3180
q11	598	499	488	488
q12	809	627	622	622
q13	15926	3155	3181	3155
q14	313	296	293	293
q15	531	502	475	475
q16	497	473	454	454
q17	1799	1530	1520	1520
q18	8200	7980	7924	7924
q19	1697	1622	1722	1622
q20	2063	1938	1851	1851
q21	7361	5331	5035	5035
q22	1103	1017	1045	1017
Total cold run time: 74100 ms
Total hot run time: 55929 ms

@doris-robot
Copy link
Copy Markdown

doris-robot commented Aug 9, 2024

TPC-DS: Total hot run time: 200999 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 3f663fe08624f2b95a5c1660d18687534c91cdad, data reload: false

query1	920	368	365	365
query2	6448	1959	1759	1759
query3	6632	207	215	207
query4	31368	23158	23101	23101
query5	3604	478	500	478
query6	265	170	171	170
query7	4575	294	298	294
query8	251	204	203	203
query9	8645	2479	2463	2463
query10	533	422	438	422
query11	17827	14878	15003	14878
query12	132	97	94	94
query13	1627	373	353	353
query14	10132	6870	7525	6870
query15	279	205	237	205
query16	7455	464	454	454
query17	1726	569	536	536
query18	1250	274	276	274
query19	194	143	133	133
query20	113	100	105	100
query21	206	101	99	99
query22	4464	4472	4388	4388
query23	35464	33540	33535	33535
query24	10585	2653	2652	2652
query25	528	390	401	390
query26	682	159	167	159
query27	2387	284	295	284
query28	6037	2075	2032	2032
query29	623	409	407	407
query30	268	150	149	149
query31	995	761	751	751
query32	94	53	52	52
query33	615	286	281	281
query34	856	463	471	463
query35	957	809	794	794
query36	1052	908	911	908
query37	132	82	81	81
query38	4324	4087	4139	4087
query39	1427	1376	1401	1376
query40	198	117	111	111
query41	44	43	46	43
query42	116	96	97	96
query43	504	469	465	465
query44	1111	736	726	726
query45	235	207	202	202
query46	1102	719	741	719
query47	1851	1784	1772	1772
query48	372	302	298	298
query49	859	437	444	437
query50	813	414	408	408
query51	6812	6692	6672	6672
query52	108	91	99	91
query53	253	182	188	182
query54	874	452	452	452
query55	79	82	76	76
query56	288	265	248	248
query57	1136	1082	1046	1046
query58	231	238	228	228
query59	3029	2698	2660	2660
query60	300	268	275	268
query61	117	110	115	110
query62	779	646	647	646
query63	206	190	184	184
query64	9284	2324	1824	1824
query65	3170	3121	3136	3121
query66	745	349	341	341
query67	15345	14766	14847	14766
query68	5658	565	566	565
query69	452	417	379	379
query70	1149	1064	1112	1064
query71	517	275	278	275
query72	19403	16710	16670	16670
query73	786	329	328	328
query74	9135	8718	8711	8711
query75	4209	2667	2664	2664
query76	3823	987	956	956
query77	659	346	309	309
query78	9710	9015	8984	8984
query79	7659	531	522	522
query80	1666	485	495	485
query81	608	224	230	224
query82	1521	136	135	135
query83	342	142	143	142
query84	279	73	76	73
query85	1441	264	292	264
query86	449	293	296	293
query87	4787	4578	4574	4574
query88	5029	2474	2495	2474
query89	468	284	284	284
query90	2001	193	199	193
query91	119	92	93	92
query92	59	48	49	48
query93	6430	540	540	540
query94	1012	291	296	291
query95	356	263	259	259
query96	612	274	280	274
query97	3261	3040	3060	3040
query98	227	201	194	194
query99	1603	1304	1254	1254
Total cold run time: 321063 ms
Total hot run time: 200999 ms

@doris-robot
Copy link
Copy Markdown

doris-robot commented Aug 9, 2024

ClickBench: Total hot run time: 30.79 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 3f663fe08624f2b95a5c1660d18687534c91cdad, data reload: false

query1	0.05	0.04	0.04
query2	0.08	0.04	0.04
query3	0.23	0.05	0.05
query4	1.67	0.07	0.08
query5	0.50	0.47	0.50
query6	1.13	0.73	0.72
query7	0.03	0.02	0.02
query8	0.05	0.04	0.05
query9	0.55	0.49	0.49
query10	0.55	0.55	0.54
query11	0.15	0.12	0.12
query12	0.15	0.12	0.12
query13	0.61	0.59	0.58
query14	0.78	0.78	0.77
query15	0.85	0.81	0.82
query16	0.38	0.36	0.37
query17	1.01	0.95	0.98
query18	0.24	0.22	0.22
query19	1.79	1.70	1.70
query20	0.01	0.03	0.01
query21	15.40	0.77	0.65
query22	4.58	6.84	2.17
query23	18.29	1.34	1.26
query24	2.14	0.23	0.21
query25	0.17	0.07	0.08
query26	0.29	0.21	0.21
query27	0.46	0.23	0.22
query28	13.21	1.02	0.99
query29	12.63	3.26	3.30
query30	0.24	0.05	0.05
query31	2.88	0.40	0.38
query32	3.26	0.47	0.47
query33	2.86	2.92	2.93
query34	17.08	4.34	4.35
query35	4.43	4.40	4.44
query36	0.66	0.49	0.47
query37	0.19	0.16	0.16
query38	0.16	0.15	0.15
query39	0.05	0.03	0.03
query40	0.16	0.12	0.13
query41	0.09	0.05	0.05
query42	0.06	0.05	0.05
query43	0.04	0.04	0.04
Total cold run time: 110.14 s
Total hot run time: 30.79 s

@sollhui sollhui marked this pull request as draft August 10, 2024 02:06
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Aug 12, 2024

clang-tidy review says "All clean, LGTM! 👍"

@sollhui
Copy link
Copy Markdown
Contributor Author

sollhui commented Aug 12, 2024

run buildall

@doris-robot
Copy link
Copy Markdown

doris-robot commented Aug 12, 2024

TPC-H: Total hot run time: 39289 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 25a967eebe879dca270fc8fb07899a7cacaa8a36, data reload: false

------ Round 1 ----------------------------------
q1	18511	4420	4329	4329
q2	2036	208	206	206
q3	11517	1007	1150	1007
q4	10504	779	738	738
q5	7522	2582	2538	2538
q6	222	135	135	135
q7	967	591	600	591
q8	9314	1908	1976	1908
q9	8735	6527	6543	6527
q10	7012	2257	2202	2202
q11	457	238	242	238
q12	387	216	214	214
q13	17749	2994	2936	2936
q14	271	238	225	225
q15	525	483	481	481
q16	494	390	380	380
q17	967	664	634	634
q18	8095	7345	7561	7345
q19	3767	1069	1019	1019
q20	705	331	332	331
q21	5500	4473	4257	4257
q22	1124	1060	1048	1048
Total cold run time: 116381 ms
Total hot run time: 39289 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4466	4247	4227	4227
q2	403	266	269	266
q3	2837	2660	2775	2660
q4	2001	1812	1712	1712
q5	5638	5565	5462	5462
q6	215	131	126	126
q7	2118	1762	1757	1757
q8	3303	3411	3435	3411
q9	8774	8692	8893	8692
q10	3488	3296	3290	3290
q11	577	500	486	486
q12	782	681	615	615
q13	16232	3233	3170	3170
q14	337	291	284	284
q15	529	491	500	491
q16	493	444	442	442
q17	1843	1522	1529	1522
q18	8055	7949	7664	7664
q19	4580	1649	1550	1550
q20	2075	1877	1925	1877
q21	5676	5380	5242	5242
q22	1116	1009	1020	1009
Total cold run time: 75538 ms
Total hot run time: 55955 ms

@doris-robot
Copy link
Copy Markdown

doris-robot commented Aug 12, 2024

TPC-DS: Total hot run time: 201599 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 25a967eebe879dca270fc8fb07899a7cacaa8a36, data reload: false

query1	1697	1528	1458	1458
query2	6440	2030	1943	1943
query3	14038	8816	313	313
query4	33680	23027	23108	23027
query5	4234	494	496	494
query6	254	160	166	160
query7	3935	292	294	292
query8	240	197	195	195
query9	8326	2361	2337	2337
query10	536	472	464	464
query11	15556	14911	14909	14909
query12	148	98	98	98
query13	1523	367	371	367
query14	9857	6983	7968	6983
query15	273	239	235	235
query16	7576	509	525	509
query17	1628	588	554	554
query18	1690	283	272	272
query19	193	140	146	140
query20	123	109	107	107
query21	204	104	102	102
query22	4730	4393	4352	4352
query23	34278	33608	33572	33572
query24	10529	2604	2517	2517
query25	554	367	358	358
query26	701	150	153	150
query27	2120	265	291	265
query28	5991	1993	1980	1980
query29	734	404	399	399
query30	252	151	150	150
query31	951	777	738	738
query32	89	53	53	53
query33	621	273	277	273
query34	858	467	450	450
query35	939	848	865	848
query36	1074	929	919	919
query37	134	82	80	80
query38	4349	4293	4144	4144
query39	1427	1373	1433	1373
query40	192	107	112	107
query41	46	44	47	44
query42	113	96	93	93
query43	501	456	486	456
query44	1065	730	740	730
query45	227	198	205	198
query46	1073	725	717	717
query47	1860	1799	1769	1769
query48	364	297	301	297
query49	822	411	422	411
query50	812	398	400	398
query51	6810	6759	6592	6592
query52	102	84	93	84
query53	248	176	178	176
query54	833	438	435	435
query55	77	80	76	76
query56	269	235	245	235
query57	1125	1073	1047	1047
query58	248	241	225	225
query59	2976	2886	2712	2712
query60	283	258	275	258
query61	99	94	94	94
query62	764	631	614	614
query63	213	175	180	175
query64	9195	2277	1764	1764
query65	3191	3157	3200	3157
query66	745	319	316	316
query67	15227	14896	14739	14739
query68	4531	529	523	523
query69	393	354	404	354
query70	1087	1113	1152	1113
query71	396	272	280	272
query72	17333	16995	16261	16261
query73	753	322	327	322
query74	9006	8788	8744	8744
query75	3362	2680	2645	2645
query76	2498	976	905	905
query77	468	310	317	310
query78	9536	9011	9018	9011
query79	2448	512	516	512
query80	2344	522	529	522
query81	593	226	218	218
query82	937	204	133	133
query83	297	146	142	142
query84	254	82	74	74
query85	1992	272	266	266
query86	496	295	289	289
query87	4647	4597	4505	4505
query88	4460	2508	2499	2499
query89	406	291	280	280
query90	1835	205	194	194
query91	113	96	95	95
query92	61	48	52	48
query93	3828	540	538	538
query94	897	288	289	288
query95	360	259	266	259
query96	613	279	274	274
query97	3241	3051	3079	3051
query98	213	207	255	207
query99	1702	1246	1312	1246
Total cold run time: 313062 ms
Total hot run time: 201599 ms

@doris-robot
Copy link
Copy Markdown

doris-robot commented Aug 12, 2024

ClickBench: Total hot run time: 30.47 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 25a967eebe879dca270fc8fb07899a7cacaa8a36, data reload: false

query1	0.05	0.04	0.04
query2	0.08	0.04	0.04
query3	0.23	0.05	0.05
query4	1.66	0.09	0.10
query5	0.50	0.49	0.49
query6	1.14	0.73	0.73
query7	0.02	0.01	0.01
query8	0.05	0.04	0.04
query9	0.56	0.49	0.48
query10	0.56	0.52	0.53
query11	0.15	0.11	0.12
query12	0.15	0.12	0.13
query13	0.60	0.58	0.59
query14	0.76	0.78	0.76
query15	0.82	0.81	0.82
query16	0.35	0.36	0.38
query17	1.04	0.94	1.04
query18	0.24	0.23	0.22
query19	1.80	1.65	1.67
query20	0.02	0.01	0.01
query21	15.39	0.74	0.66
query22	4.33	7.33	1.86
query23	18.22	1.36	1.32
query24	2.11	0.21	0.22
query25	0.14	0.08	0.07
query26	0.30	0.21	0.20
query27	0.45	0.22	0.22
query28	13.31	1.02	0.99
query29	12.58	3.27	3.29
query30	0.23	0.05	0.04
query31	2.91	0.38	0.38
query32	3.25	0.49	0.48
query33	2.86	2.93	2.88
query34	17.14	4.36	4.45
query35	4.48	4.44	4.46
query36	0.64	0.46	0.49
query37	0.18	0.15	0.16
query38	0.15	0.15	0.15
query39	0.04	0.04	0.03
query40	0.15	0.12	0.12
query41	0.10	0.04	0.05
query42	0.06	0.05	0.05
query43	0.04	0.04	0.04
Total cold run time: 109.84 s
Total hot run time: 30.47 s

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Aug 12, 2024

clang-tidy review says "All clean, LGTM! 👍"

@sollhui sollhui marked this pull request as ready for review August 19, 2024 07:40
@sollhui
Copy link
Copy Markdown
Contributor Author

sollhui commented Aug 19, 2024

run buildall

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Aug 21, 2024

clang-tidy review says "All clean, LGTM! 👍"

1 similar comment
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Aug 21, 2024

clang-tidy review says "All clean, LGTM! 👍"

@sollhui
Copy link
Copy Markdown
Contributor Author

sollhui commented Aug 21, 2024

run buildall

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Aug 21, 2024

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link
Copy Markdown

doris-robot commented Aug 21, 2024

TPC-H: Total hot run time: 38540 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 22d2d73777754a39777e556c4bbf715aca3f6f81, data reload: false

------ Round 1 ----------------------------------
q1	18257	4580	4351	4351
q2	2060	215	229	215
q3	11527	984	1289	984
q4	10551	769	706	706
q5	7782	2893	2835	2835
q6	265	161	160	160
q7	1015	659	654	654
q8	9409	2117	2116	2116
q9	7229	6576	6571	6571
q10	7083	2199	2240	2199
q11	492	279	278	278
q12	431	268	267	267
q13	18193	2985	3024	2985
q14	305	265	264	264
q15	563	523	544	523
q16	533	428	417	417
q17	1017	720	769	720
q18	7472	6913	6812	6812
q19	6759	1110	1066	1066
q20	807	359	383	359
q21	4154	3007	3020	3007
q22	1143	1051	1066	1051
Total cold run time: 117047 ms
Total hot run time: 38540 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4533	4275	4313	4275
q2	431	314	310	310
q3	2835	2604	2686	2604
q4	2031	1734	1689	1689
q5	5746	5694	5663	5663
q6	249	165	147	147
q7	2186	1797	1747	1747
q8	3287	3502	3492	3492
q9	8855	8799	8801	8799
q10	3628	3371	3312	3312
q11	637	538	541	538
q12	862	682	630	630
q13	17283	3205	3058	3058
q14	323	282	313	282
q15	582	519	522	519
q16	514	445	450	445
q17	1878	1553	1545	1545
q18	8390	7900	7759	7759
q19	9961	1676	1632	1632
q20	2183	1917	1878	1878
q21	13795	5357	5332	5332
q22	1184	1098	1079	1079
Total cold run time: 91373 ms
Total hot run time: 56735 ms

@doris-robot
Copy link
Copy Markdown

doris-robot commented Aug 21, 2024

ClickBench: Total hot run time: 31.69 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit f4aca4ba0a24c3783addffb3382f914f3b74ffa3, data reload: false

query1	0.05	0.05	0.05
query2	0.08	0.04	0.04
query3	0.22	0.06	0.05
query4	1.66	0.08	0.08
query5	0.52	0.48	0.48
query6	1.13	0.73	0.72
query7	0.02	0.02	0.01
query8	0.06	0.05	0.05
query9	0.56	0.49	0.49
query10	0.56	0.55	0.54
query11	0.17	0.13	0.12
query12	0.15	0.13	0.13
query13	0.62	0.61	0.62
query14	0.77	0.80	0.78
query15	0.85	0.82	0.82
query16	0.37	0.36	0.37
query17	1.04	1.06	0.98
query18	0.21	0.20	0.23
query19	1.86	1.79	1.73
query20	0.01	0.01	0.01
query21	15.41	0.66	0.65
query22	3.92	7.07	2.31
query23	18.26	1.34	1.32
query24	2.14	0.25	0.22
query25	0.16	0.08	0.09
query26	0.27	0.19	0.18
query27	0.10	0.08	0.09
query28	13.25	1.02	1.02
query29	12.64	3.34	3.37
query30	0.43	0.25	0.24
query31	2.81	0.40	0.40
query32	3.23	0.49	0.49
query33	2.97	3.01	3.05
query34	17.14	4.40	4.46
query35	4.51	4.55	4.49
query36	0.68	0.51	0.49
query37	0.21	0.18	0.17
query38	0.19	0.17	0.16
query39	0.07	0.06	0.06
query40	0.19	0.15	0.16
query41	0.12	0.06	0.07
query42	0.08	0.06	0.07
query43	0.06	0.06	0.06
Total cold run time: 109.75 s
Total hot run time: 31.69 s

@sollhui
Copy link
Copy Markdown
Contributor Author

sollhui commented Aug 22, 2024

run buildall

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Aug 22, 2024

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link
Copy Markdown

doris-robot commented Aug 22, 2024

TPC-H: Total hot run time: 39141 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 2decdda20e666df4548f7147927c1c2107a5e6ae, data reload: false

------ Round 1 ----------------------------------
q1	18652	5412	4389	4389
q2	2053	226	237	226
q3	11077	1014	1108	1014
q4	10209	806	812	806
q5	7784	2906	2859	2859
q6	267	159	158	158
q7	1020	660	651	651
q8	9409	2136	2135	2135
q9	7326	6654	6664	6654
q10	7100	2306	2265	2265
q11	508	275	272	272
q12	433	263	266	263
q13	17850	3089	3037	3037
q14	311	265	261	261
q15	576	539	517	517
q16	536	426	418	418
q17	1014	716	665	665
q18	7456	6967	7042	6967
q19	1782	1190	1129	1129
q20	746	381	383	381
q21	3940	3130	3005	3005
q22	1194	1107	1069	1069
Total cold run time: 111243 ms
Total hot run time: 39141 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4486	4334	4307	4307
q2	423	323	321	321
q3	2926	2632	2687	2632
q4	1955	1777	1815	1777
q5	5684	5654	5748	5654
q6	239	147	147	147
q7	2254	1846	1835	1835
q8	3360	3484	3469	3469
q9	8887	8844	8837	8837
q10	3574	3291	3407	3291
q11	628	539	533	533
q12	852	659	708	659
q13	17229	3227	3243	3227
q14	363	317	301	301
q15	581	511	528	511
q16	508	449	449	449
q17	1832	1577	1548	1548
q18	8301	7946	7837	7837
q19	2851	1558	1422	1422
q20	2131	1935	1954	1935
q21	5770	5407	5562	5407
q22	1232	1129	1143	1129
Total cold run time: 76066 ms
Total hot run time: 57228 ms

@doris-robot
Copy link
Copy Markdown

doris-robot commented Aug 22, 2024

TPC-DS: Total hot run time: 198650 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 2decdda20e666df4548f7147927c1c2107a5e6ae, data reload: false

query1	1308	919	879	879
query2	6459	2058	2085	2058
query3	10834	4182	4129	4129
query4	60424	24000	23298	23298
query5	5650	671	656	656
query6	450	188	195	188
query7	5781	317	326	317
query8	466	378	439	378
query9	9007	2559	2576	2559
query10	574	353	342	342
query11	18174	15254	15481	15254
query12	193	143	145	143
query13	1631	507	451	451
query14	10930	7009	7603	7009
query15	256	199	195	195
query16	7548	486	561	486
query17	1156	630	629	629
query18	2067	358	366	358
query19	333	177	171	171
query20	157	144	139	139
query21	247	156	147	147
query22	4885	4509	4683	4509
query23	34375	34075	34036	34036
query24	6115	3059	3106	3059
query25	587	439	429	429
query26	735	195	191	191
query27	1789	309	310	309
query28	3833	2181	2171	2171
query29	726	447	449	447
query30	227	194	179	179
query31	1016	869	818	818
query32	101	74	79	74
query33	507	344	353	344
query34	886	504	536	504
query35	897	751	782	751
query36	1116	971	973	971
query37	167	112	104	104
query38	4076	3845	3892	3845
query39	1593	1464	1461	1461
query40	235	159	152	152
query41	144	138	139	138
query42	132	117	114	114
query43	553	539	537	537
query44	1145	793	801	793
query45	225	196	197	196
query46	1146	795	794	794
query47	1989	1916	1934	1916
query48	419	348	354	348
query49	941	591	588	588
query50	866	468	477	468
query51	7246	7127	7199	7127
query52	118	108	108	108
query53	300	230	247	230
query54	616	525	530	525
query55	92	89	87	87
query56	344	330	314	314
query57	1248	1147	1153	1147
query58	294	304	309	304
query59	3178	2897	2886	2886
query60	354	330	334	330
query61	153	153	152	152
query62	813	709	698	698
query63	266	235	227	227
query64	3467	1862	1844	1844
query65	3227	3197	3217	3197
query66	1037	680	681	680
query67	15525	15461	15273	15273
query68	4367	602	597	597
query69	473	333	327	327
query70	1223	1204	1146	1146
query71	407	319	327	319
query72	6502	2404	2129	2129
query73	803	355	359	355
query74	9414	8972	8916	8916
query75	3473	2799	2845	2799
query76	1816	1043	1001	1001
query77	680	445	440	440
query78	9897	9189	9902	9189
query79	2191	552	562	552
query80	1068	631	630	630
query81	605	268	262	262
query82	450	158	159	158
query83	258	210	212	210
query84	303	101	99	99
query85	1141	374	366	366
query86	336	329	328	328
query87	4403	4343	4257	4257
query88	4034	2541	2565	2541
query89	433	323	324	323
query90	1542	231	247	231
query91	153	128	133	128
query92	82	82	75	75
query93	2377	553	551	551
query94	599	325	313	313
query95	381	301	297	297
query96	629	284	282	282
query97	3242	3127	3158	3127
query98	265	224	226	224
query99	1966	1317	1328	1317
Total cold run time: 318862 ms
Total hot run time: 198650 ms

@doris-robot
Copy link
Copy Markdown

doris-robot commented Aug 22, 2024

ClickBench: Total hot run time: 31.05 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 2decdda20e666df4548f7147927c1c2107a5e6ae, data reload: false

query1	0.04	0.05	0.04
query2	0.08	0.04	0.05
query3	0.23	0.05	0.06
query4	1.66	0.08	0.08
query5	0.50	0.49	0.48
query6	1.14	0.74	0.73
query7	0.02	0.01	0.01
query8	0.05	0.05	0.05
query9	0.55	0.49	0.50
query10	0.56	0.55	0.56
query11	0.17	0.12	0.12
query12	0.15	0.13	0.14
query13	0.62	0.59	0.59
query14	0.76	0.81	0.79
query15	0.84	0.84	0.84
query16	0.38	0.38	0.37
query17	0.99	0.96	1.03
query18	0.23	0.21	0.20
query19	1.84	1.86	1.75
query20	0.02	0.01	0.01
query21	15.41	0.66	0.66
query22	4.74	7.40	1.64
query23	18.26	1.44	1.27
query24	2.08	0.23	0.23
query25	0.16	0.09	0.09
query26	0.26	0.18	0.18
query27	0.09	0.09	0.08
query28	13.22	1.03	1.00
query29	12.66	3.37	3.36
query30	0.44	0.25	0.25
query31	2.81	0.40	0.40
query32	3.26	0.49	0.49
query33	2.99	3.00	3.04
query34	17.12	4.40	4.45
query35	4.42	4.50	4.47
query36	0.68	0.48	0.49
query37	0.21	0.18	0.18
query38	0.17	0.18	0.17
query39	0.07	0.06	0.06
query40	0.19	0.16	0.16
query41	0.12	0.07	0.07
query42	0.08	0.07	0.07
query43	0.07	0.07	0.06
Total cold run time: 110.34 s
Total hot run time: 31.05 s

dataroaring
dataroaring previously approved these changes Aug 23, 2024
View reviewed changes
Copy link
Copy Markdown
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Aug 23, 2024
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Aug 23, 2024

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Aug 23, 2024

PR approved by anyone and no changes requested.

liaoxin01
liaoxin01 previously approved these changes Aug 23, 2024
View reviewed changes
Copy link
Copy Markdown
Contributor

@liaoxin01 liaoxin01 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@sollhui sollhui dismissed stale reviews from liaoxin01 and dataroaring via 958ab7f August 24, 2024 03:11
@sollhui
Copy link
Copy Markdown
Contributor Author

sollhui commented Aug 24, 2024

run buildall

@github-actions github-actions bot removed the approved Indicates a PR has been approved by one committer. label Aug 24, 2024
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Aug 24, 2024

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link
Copy Markdown

doris-robot commented Aug 24, 2024

TPC-H: Total hot run time: 38142 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 958ab7fc65dd75465f5488d6e04f9f27f35ad0b9, data reload: false

------ Round 1 ----------------------------------
q1	18397	4715	4329	4329
q2	2041	189	181	181
q3	11477	966	1101	966
q4	10498	721	733	721
q5	7793	2844	2886	2844
q6	222	137	137	137
q7	970	616	611	611
q8	9326	2094	2115	2094
q9	7247	6536	6559	6536
q10	7014	2192	2271	2192
q11	434	253	243	243
q12	402	225	220	220
q13	17769	3056	3019	3019
q14	297	236	237	236
q15	529	479	484	479
q16	504	412	392	392
q17	996	705	666	666
q18	7549	6835	6924	6835
q19	1391	1098	1071	1071
q20	699	336	326	326
q21	3993	3027	3038	3027
q22	1101	1041	1017	1017
Total cold run time: 110649 ms
Total hot run time: 38142 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4329	4268	4295	4268
q2	377	281	286	281
q3	2890	2611	2685	2611
q4	1955	1708	1740	1708
q5	5649	5690	5734	5690
q6	240	148	145	145
q7	2195	1869	1835	1835
q8	3271	3470	3486	3470
q9	8875	8838	8842	8838
q10	3566	3423	3334	3334
q11	612	511	526	511
q12	798	672	624	624
q13	14511	3244	3158	3158
q14	317	285	282	282
q15	540	500	506	500
q16	513	445	471	445
q17	1893	1577	1538	1538
q18	8211	7893	7951	7893
q19	1740	1428	1509	1428
q20	2154	1885	1926	1885
q21	5728	5583	5543	5543
q22	1162	1091	990	990
Total cold run time: 71526 ms
Total hot run time: 56977 ms

@doris-robot
Copy link
Copy Markdown

doris-robot commented Aug 24, 2024

TPC-DS: Total hot run time: 192145 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 958ab7fc65dd75465f5488d6e04f9f27f35ad0b9, data reload: false

query1	1257	882	864	864
query2	6181	1933	1919	1919
query3	10619	3975	3932	3932
query4	60028	25312	23198	23198
query5	5381	502	501	501
query6	420	158	173	158
query7	5885	298	301	298
query8	290	205	199	199
query9	9000	2473	2445	2445
query10	524	271	259	259
query11	17992	14958	15197	14958
query12	149	105	103	103
query13	1576	409	396	396
query14	11243	7406	7252	7252
query15	224	180	165	165
query16	7516	425	435	425
query17	1250	592	581	581
query18	1825	304	300	300
query19	321	149	154	149
query20	118	120	109	109
query21	204	104	100	100
query22	4461	4599	4501	4501
query23	34233	33580	33483	33483
query24	5934	2954	2883	2883
query25	553	399	404	399
query26	692	167	165	165
query27	1767	286	296	286
query28	3673	2057	2041	2041
query29	724	429	430	429
query30	231	149	148	148
query31	953	769	756	756
query32	92	56	59	56
query33	506	304	296	296
query34	939	481	484	481
query35	842	732	711	711
query36	1079	944	945	944
query37	152	92	89	89
query38	4094	3824	3828	3824
query39	1602	1386	1419	1386
query40	209	120	116	116
query41	47	47	47	47
query42	116	98	99	98
query43	529	486	473	473
query44	1134	758	751	751
query45	202	170	169	169
query46	1165	778	750	750
query47	1848	1783	1794	1783
query48	389	308	324	308
query49	807	441	448	441
query50	883	430	437	430
query51	7155	7081	6942	6942
query52	107	89	89	89
query53	268	192	187	187
query54	585	471	468	468
query55	79	78	80	78
query56	299	273	265	265
query57	1194	1080	1041	1041
query58	227	256	274	256
query59	3036	2731	2896	2731
query60	302	271	270	270
query61	106	98	97	97
query62	765	633	644	633
query63	230	191	186	186
query64	3837	2228	1790	1790
query65	3164	3130	3167	3130
query66	663	340	335	335
query67	15345	15136	15459	15136
query68	3213	605	589	589
query69	416	303	290	290
query70	1133	1115	1122	1115
query71	374	289	283	283
query72	6242	2312	2055	2055
query73	817	328	332	328
query74	9160	8816	8931	8816
query75	3447	2717	2684	2684
query76	1580	1062	1094	1062
query77	526	328	331	328
query78	9668	8969	9080	8969
query79	1132	547	540	540
query80	741	514	519	514
query81	458	229	227	227
query82	240	142	134	134
query83	175	149	149	149
query84	254	81	74	74
query85	687	337	286	286
query86	337	276	315	276
query87	4396	4310	4312	4310
query88	3163	2385	2417	2385
query89	402	285	280	280
query90	2028	208	197	197
query91	122	101	100	100
query92	62	54	56	54
query93	1099	545	541	541
query94	735	298	299	298
query95	359	259	266	259
query96	644	273	276	273
query97	3198	3048	3069	3048
query98	210	203	220	203
query99	1545	1289	1290	1289
Total cold run time: 307784 ms
Total hot run time: 192145 ms

@doris-robot
Copy link
Copy Markdown

doris-robot commented Aug 24, 2024

ClickBench: Total hot run time: 30.98 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 958ab7fc65dd75465f5488d6e04f9f27f35ad0b9, data reload: false

query1	0.05	0.05	0.04
query2	0.08	0.04	0.04
query3	0.22	0.05	0.05
query4	1.67	0.09	0.10
query5	0.50	0.50	0.48
query6	1.15	0.73	0.73
query7	0.02	0.01	0.01
query8	0.05	0.04	0.05
query9	0.54	0.48	0.48
query10	0.53	0.55	0.54
query11	0.15	0.11	0.12
query12	0.15	0.13	0.13
query13	0.61	0.60	0.59
query14	0.75	0.80	0.75
query15	0.84	0.82	0.80
query16	0.35	0.35	0.35
query17	1.00	0.95	1.01
query18	0.22	0.23	0.21
query19	1.90	1.79	1.81
query20	0.01	0.01	0.01
query21	15.41	0.72	0.66
query22	4.14	6.54	2.29
query23	18.24	1.32	1.20
query24	2.08	0.22	0.24
query25	0.15	0.08	0.08
query26	0.26	0.18	0.18
query27	0.07	0.08	0.07
query28	13.25	1.02	1.00
query29	12.65	3.33	3.30
query30	0.24	0.06	0.05
query31	2.87	0.39	0.39
query32	3.26	0.47	0.47
query33	2.94	2.98	3.02
query34	17.08	4.39	4.44
query35	4.47	4.43	4.43
query36	0.66	0.48	0.49
query37	0.20	0.16	0.16
query38	0.16	0.15	0.15
query39	0.05	0.04	0.03
query40	0.16	0.13	0.13
query41	0.10	0.05	0.05
query42	0.06	0.04	0.04
query43	0.05	0.05	0.04
Total cold run time: 109.34 s
Total hot run time: 30.98 s

dataroaring
dataroaring approved these changes Aug 24, 2024
View reviewed changes
Copy link
Copy Markdown
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Aug 24, 2024
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Aug 24, 2024

PR approved by at least one committer and no changes requested.

@liaoxin01 liaoxin01 merged commit 2c64b52 into apache:master Aug 26, 2024
dataroaring pushed a commit that referenced this pull request Aug 26, 2024
@sollhui @dataroaring
[fix](cloud) fix alter consume offset can not work in cloud mode (#39159
07e3e9f 
)

In cloud mode, if update consume offset, it will not update progress in
meta service, which causes alter consume offset can not work for
progress will regularly update with persistent progress values in meta
service.
@gavinchou gavinchou mentioned this pull request Oct 13, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@liaoxin01 liaoxin01 liaoxin01 approved these changes

@dataroaring dataroaring dataroaring approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/3.0.2-merged doing meta-change p0_b reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@sollhui @doris-robot @liaoxin01 @dataroaring