[refactor](wg&memtracker) using weak ptr to delete memtracker and query context automatically by yiguolei · Pull Request #41549 · apache/doris

yiguolei · 2024-10-08T07:41:38Z

Proposed changes

Issue Number: close #xxx

…ry context automatically

doris-robot · 2024-10-08T07:41:44Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

yiguolei · 2024-10-08T07:41:50Z

run buildall

github-actions · 2024-10-08T07:46:42Z

clang-tidy review says "All clean, LGTM! 👍"

doris-robot · 2024-10-08T09:04:46Z

TPC-H: Total hot run time: 40719 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d3b924c8774e8276428b3466ced328febfda681c, data reload: false

------ Round 1 ----------------------------------
q1	17566	7515	7287	7287
q2	2047	291	280	280
q3	11751	1090	1240	1090
q4	10574	766	684	684
q5	7731	2904	2966	2904
q6	236	153	151	151
q7	1031	649	602	602
q8	9350	1957	2042	1957
q9	6629	6511	6464	6464
q10	7014	2286	2285	2285
q11	445	257	252	252
q12	425	222	215	215
q13	17801	2968	2938	2938
q14	242	224	206	206
q15	573	534	514	514
q16	642	587	577	577
q17	990	570	585	570
q18	7242	6654	6606	6606
q19	1353	913	868	868
q20	492	201	202	201
q21	4068	3325	3102	3102
q22	1107	966	1006	966
Total cold run time: 109309 ms
Total hot run time: 40719 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7254	7262	7296	7262
q2	332	224	228	224
q3	3169	2989	3016	2989
q4	2109	1950	1774	1774
q5	5778	5806	5786	5786
q6	235	146	146	146
q7	2260	1876	1805	1805
q8	3408	3593	3466	3466
q9	8943	8875	8813	8813
q10	3617	3567	3537	3537
q11	589	510	497	497
q12	828	629	623	623
q13	9014	3143	3135	3135
q14	305	294	279	279
q15	579	521	520	520
q16	703	655	667	655
q17	1873	1633	1594	1594
q18	8313	7756	7507	7507
q19	1739	1465	1366	1366
q20	2106	1889	1880	1880
q21	5528	5809	5609	5609
q22	1102	1047	1055	1047
Total cold run time: 69784 ms
Total hot run time: 60514 ms

doris-robot · 2024-10-08T09:10:36Z

TeamCity be ut coverage result:
Function Coverage: 37.29% (9632/25830)
Line Coverage: 28.67% (79866/278587)
Region Coverage: 28.11% (41304/146936)
Branch Coverage: 24.73% (21043/85094)
Coverage Report: http://coverage.selectdb-in.cc/coverage/d3b924c8774e8276428b3466ced328febfda681c_d3b924c8774e8276428b3466ced328febfda681c/report/index.html

yiguolei · 2024-10-08T09:43:15Z

run buildall

github-actions · 2024-10-08T09:48:24Z

clang-tidy review says "All clean, LGTM! 👍"

doris-robot · 2024-10-08T10:02:45Z

TPC-H: Total hot run time: 40808 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 5b451012ff8478de0ff243e4c8af3e368065a87e, data reload: false

------ Round 1 ----------------------------------
q1	17990	7558	7409	7409
q2	2790	161	177	161
q3	10555	1143	1153	1143
q4	10209	754	783	754
q5	7705	2832	2768	2768
q6	242	149	152	149
q7	999	637	627	627
q8	9345	1912	1950	1912
q9	6513	6419	6394	6394
q10	6931	2256	2315	2256
q11	433	245	247	245
q12	403	211	213	211
q13	17770	2969	2962	2962
q14	254	232	217	217
q15	566	531	539	531
q16	642	579	577	577
q17	972	562	585	562
q18	7133	6663	6642	6642
q19	1346	987	940	940
q20	471	200	198	198
q21	3966	3254	3174	3174
q22	1089	976	992	976
Total cold run time: 108324 ms
Total hot run time: 40808 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7239	7254	7260	7254
q2	355	229	224	224
q3	2960	2759	2810	2759
q4	1911	1722	1683	1683
q5	5433	5437	5468	5437
q6	223	142	140	140
q7	2159	1709	1707	1707
q8	3226	3396	3411	3396
q9	8524	8523	8500	8500
q10	3502	3427	3438	3427
q11	580	471	477	471
q12	789	587	616	587
q13	6096	3003	2966	2966
q14	285	260	271	260
q15	572	509	517	509
q16	670	622	625	622
q17	1798	1583	1547	1547
q18	7689	7246	7358	7246
q19	1669	1436	1421	1421
q20	2069	1819	1826	1819
q21	5355	5221	5251	5221
q22	1098	1031	965	965
Total cold run time: 64202 ms
Total hot run time: 58161 ms

doris-robot · 2024-10-08T10:07:18Z

TeamCity be ut coverage result:
Function Coverage: 37.28% (9630/25830)
Line Coverage: 28.67% (79876/278587)
Region Coverage: 28.10% (41288/146936)
Branch Coverage: 24.73% (21040/85094)
Coverage Report: http://coverage.selectdb-in.cc/coverage/5b451012ff8478de0ff243e4c8af3e368065a87e_5b451012ff8478de0ff243e4c8af3e368065a87e/report/index.html

doris-robot · 2024-10-08T10:14:49Z

TPC-DS: Total hot run time: 191564 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 5b451012ff8478de0ff243e4c8af3e368065a87e, data reload: false

query1	984	373	374	373
query2	6567	2076	1989	1989
query3	6703	211	223	211
query4	34013	23486	23368	23368
query5	4377	465	467	465
query6	272	176	170	170
query7	4636	301	308	301
query8	288	243	226	226
query9	9628	2656	2673	2656
query10	476	284	278	278
query11	17875	15107	15093	15093
query12	154	100	100	100
query13	1686	458	442	442
query14	10376	7530	7461	7461
query15	334	170	184	170
query16	7932	463	466	463
query17	1718	576	573	573
query18	2131	320	333	320
query19	377	151	153	151
query20	122	119	113	113
query21	220	105	103	103
query22	4593	4335	4202	4202
query23	34766	33978	34075	33978
query24	11148	2829	2838	2829
query25	639	396	400	396
query26	1125	165	163	163
query27	2505	293	295	293
query28	7784	2441	2416	2416
query29	838	435	425	425
query30	334	162	156	156
query31	1043	803	827	803
query32	99	55	56	55
query33	790	303	300	300
query34	964	501	511	501
query35	866	736	729	729
query36	1120	928	916	916
query37	153	89	86	86
query38	3954	3886	3918	3886
query39	1498	1453	1475	1453
query40	216	99	100	99
query41	47	46	46	46
query42	116	96	97	96
query43	527	478	489	478
query44	1187	818	813	813
query45	194	178	170	170
query46	1163	719	698	698
query47	1934	1852	1851	1851
query48	423	345	338	338
query49	1157	413	409	409
query50	832	423	419	419
query51	7152	6940	6916	6916
query52	99	87	88	87
query53	263	187	186	186
query54	1340	485	486	485
query55	80	73	75	73
query56	273	248	251	248
query57	1277	1154	1120	1120
query58	245	248	252	248
query59	3294	3060	2978	2978
query60	297	273	270	270
query61	109	101	98	98
query62	858	687	673	673
query63	215	190	188	188
query64	5283	642	613	613
query65	3289	3199	3203	3199
query66	1356	338	301	301
query67	16019	15586	15487	15487
query68	4846	544	547	544
query69	453	299	320	299
query70	1203	1115	1062	1062
query71	340	271	277	271
query72	6351	4012	4193	4012
query73	762	344	337	337
query74	10141	9071	8967	8967
query75	3374	2589	2676	2589
query76	3024	927	905	905
query77	445	295	286	286
query78	10592	9631	9536	9536
query79	1940	596	593	593
query80	2068	465	442	442
query81	584	241	243	241
query82	942	137	138	137
query83	288	138	143	138
query84	268	84	83	83
query85	1459	283	276	276
query86	438	294	325	294
query87	4491	4332	4339	4332
query88	3994	2408	2379	2379
query89	398	293	288	288
query90	2103	183	187	183
query91	143	108	106	106
query92	70	44	46	44
query93	1823	555	546	546
query94	1194	303	299	299
query95	362	249	257	249
query96	611	286	280	280
query97	3259	3235	3153	3153
query98	217	198	190	190
query99	1706	1316	1303	1303
Total cold run time: 304222 ms
Total hot run time: 191564 ms

doris-robot · 2024-10-08T10:20:16Z

ClickBench: Total hot run time: 32.08 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 5b451012ff8478de0ff243e4c8af3e368065a87e, data reload: false

query1	0.05	0.04	0.04
query2	0.06	0.03	0.03
query3	0.23	0.06	0.06
query4	1.65	0.10	0.11
query5	0.50	0.56	0.54
query6	1.13	0.73	0.73
query7	0.02	0.01	0.01
query8	0.03	0.03	0.03
query9	0.56	0.50	0.50
query10	0.56	0.57	0.56
query11	0.14	0.11	0.10
query12	0.14	0.11	0.11
query13	0.61	0.61	0.60
query14	2.75	2.76	2.74
query15	0.92	0.83	0.83
query16	0.40	0.38	0.37
query17	1.04	1.06	1.08
query18	0.23	0.23	0.23
query19	1.96	1.86	2.00
query20	0.01	0.01	0.01
query21	15.35	0.58	0.60
query22	3.05	2.96	1.20
query23	17.01	1.12	0.89
query24	2.50	0.93	0.76
query25	0.15	0.18	0.09
query26	0.35	0.13	0.12
query27	0.04	0.05	0.05
query28	11.31	1.09	1.08
query29	12.57	3.20	3.18
query30	0.24	0.06	0.07
query31	2.87	0.38	0.36
query32	3.30	0.47	0.47
query33	3.02	3.02	3.08
query34	17.13	4.43	4.47
query35	4.52	4.52	4.50
query36	0.67	0.48	0.50
query37	0.08	0.06	0.06
query38	0.04	0.03	0.03
query39	0.04	0.02	0.03
query40	0.16	0.12	0.13
query41	0.08	0.03	0.02
query42	0.03	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 107.54 s
Total hot run time: 32.08 s

yiguolei · 2024-10-09T01:54:21Z

run buildall

github-actions · 2024-10-09T01:59:00Z

clang-tidy review says "All clean, LGTM! 👍"

doris-robot · 2024-10-09T03:25:03Z

TeamCity be ut coverage result:
Function Coverage: 37.28% (9629/25832)
Line Coverage: 28.66% (79835/278588)
Region Coverage: 28.10% (41284/146931)
Branch Coverage: 24.71% (21027/85084)
Coverage Report: http://coverage.selectdb-in.cc/coverage/c4b9ed1d1f820c07345f6c538724a93d4e185410_c4b9ed1d1f820c07345f6c538724a93d4e185410/report/index.html

doris-robot · 2024-10-09T03:33:39Z

TPC-H: Total hot run time: 40939 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c4b9ed1d1f820c07345f6c538724a93d4e185410, data reload: false

------ Round 1 ----------------------------------
q1	17802	7529	7305	7305
q2	2034	280	274	274
q3	11783	1064	1156	1064
q4	10526	709	758	709
q5	7743	2929	2802	2802
q6	241	150	146	146
q7	1025	633	612	612
q8	9345	1959	1990	1959
q9	6468	6442	6454	6442
q10	6959	2286	2295	2286
q11	444	256	238	238
q12	420	226	217	217
q13	17771	2966	3003	2966
q14	243	210	217	210
q15	592	531	527	527
q16	641	595	582	582
q17	982	519	668	519
q18	7363	6674	6668	6668
q19	1354	1036	977	977
q20	486	205	201	201
q21	4085	3233	3319	3233
q22	1105	1011	1002	1002
Total cold run time: 109412 ms
Total hot run time: 40939 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7261	7270	7239	7239
q2	328	231	228	228
q3	3031	2910	2920	2910
q4	2075	1919	1768	1768
q5	5805	5762	5742	5742
q6	236	144	145	144
q7	2215	1840	1806	1806
q8	3362	3619	3431	3431
q9	8933	8939	8884	8884
q10	3584	3582	3501	3501
q11	585	504	500	500
q12	832	650	621	621
q13	11636	3227	3182	3182
q14	308	271	274	271
q15	583	521	519	519
q16	689	636	658	636
q17	1879	1608	1601	1601
q18	8282	7834	7649	7649
q19	1749	1551	1555	1551
q20	2119	1867	1895	1867
q21	5671	5456	5559	5456
q22	1134	1073	1030	1030
Total cold run time: 72297 ms
Total hot run time: 60536 ms

doris-robot · 2024-10-09T03:45:42Z

TPC-DS: Total hot run time: 192566 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c4b9ed1d1f820c07345f6c538724a93d4e185410, data reload: false

query1	878	405	418	405
query2	6254	2095	1990	1990
query3	8692	193	199	193
query4	33984	23575	23979	23575
query5	3324	492	471	471
query6	258	173	167	167
query7	4197	311	304	304
query8	277	220	222	220
query9	9573	2682	2685	2682
query10	464	282	291	282
query11	17876	15291	15186	15186
query12	150	105	100	100
query13	1577	474	452	452
query14	9754	7490	7566	7490
query15	254	172	178	172
query16	8006	477	512	477
query17	1542	619	588	588
query18	2211	313	317	313
query19	374	156	165	156
query20	123	112	111	111
query21	211	114	110	110
query22	4782	4764	4521	4521
query23	34966	34257	34168	34168
query24	10975	2853	2934	2853
query25	607	412	405	405
query26	770	162	166	162
query27	2071	303	311	303
query28	6517	2436	2405	2405
query29	777	451	445	445
query30	306	155	153	153
query31	1015	805	821	805
query32	96	53	60	53
query33	778	303	320	303
query34	903	511	516	511
query35	902	754	729	729
query36	1127	941	965	941
query37	146	91	88	88
query38	4129	3928	3890	3890
query39	1489	1468	1448	1448
query40	206	99	98	98
query41	48	43	44	43
query42	116	96	94	94
query43	527	493	476	476
query44	1237	828	811	811
query45	213	167	166	166
query46	1130	700	711	700
query47	1933	1849	1818	1818
query48	463	347	356	347
query49	968	414	421	414
query50	817	422	408	408
query51	7073	6925	6937	6925
query52	98	85	89	85
query53	255	182	187	182
query54	1169	476	470	470
query55	79	76	81	76
query56	286	278	253	253
query57	1231	1131	1111	1111
query58	237	232	231	231
query59	3112	2955	2865	2865
query60	296	291	279	279
query61	118	106	107	106
query62	863	685	665	665
query63	222	187	190	187
query64	3723	655	625	625
query65	3271	3189	3230	3189
query66	816	310	307	307
query67	15892	15487	15857	15487
query68	4445	576	571	571
query69	499	307	290	290
query70	1203	1061	1140	1061
query71	351	280	273	273
query72	7299	3989	4012	3989
query73	758	343	347	343
query74	10193	8963	8939	8939
query75	3421	2671	2683	2671
query76	2698	938	914	914
query77	636	300	307	300
query78	10379	9626	9537	9537
query79	1695	601	598	598
query80	2450	446	467	446
query81	589	237	235	235
query82	725	136	141	136
query83	296	140	132	132
query84	280	85	74	74
query85	1631	293	292	292
query86	432	295	302	295
query87	4488	4346	4396	4346
query88	3493	2466	2409	2409
query89	410	290	291	290
query90	2121	192	190	190
query91	151	112	113	112
query92	62	45	48	45
query93	1097	548	557	548
query94	1197	301	302	301
query95	367	263	261	261
query96	622	280	288	280
query97	3287	3168	3132	3132
query98	226	202	196	196
query99	1722	1317	1315	1315
Total cold run time: 297957 ms
Total hot run time: 192566 ms

doris-robot · 2024-10-09T03:51:10Z

ClickBench: Total hot run time: 32.24 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit c4b9ed1d1f820c07345f6c538724a93d4e185410, data reload: false

query1	0.05	0.05	0.04
query2	0.07	0.03	0.02
query3	0.23	0.06	0.06
query4	1.64	0.10	0.09
query5	0.52	0.52	0.51
query6	1.14	0.75	0.73
query7	0.01	0.01	0.01
query8	0.04	0.03	0.04
query9	0.57	0.51	0.50
query10	0.54	0.55	0.54
query11	0.14	0.11	0.10
query12	0.13	0.12	0.11
query13	0.60	0.59	0.61
query14	2.68	2.77	2.73
query15	0.89	0.83	0.83
query16	0.39	0.39	0.39
query17	1.06	1.09	1.06
query18	0.19	0.19	0.19
query19	2.00	1.87	2.02
query20	0.01	0.00	0.01
query21	15.36	0.58	0.58
query22	2.66	2.80	1.86
query23	17.07	1.16	0.78
query24	2.58	1.35	0.47
query25	0.15	0.06	0.20
query26	0.46	0.14	0.13
query27	0.04	0.04	0.04
query28	11.30	1.08	1.07
query29	12.57	3.27	3.27
query30	0.25	0.06	0.06
query31	2.86	0.38	0.38
query32	3.26	0.48	0.45
query33	3.03	2.99	3.05
query34	17.12	4.45	4.47
query35	4.60	4.47	4.47
query36	0.70	0.48	0.48
query37	0.08	0.06	0.06
query38	0.04	0.04	0.03
query39	0.03	0.02	0.02
query40	0.16	0.12	0.13
query41	0.08	0.02	0.02
query42	0.04	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 107.37 s
Total hot run time: 32.24 s

xinyiZzz

LGTM

github-actions · 2024-10-09T09:16:23Z

PR approved by at least one committer and no changes requested.

github-actions · 2024-10-09T09:16:26Z

PR approved by anyone and no changes requested.

wangbo

LGTM

…ry context automatically (apache#41549) ## Proposed changes Issue Number: close #xxx  --------- Co-authored-by: yiguolei <yiguolei@gmail.com>

Doris-Extras added 2 commits October 8, 2024 14:30

[refactor](wg&memtracker) using weak ptr to delete memtracker and que…

9f40829

…ry context automatically

f

d3b924c

f

5b45101

f

c4b9ed1

wangbo reviewed Oct 9, 2024

View reviewed changes

Comment thread be/src/runtime/load_channel.cpp

xinyiZzz approved these changes Oct 9, 2024

View reviewed changes

github-actions bot added approved Indicates a PR has been approved by one committer. reviewed labels Oct 9, 2024

wangbo approved these changes Oct 9, 2024

View reviewed changes

yiguolei merged commit 7229aea into apache:master Oct 9, 2024

yiguolei added the dev/3.0.x label Oct 15, 2024

yiguolei mentioned this pull request Oct 19, 2024

[branch-3.0] pick some bugfix #42128

Merged

yiguolei added dev/3.0.3-merged and removed dev/3.0.x labels Oct 19, 2024

Conversation

yiguolei commented Oct 8, 2024

Proposed changes

Uh oh!

doris-robot commented Oct 8, 2024

Uh oh!

yiguolei commented Oct 8, 2024

Uh oh!

github-actions bot commented Oct 8, 2024

Uh oh!

doris-robot commented Oct 8, 2024

Uh oh!

doris-robot commented Oct 8, 2024

Uh oh!

yiguolei commented Oct 8, 2024

Uh oh!

github-actions bot commented Oct 8, 2024

Uh oh!

doris-robot commented Oct 8, 2024

Uh oh!

doris-robot commented Oct 8, 2024

Uh oh!

doris-robot commented Oct 8, 2024

Uh oh!

doris-robot commented Oct 8, 2024

Uh oh!

yiguolei commented Oct 9, 2024

Uh oh!

github-actions bot commented Oct 9, 2024

Uh oh!

doris-robot commented Oct 9, 2024

Uh oh!

doris-robot commented Oct 9, 2024

Uh oh!

doris-robot commented Oct 9, 2024

Uh oh!

doris-robot commented Oct 9, 2024

Uh oh!

Uh oh!

xinyiZzz left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Oct 9, 2024

Uh oh!

github-actions bot commented Oct 9, 2024

Uh oh!

wangbo left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants