Skip to content

[fix](microbench) Fix some file cache microbench compile bug#49269

Merged
BiteTheDDDDt merged 2 commits intoapache:masterfrom
deardeng:fix-comp
Apr 1, 2025
Merged

[fix](microbench) Fix some file cache microbench compile bug#49269
BiteTheDDDDt merged 2 commits intoapache:masterfrom
deardeng:fix-comp

Conversation

@deardeng
Copy link
Copy Markdown
Collaborator

@deardeng deardeng commented Mar 19, 2025

What problem does this PR solve?

  1. fix micro BUILD_FILE_CACHE_MICROBENCH_TOOL not work in file_cache_microbench.cpp
  2. fix MircobenchS3FileWriter typo error
  3. ./build.sh default not compile file_cache_microbench
  4. fix , exit coredump
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x00007f54c898a550 in __pthread_timedjoin_ex () from /lib64/libpthread.so.0
[Current thread is 1 (Thread 0x7f54cafad940 (LWP 2810860))]
(gdb) bt
#0  0x00007f54c898a550 in __pthread_timedjoin_ex () from /lib64/libpthread.so.0
#1  0x000055eb6a7ac513 in __gthread_join (__value_ptr=0x0, __threadid=<optimized out>) at /data/gcc-11.1.0/build/x86_64-pc-linux-gnu/libstdc++-v3/include/x86_64-pc-linux-gnu/bits/gthr-default.h:669
#2  std::thread::join (this=0x55eb6eda1ce0) at ../../../../../libstdc++-v3/src/c++11/thread.cc:112
#3  0x000055eb5cb548bb in ThreadPool::~ThreadPool (this=this@entry=0x7fff65768480) 
#4  0x000055eb5cb4aebe in JobManager::~JobManager (this=this@entry=0x7fff65768420) 
#5  0x000055eb5cb49a14 in main (argc=1, argv=<optimized out>) 

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen
Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@deardeng
Copy link
Copy Markdown
Collaborator Author

run buildall

@doris-robot
Copy link
Copy Markdown

TPC-H: Total hot run time: 32365 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 0cfc601458fa9ff56003cb8077777b8ee218a7a3, data reload: false

------ Round 1 ----------------------------------
q1	24004	5125	4995	4995
q2	2043	288	174	174
q3	10403	1209	690	690
q4	10222	1005	534	534
q5	7584	2402	2329	2329
q6	180	160	130	130
q7	898	736	611	611
q8	9292	1275	1054	1054
q9	4873	4715	4724	4715
q10	6815	2324	1893	1893
q11	460	279	264	264
q12	352	360	218	218
q13	19811	3715	3127	3127
q14	229	236	210	210
q15	544	480	490	480
q16	633	624	586	586
q17	581	858	342	342
q18	7057	6555	6365	6365
q19	1201	943	550	550
q20	323	335	204	204
q21	2850	2114	1918	1918
q22	1054	1020	976	976
Total cold run time: 111409 ms
Total hot run time: 32365 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5167	5141	5155	5141
q2	243	336	229	229
q3	2160	2738	2353	2353
q4	1454	1897	1421	1421
q5	4407	4387	4557	4387
q6	225	170	132	132
q7	2075	1983	1861	1861
q8	2717	2720	2706	2706
q9	7604	7316	7234	7234
q10	2984	3183	2639	2639
q11	573	501	492	492
q12	665	818	667	667
q13	3625	4017	3322	3322
q14	284	294	291	291
q15	538	499	502	499
q16	645	680	658	658
q17	1147	1570	1316	1316
q18	7807	7611	7480	7480
q19	799	798	876	798
q20	1988	2030	1852	1852
q21	5354	4727	4807	4727
q22	1074	1056	1037	1037
Total cold run time: 53535 ms
Total hot run time: 51242 ms

@doris-robot
Copy link
Copy Markdown

TPC-DS: Total hot run time: 192464 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 0cfc601458fa9ff56003cb8077777b8ee218a7a3, data reload: false

query1	1393	1047	1048	1047
query2	6494	1971	1944	1944
query3	11012	4511	4377	4377
query4	53288	25459	23003	23003
query5	5309	570	474	474
query6	358	186	173	173
query7	5149	500	292	292
query8	331	236	222	222
query9	6619	2597	2619	2597
query10	413	311	260	260
query11	15292	15198	14872	14872
query12	162	112	106	106
query13	1136	533	379	379
query14	10179	6989	6757	6757
query15	199	203	176	176
query16	7008	645	518	518
query17	1073	713	566	566
query18	1551	400	308	308
query19	191	199	160	160
query20	129	127	120	120
query21	207	123	103	103
query22	4428	4691	4458	4458
query23	33940	33523	33270	33270
query24	5735	2393	2401	2393
query25	470	456	391	391
query26	647	272	160	160
query27	1733	512	335	335
query28	2782	2458	2445	2445
query29	627	577	473	473
query30	275	236	197	197
query31	876	868	790	790
query32	76	62	64	62
query33	476	363	322	322
query34	751	850	526	526
query35	805	832	759	759
query36	967	1004	910	910
query37	129	108	85	85
query38	4148	4244	4103	4103
query39	1506	1452	1450	1450
query40	218	133	112	112
query41	59	57	53	53
query42	138	108	110	108
query43	512	539	515	515
query44	1326	801	810	801
query45	180	177	167	167
query46	842	1055	662	662
query47	1858	1881	1775	1775
query48	423	413	308	308
query49	692	533	436	436
query50	717	769	440	440
query51	4256	4348	4301	4301
query52	110	100	97	97
query53	236	260	192	192
query54	491	508	424	424
query55	86	81	84	81
query56	279	276	255	255
query57	1151	1182	1182	1182
query58	252	254	237	237
query59	3019	3050	2746	2746
query60	280	278	269	269
query61	124	116	130	116
query62	729	756	703	703
query63	232	191	190	190
query64	1399	1012	740	740
query65	4535	4486	4443	4443
query66	860	403	297	297
query67	15919	15691	15455	15455
query68	7145	870	497	497
query69	537	313	259	259
query70	1215	1142	1117	1117
query71	474	321	282	282
query72	5545	3589	3711	3589
query73	1211	738	361	361
query74	8903	9179	8948	8948
query75	3639	3148	2731	2731
query76	4257	1171	748	748
query77	571	360	281	281
query78	10135	10150	9283	9283
query79	2409	827	582	582
query80	619	520	453	453
query81	492	256	220	220
query82	699	125	96	96
query83	179	239	151	151
query84	293	94	79	79
query85	797	348	310	310
query86	406	302	283	283
query87	4514	4581	4352	4352
query88	3604	2305	2293	2293
query89	423	316	285	285
query90	1805	216	239	216
query91	140	139	108	108
query92	70	61	58	58
query93	1846	1067	572	572
query94	650	385	271	271
query95	357	267	267	267
query96	482	576	276	276
query97	3327	3368	3344	3344
query98	224	210	199	199
query99	1636	1438	1265	1265
Total cold run time: 297391 ms
Total hot run time: 192464 ms

@doris-robot
Copy link
Copy Markdown

ClickBench: Total hot run time: 31.41 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 0cfc601458fa9ff56003cb8077777b8ee218a7a3, data reload: false

query1	0.05	0.04	0.04
query2	0.12	0.11	0.10
query3	0.24	0.18	0.19
query4	1.59	0.19	0.20
query5	0.61	0.57	0.59
query6	1.18	0.70	0.71
query7	0.03	0.02	0.02
query8	0.04	0.04	0.04
query9	0.58	0.52	0.53
query10	0.58	0.59	0.58
query11	0.16	0.11	0.11
query12	0.14	0.10	0.11
query13	0.62	0.60	0.60
query14	2.65	2.81	2.68
query15	0.93	0.86	0.85
query16	0.38	0.37	0.37
query17	1.02	1.05	1.06
query18	0.21	0.19	0.19
query19	1.90	1.88	1.82
query20	0.01	0.01	0.01
query21	15.35	0.92	0.54
query22	0.75	1.23	0.70
query23	14.82	1.36	0.62
query24	7.08	1.67	1.13
query25	0.50	0.13	0.12
query26	0.56	0.16	0.13
query27	0.06	0.05	0.05
query28	10.02	0.84	0.41
query29	12.54	3.96	3.31
query30	0.25	0.09	0.06
query31	2.82	0.58	0.38
query32	3.23	0.55	0.46
query33	2.99	3.07	2.98
query34	15.74	5.14	4.52
query35	4.51	4.49	4.47
query36	0.66	0.50	0.48
query37	0.08	0.06	0.07
query38	0.06	0.04	0.04
query39	0.04	0.03	0.02
query40	0.18	0.13	0.13
query41	0.08	0.03	0.03
query42	0.03	0.02	0.02
query43	0.04	0.04	0.03
Total cold run time: 105.43 s
Total hot run time: 31.41 s

@deardeng
Copy link
Copy Markdown
Collaborator Author

run buildall

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Mar 19, 2025
@github-actions
Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Copy Markdown
Contributor

PR approved by anyone and no changes requested.

@doris-robot
Copy link
Copy Markdown

TPC-H: Total hot run time: 32263 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4cfb4b18ef18b705bac1017ebc29c6dd54feb28d, data reload: false

------ Round 1 ----------------------------------
q1	24341	5044	5029	5029
q2	2041	302	170	170
q3	10395	1256	711	711
q4	10231	1031	538	538
q5	7495	2445	2377	2377
q6	196	166	136	136
q7	925	750	626	626
q8	9313	1314	1001	1001
q9	4951	4606	4757	4606
q10	6810	2294	1910	1910
q11	468	269	249	249
q12	338	354	221	221
q13	17773	3673	3101	3101
q14	227	229	213	213
q15	541	488	477	477
q16	625	637	574	574
q17	574	853	357	357
q18	6851	6468	6245	6245
q19	1211	967	556	556
q20	328	345	204	204
q21	2898	2173	1970	1970
q22	1007	1002	992	992
Total cold run time: 109539 ms
Total hot run time: 32263 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5218	5105	5127	5105
q2	250	340	232	232
q3	2191	2658	2277	2277
q4	1436	1834	1374	1374
q5	4242	4128	4396	4128
q6	217	167	128	128
q7	2064	1913	1799	1799
q8	2626	2566	2552	2552
q9	7254	7180	7052	7052
q10	3008	3212	2824	2824
q11	586	508	490	490
q12	668	810	635	635
q13	3534	3880	3260	3260
q14	281	302	278	278
q15	519	488	476	476
q16	645	688	633	633
q17	1140	1631	1326	1326
q18	7657	7625	7322	7322
q19	811	821	988	821
q20	1961	2000	1893	1893
q21	5502	4925	4577	4577
q22	1077	1009	988	988
Total cold run time: 52887 ms
Total hot run time: 50170 ms

@doris-robot
Copy link
Copy Markdown

TPC-DS: Total hot run time: 184614 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4cfb4b18ef18b705bac1017ebc29c6dd54feb28d, data reload: false

query1	1018	494	469	469
query2	6547	1973	1977	1973
query3	6811	222	218	218
query4	26807	23800	22917	22917
query5	4370	644	482	482
query6	293	190	176	176
query7	4602	482	302	302
query8	293	237	218	218
query9	8596	2619	2627	2619
query10	478	315	263	263
query11	15540	15258	14987	14987
query12	153	111	109	109
query13	1656	539	394	394
query14	9627	6107	6335	6107
query15	203	192	173	173
query16	7082	620	457	457
query17	1212	717	567	567
query18	1839	388	311	311
query19	199	183	172	172
query20	122	119	119	119
query21	216	129	103	103
query22	4353	4263	3930	3930
query23	34092	33077	33004	33004
query24	7710	2362	2363	2362
query25	516	459	387	387
query26	1225	265	154	154
query27	2112	523	336	336
query28	3874	2405	2395	2395
query29	651	545	422	422
query30	286	213	187	187
query31	952	870	764	764
query32	74	65	65	65
query33	558	382	297	297
query34	813	840	503	503
query35	776	826	731	731
query36	952	965	877	877
query37	119	108	78	78
query38	4140	4293	4133	4133
query39	1456	1418	1373	1373
query40	211	121	107	107
query41	56	52	51	51
query42	117	102	98	98
query43	512	516	490	490
query44	1286	785	774	774
query45	174	174	164	164
query46	829	1013	615	615
query47	1763	1781	1683	1683
query48	380	413	314	314
query49	803	522	421	421
query50	676	710	402	402
query51	4247	4249	4138	4138
query52	131	108	95	95
query53	223	255	188	188
query54	485	486	426	426
query55	78	78	81	78
query56	252	272	240	240
query57	1087	1151	1076	1076
query58	247	250	230	230
query59	2573	2770	2759	2759
query60	279	274	252	252
query61	123	121	117	117
query62	808	752	646	646
query63	225	188	189	188
query64	4330	991	659	659
query65	4409	4338	4379	4338
query66	1075	399	293	293
query67	15620	15662	15262	15262
query68	7973	900	507	507
query69	468	307	265	265
query70	1179	1113	1115	1113
query71	466	290	279	279
query72	5571	3609	3800	3609
query73	760	733	350	350
query74	9073	9111	8683	8683
query75	3824	3173	2704	2704
query76	3690	1202	762	762
query77	780	390	286	286
query78	10162	10338	9261	9261
query79	3113	813	568	568
query80	678	506	449	449
query81	496	256	219	219
query82	733	129	95	95
query83	184	175	164	164
query84	244	86	74	74
query85	810	370	309	309
query86	393	305	304	304
query87	4543	4483	4309	4309
query88	3880	2249	2288	2249
query89	397	307	293	293
query90	1887	214	224	214
query91	138	142	109	109
query92	76	65	56	56
query93	2438	1049	568	568
query94	687	404	306	306
query95	351	268	252	252
query96	484	563	273	273
query97	3305	3408	3279	3279
query98	237	204	198	198
query99	1305	1414	1269	1269
Total cold run time: 275647 ms
Total hot run time: 184614 ms

@doris-robot
Copy link
Copy Markdown

ClickBench: Total hot run time: 30.78 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 4cfb4b18ef18b705bac1017ebc29c6dd54feb28d, data reload: false

query1	0.04	0.03	0.03
query2	0.13	0.10	0.11
query3	0.25	0.19	0.19
query4	1.59	0.20	0.19
query5	0.58	0.58	0.58
query6	1.20	0.71	0.72
query7	0.02	0.01	0.02
query8	0.04	0.03	0.03
query9	0.58	0.52	0.54
query10	0.57	0.58	0.57
query11	0.16	0.11	0.11
query12	0.15	0.12	0.11
query13	0.62	0.59	0.60
query14	2.68	2.69	2.70
query15	0.92	0.84	0.83
query16	0.39	0.38	0.38
query17	1.03	1.06	1.01
query18	0.20	0.20	0.19
query19	1.85	1.88	1.89
query20	0.01	0.01	0.01
query21	15.39	0.94	0.54
query22	0.74	1.18	0.67
query23	14.89	1.40	0.61
query24	7.11	0.85	0.52
query25	0.50	0.30	0.08
query26	0.64	0.15	0.13
query27	0.05	0.05	0.05
query28	8.95	0.93	0.44
query29	12.55	4.01	3.32
query30	0.24	0.08	0.06
query31	2.83	0.60	0.38
query32	3.23	0.55	0.46
query33	2.97	3.03	3.07
query34	15.77	5.10	4.47
query35	4.53	4.51	4.55
query36	0.68	0.51	0.48
query37	0.09	0.06	0.06
query38	0.05	0.03	0.03
query39	0.03	0.03	0.02
query40	0.17	0.13	0.12
query41	0.08	0.02	0.02
query42	0.03	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 104.57 s
Total hot run time: 30.78 s

@doris-robot
Copy link
Copy Markdown

BE UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 48.81% (13072/26784)
Line Coverage 38.38% (112716/293708)
Region Coverage 37.18% (57309/154140)
Branch Coverage 32.28% (28811/89240)

Copy link
Copy Markdown
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Copy Markdown
Contributor

@zclllyybb zclllyybb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Copy Markdown
Contributor

@xinyiZzz xinyiZzz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@BiteTheDDDDt BiteTheDDDDt merged commit c5910ea into apache:master Apr 1, 2025
30 of 32 checks passed
github-actions bot pushed a commit that referenced this pull request Apr 1, 2025
### What problem does this PR solve?

1. fix micro BUILD_FILE_CACHE_MICROBENCH_TOOL not work in
file_cache_microbench.cpp
2. fix MircobenchS3FileWriter typo error
3. ./build.sh default not compile file_cache_microbench
4. fix , exit coredump
```
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x00007f54c898a550 in __pthread_timedjoin_ex () from /lib64/libpthread.so.0
[Current thread is 1 (Thread 0x7f54cafad940 (LWP 2810860))]
(gdb) bt
#0  0x00007f54c898a550 in __pthread_timedjoin_ex () from /lib64/libpthread.so.0
#1  0x000055eb6a7ac513 in __gthread_join (__value_ptr=0x0, __threadid=<optimized out>) at /data/gcc-11.1.0/build/x86_64-pc-linux-gnu/libstdc++-v3/include/x86_64-pc-linux-gnu/bits/gthr-default.h:669
#2  std::thread::join (this=0x55eb6eda1ce0) at ../../../../../libstdc++-v3/src/c++11/thread.cc:112
#3  0x000055eb5cb548bb in ThreadPool::~ThreadPool (this=this@entry=0x7fff65768480) 
#4  0x000055eb5cb4aebe in JobManager::~JobManager (this=this@entry=0x7fff65768420) 
#5  0x000055eb5cb49a14 in main (argc=1, argv=<optimized out>) 
```

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

### Release note

None

### Check List (For Author)

- Test <!-- At least one of them must be included. -->
    - [ ] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [x] No need to test or manual test. Explain why:
- [x] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [x] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [x] No.
- [ ] Yes. <!-- Add document PR link here. eg:
apache/doris-website#1214 -->

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
dataroaring pushed a commit that referenced this pull request Apr 8, 2025
…bug #49269 (#49720)

Cherry-picked from #49269

Co-authored-by: deardeng <dengxin@selectdb.com>
koarz pushed a commit to koarz/doris that referenced this pull request Jun 4, 2025
…49269)

### What problem does this PR solve?

1. fix micro BUILD_FILE_CACHE_MICROBENCH_TOOL not work in
file_cache_microbench.cpp
2. fix MircobenchS3FileWriter typo error
3. ./build.sh default not compile file_cache_microbench
4. fix , exit coredump
```
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x00007f54c898a550 in __pthread_timedjoin_ex () from /lib64/libpthread.so.0
[Current thread is 1 (Thread 0x7f54cafad940 (LWP 2810860))]
(gdb) bt
#0  0x00007f54c898a550 in __pthread_timedjoin_ex () from /lib64/libpthread.so.0
apache#1  0x000055eb6a7ac513 in __gthread_join (__value_ptr=0x0, __threadid=<optimized out>) at /data/gcc-11.1.0/build/x86_64-pc-linux-gnu/libstdc++-v3/include/x86_64-pc-linux-gnu/bits/gthr-default.h:669
apache#2  std::thread::join (this=0x55eb6eda1ce0) at ../../../../../libstdc++-v3/src/c++11/thread.cc:112
apache#3  0x000055eb5cb548bb in ThreadPool::~ThreadPool (this=this@entry=0x7fff65768480) 
apache#4  0x000055eb5cb4aebe in JobManager::~JobManager (this=this@entry=0x7fff65768420) 
apache#5  0x000055eb5cb49a14 in main (argc=1, argv=<optimized out>) 
```

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

### Release note

None

### Check List (For Author)

- Test <!-- At least one of them must be included. -->
    - [ ] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [x] No need to test or manual test. Explain why:
- [x] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [x] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [x] No.
- [ ] Yes. <!-- Add document PR link here. eg:
apache/doris-website#1214 -->

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. dev/3.0.5-merged reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants