Skip to content

[fix](jni) avoid coredump if failed to get jni env (#32950)#33012

Merged
morningman merged 1 commit intoapache:branch-2.0from
morningman:bp32950
Mar 29, 2024
Merged

[fix](jni) avoid coredump if failed to get jni env (#32950)#33012
morningman merged 1 commit intoapache:branch-2.0from
morningman:bp32950

Conversation

@morningman
Copy link
Copy Markdown
Contributor

@morningman morningman commented Mar 29, 2024

bp #32950

@morningman
[fix](jni) avoid coredump if failed to get jni env (apache#32950)
2ee9ca3 
This PR apache#32217 find a problem that may failed to get jni env.
And it did a work around to avoid BE crash.

This PR followup this issue, to avoid BE crash when doing `close()` of JniConnector
if failed to get jni env.

The `close()` method will return error when:
1. Failed to get jni env
2. Failed to release jni resource.

This PR will ignore the first error, and still log fatal for second error
@doris-robot
Copy link
Copy Markdown

doris-robot commented Mar 29, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@morningman
Copy link
Copy Markdown
Contributor Author

morningman commented Mar 29, 2024

run buildall

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Mar 29, 2024

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link
Copy Markdown

doris-robot commented Mar 29, 2024

TPC-H: Total hot run time: 50117 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 2ee9ca37507f61c13a05ef6338260be4005f63b7, data reload: false

------ Round 1 ----------------------------------
q1	17539	4363	4287	4287
q2	2025	147	142	142
q3	10276	1884	1939	1884
q4	10093	1257	1312	1257
q5	8401	3965	3970	3965
q6	232	124	122	122
q7	2046	1603	1583	1583
q8	9294	2714	2715	2714
q9	10843	11288	10598	10598
q10	8632	3522	3505	3505
q11	411	249	250	249
q12	462	292	297	292
q13	18365	4007	4024	4007
q14	354	320	332	320
q15	507	454	456	454
q16	706	596	599	596
q17	1127	968	1011	968
q18	7332	6904	6844	6844
q19	1700	1571	1520	1520
q20	545	299	321	299
q21	4500	4123	4110	4110
q22	494	401	407	401
Total cold run time: 115884 ms
Total hot run time: 50117 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4361	4347	4272	4272
q2	321	221	221	221
q3	4164	4185	4159	4159
q4	2751	2753	2739	2739
q5	7315	7174	7205	7174
q6	238	121	119	119
q7	3241	2858	2828	2828
q8	4354	4466	4471	4466
q9	17166	17019	17024	17019
q10	4274	4240	4284	4240
q11	751	659	673	659
q12	1031	868	844	844
q13	5817	3746	3751	3746
q14	453	430	417	417
q15	503	455	458	455
q16	749	697	701	697
q17	3807	3830	3818	3818
q18	8730	8761	8870	8761
q19	1713	1710	1632	1632
q20	2343	2115	2125	2115
q21	8481	8500	8414	8414
q22	1032	935	964	935
Total cold run time: 83595 ms
Total hot run time: 79730 ms

@doris-robot
Copy link
Copy Markdown

doris-robot commented Mar 29, 2024

TeamCity be ut coverage result:
Function Coverage: 37.80% (8048/21293)
Line Coverage: 29.47% (65729/223070)
Region Coverage: 28.93% (33827/116923)
Branch Coverage: 24.78% (17367/70078)
Coverage Report: http://coverage.selectdb-in.cc/coverage/2ee9ca37507f61c13a05ef6338260be4005f63b7_2ee9ca37507f61c13a05ef6338260be4005f63b7/report/index.html

@doris-robot
Copy link
Copy Markdown

doris-robot commented Mar 29, 2024

TPC-DS: Total hot run time: 200764 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 2ee9ca37507f61c13a05ef6338260be4005f63b7, data reload: false

query1	937	397	377	377
query2	6517	2202	2041	2041
query3	6918	203	194	194
query4	21209	18033	17924	17924
query5	19730	6549	6567	6549
query6	300	217	240	217
query7	4163	306	298	298
query8	257	251	230	230
query9	3167	2754	2657	2657
query10	425	294	311	294
query11	11403	10753	10649	10649
query12	120	77	70	70
query13	5578	681	612	612
query14	17308	13360	13274	13274
query15	355	226	231	226
query16	6447	265	255	255
query17	1577	1467	882	882
query18	2280	405	397	397
query19	211	148	145	145
query20	76	75	73	73
query21	198	100	90	90
query22	5351	5131	5139	5131
query23	32591	32043	31855	31855
query24	7046	6447	6493	6447
query25	516	448	424	424
query26	526	170	158	158
query27	1908	291	286	286
query28	6155	2264	2231	2231
query29	2903	2808	2801	2801
query30	240	161	160	160
query31	918	732	724	724
query32	71	56	58	56
query33	390	250	254	250
query34	859	454	483	454
query35	1114	905	861	861
query36	1452	1148	1177	1148
query37	88	65	62	62
query38	3081	2890	2915	2890
query39	1376	1336	1320	1320
query40	194	97	95	95
query41	37	32	37	32
query42	82	86	86	86
query43	686	668	570	570
query44	1154	712	718	712
query45	242	219	223	219
query46	1249	966	969	966
query47	1960	1637	2016	1637
query48	974	675	673	673
query49	605	368	362	362
query50	862	606	595	595
query51	4781	4626	4630	4626
query52	93	79	72	72
query53	448	315	314	314
query54	2637	2496	2478	2478
query55	99	80	84	80
query56	209	198	199	198
query57	1140	1042	1115	1042
query58	213	197	174	174
query59	3386	3223	3098	3098
query60	213	193	188	188
query61	83	84	81	81
query62	844	471	461	461
query63	479	335	332	332
query64	2506	1460	1445	1445
query65	3671	3551	3562	3551
query66	818	386	366	366
query67	15421	14846	16927	14846
query68	10063	663	680	663
query69	574	343	338	338
query70	1719	1476	1391	1391
query71	403	314	307	307
query72	6607	3380	3395	3380
query73	738	313	313	313
query74	6200	5919	5825	5825
query75	5364	3717	3763	3717
query76	6456	1163	1218	1163
query77	1119	253	243	243
query78	12509	11644	12231	11644
query79	8561	651	617	617
query80	825	394	383	383
query81	451	232	231	231
query82	801	99	97	97
query83	177	134	125	125
query84	250	67	69	67
query85	856	275	275	275
query86	330	280	301	280
query87	3235	3068	3068	3068
query88	4273	2312	2317	2312
query89	482	311	288	288
query90	1912	199	215	199
query91	149	125	112	112
query92	53	55	50	50
query93	6100	586	590	586
query94	691	206	201	201
query95	1133	1060	1052	1052
query96	634	325	322	322
query97	6446	6401	6444	6401
query98	193	178	175	175
query99	2845	934	928	928
Total cold run time: 312996 ms
Total hot run time: 200764 ms

@doris-robot
Copy link
Copy Markdown

doris-robot commented Mar 29, 2024

ClickBench: Total hot run time: 31.24 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 2ee9ca37507f61c13a05ef6338260be4005f63b7, data reload: false

query1	0.03	0.02	0.01
query2	0.07	0.02	0.02
query3	0.25	0.04	0.05
query4	1.79	0.06	0.06
query5	0.53	0.52	0.51
query6	1.28	0.62	0.62
query7	0.01	0.01	0.01
query8	0.04	0.02	0.02
query9	0.53	0.47	0.48
query10	0.53	0.53	0.55
query11	0.12	0.09	0.08
query12	0.13	0.10	0.09
query13	0.63	0.61	0.61
query14	0.79	0.80	0.80
query15	0.78	0.76	0.77
query16	0.38	0.37	0.37
query17	1.00	1.03	1.00
query18	0.23	0.26	0.25
query19	1.88	1.82	1.84
query20	0.02	0.01	0.01
query21	15.47	0.58	0.58
query22	2.06	2.38	1.80
query23	17.42	1.10	1.02
query24	7.34	1.33	1.06
query25	0.38	0.07	0.07
query26	0.80	0.18	0.17
query27	0.03	0.04	0.04
query28	5.49	0.76	0.73
query29	12.63	2.34	2.36
query30	0.60	0.52	0.51
query31	2.81	0.40	0.38
query32	3.37	0.49	0.49
query33	3.05	3.12	3.08
query34	15.24	4.80	4.79
query35	4.86	4.83	4.84
query36	1.08	1.02	1.00
query37	0.06	0.04	0.05
query38	0.04	0.02	0.02
query39	0.02	0.02	0.02
query40	0.17	0.14	0.14
query41	0.07	0.02	0.02
query42	0.02	0.02	0.02
query43	0.02	0.01	0.02
Total cold run time: 104.05 s
Total hot run time: 31.24 s

@doris-robot
Copy link
Copy Markdown

doris-robot commented Mar 29, 2024

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 2ee9ca37507f61c13a05ef6338260be4005f63b7 with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet:      32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select:       20.0 seconds inserted 10000000 Rows, about 500K ops/s

@morningman morningman merged commit 2345943 into apache:branch-2.0 Mar 29, 2024
mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024
@morningman
[fix](jni) avoid coredump if failed to get jni env (apache#32950) (ap…
61677c7 
…ache#33012)

bp apache#32950
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@morningman @doris-robot