Skip to content

[bug](function)fix json_replace check return type error#37014

Merged
HappenLee merged 3 commits intoapache:masterfrom
zhangstar333:function_json_replace
Jul 5, 2024
Merged

[bug](function)fix json_replace check return type error#37014
HappenLee merged 3 commits intoapache:masterfrom
zhangstar333:function_json_replace

Conversation

@zhangstar333
Copy link
Copy Markdown
Contributor

@zhangstar333 zhangstar333 commented Jun 28, 2024 •
edited
Loading

Proposed changes

  1. fix the return type dcheck error:
mysql [test]>select (json_replace(a, '$.fparam.nested_2', "qwe")) from json_table_2 limit 1;
ERROR 1105 (HY000): errCode = 2, detailMessage = (10.16.10.8)[INTERNAL_ERROR]Function json_replace get failed, expr is VectorizedFnCall[json_replace](arguments=a, String, String, String,return=Nullable(String)) and return type is Nullable(String).
  1. improve the json_replace/json_insert/json_set function execute of not convert const column, test about could faster 1s on 1000w table rows

@doris-robot
Copy link
Copy Markdown

doris-robot commented Jun 28, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@zhangstar333
Copy link
Copy Markdown
Contributor Author

zhangstar333 commented Jun 28, 2024

run buildall

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Jun 28, 2024

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link
Copy Markdown

doris-robot commented Jun 28, 2024

TPC-H: Total hot run time: 39973 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4286df2a32dcc8645230a03863ee659e1c4fdf0d, data reload: false

------ Round 1 ----------------------------------
q1	17894	4362	4372	4362
q2	2362	205	189	189
q3	10457	1248	1108	1108
q4	10209	828	825	825
q5	7555	2603	2629	2603
q6	217	132	130	130
q7	929	604	604	604
q8	9227	2068	2034	2034
q9	8781	6456	6469	6456
q10	8916	3757	3746	3746
q11	470	234	230	230
q12	399	231	230	230
q13	17777	3005	2991	2991
q14	260	229	220	220
q15	522	484	482	482
q16	489	376	370	370
q17	964	748	700	700
q18	8151	7483	7390	7390
q19	8010	1501	1456	1456
q20	646	324	330	324
q21	4822	3185	3827	3185
q22	398	343	338	338
Total cold run time: 119455 ms
Total hot run time: 39973 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4478	4281	4233	4233
q2	380	264	262	262
q3	2961	2753	2719	2719
q4	1863	1526	1588	1526
q5	5261	5245	5263	5245
q6	211	126	124	124
q7	2101	1758	1708	1708
q8	3184	3314	3389	3314
q9	8317	8298	8280	8280
q10	3867	3591	3634	3591
q11	569	488	497	488
q12	791	648	606	606
q13	17489	2967	2992	2967
q14	291	256	257	256
q15	526	468	469	468
q16	463	410	429	410
q17	1774	1482	1462	1462
q18	7572	7452	7368	7368
q19	1666	1483	1654	1483
q20	1963	1791	1773	1773
q21	4747	4626	4737	4626
q22	604	530	531	530
Total cold run time: 71078 ms
Total hot run time: 53439 ms

@doris-robot
Copy link
Copy Markdown

doris-robot commented Jun 28, 2024

TPC-DS: Total hot run time: 172721 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4286df2a32dcc8645230a03863ee659e1c4fdf0d, data reload: false

query1	943	380	378	378
query2	6457	2262	2294	2262
query3	6655	204	215	204
query4	19296	17426	17057	17057
query5	4194	482	474	474
query6	259	170	163	163
query7	4592	301	292	292
query8	299	279	283	279
query9	8422	2449	2448	2448
query10	607	320	307	307
query11	10575	10030	10037	10030
query12	143	86	86	86
query13	1646	381	376	376
query14	10160	7493	7672	7493
query15	249	186	191	186
query16	8003	275	275	275
query17	1877	555	533	533
query18	2015	278	283	278
query19	211	153	155	153
query20	93	84	86	84
query21	219	129	134	129
query22	4228	4060	4024	4024
query23	33678	33093	33228	33093
query24	11321	2910	2839	2839
query25	655	377	374	374
query26	1576	153	159	153
query27	2906	311	320	311
query28	7525	2122	2113	2113
query29	1006	629	625	625
query30	290	154	157	154
query31	975	743	749	743
query32	96	54	57	54
query33	786	300	294	294
query34	928	457	467	457
query35	760	633	711	633
query36	1092	970	964	964
query37	156	69	73	69
query38	2872	2731	2739	2731
query39	861	822	795	795
query40	279	123	119	119
query41	59	52	50	50
query42	121	99	101	99
query43	595	549	540	540
query44	1242	751	738	738
query45	202	168	169	168
query46	1055	711	736	711
query47	1847	1750	1792	1750
query48	368	293	298	293
query49	1138	403	400	400
query50	765	375	380	375
query51	6876	6801	6725	6725
query52	106	92	95	92
query53	352	287	291	287
query54	984	439	440	439
query55	75	71	75	71
query56	274	272	261	261
query57	1136	1064	1051	1051
query58	248	231	232	231
query59	3578	2923	3199	2923
query60	316	271	264	264
query61	90	88	91	88
query62	653	436	464	436
query63	321	289	283	283
query64	9839	2208	1748	1748
query65	3172	3105	3072	3072
query66	1367	332	321	321
query67	15457	14869	14942	14869
query68	4691	550	589	550
query69	558	355	340	340
query70	1210	1114	1161	1114
query71	414	270	263	263
query72	7905	5203	5302	5203
query73	757	322	318	318
query74	5929	5451	5455	5451
query75	3774	2605	2599	2599
query76	2961	949	886	886
query77	672	298	298	298
query78	10548	9889	9778	9778
query79	3062	514	514	514
query80	1464	489	464	464
query81	556	218	221	218
query82	1132	100	103	100
query83	479	173	168	168
query84	274	92	85	85
query85	1586	275	257	257
query86	495	329	321	321
query87	3301	3123	3092	3092
query88	3944	2356	2359	2356
query89	475	385	368	368
query90	1877	183	186	183
query91	133	99	99	99
query92	59	49	52	49
query93	2731	511	515	511
query94	1326	182	230	182
query95	399	304	310	304
query96	606	280	276	276
query97	3222	3046	3053	3046
query98	223	202	197	197
query99	1215	814	854	814
Total cold run time: 279119 ms
Total hot run time: 172721 ms

@doris-robot
Copy link
Copy Markdown

doris-robot commented Jun 28, 2024

ClickBench: Total hot run time: 30.28 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 4286df2a32dcc8645230a03863ee659e1c4fdf0d, data reload: false

query1	0.05	0.03	0.03
query2	0.08	0.04	0.04
query3	0.23	0.05	0.05
query4	1.68	0.07	0.07
query5	0.51	0.47	0.50
query6	1.13	0.74	0.72
query7	0.02	0.01	0.02
query8	0.04	0.04	0.04
query9	0.56	0.50	0.51
query10	0.55	0.56	0.54
query11	0.15	0.11	0.12
query12	0.14	0.12	0.12
query13	0.59	0.59	0.59
query14	0.77	0.76	0.80
query15	0.81	0.81	0.80
query16	0.36	0.37	0.36
query17	0.97	0.96	0.96
query18	0.23	0.23	0.25
query19	1.78	1.67	1.72
query20	0.02	0.01	0.01
query21	15.83	0.75	0.65
query22	4.51	7.82	1.89
query23	18.26	1.31	1.29
query24	2.09	0.24	0.23
query25	0.15	0.08	0.09
query26	0.26	0.17	0.18
query27	0.08	0.09	0.08
query28	13.24	1.01	0.98
query29	12.66	3.31	3.29
query30	0.25	0.06	0.05
query31	2.86	0.39	0.37
query32	3.29	0.46	0.47
query33	2.87	2.90	2.84
query34	17.10	4.40	4.36
query35	4.54	4.40	4.47
query36	0.63	0.47	0.48
query37	0.18	0.15	0.15
query38	0.15	0.15	0.14
query39	0.04	0.03	0.03
query40	0.17	0.14	0.14
query41	0.08	0.05	0.04
query42	0.06	0.05	0.05
query43	0.04	0.04	0.04
Total cold run time: 110.01 s
Total hot run time: 30.28 s

HappenLee
HappenLee reviewed Jul 1, 2024
View reviewed changes
Comment thread be/src/vec/functions/function_json.cpp
@zhangstar333
Copy link
Copy Markdown
Contributor Author

zhangstar333 commented Jul 1, 2024

run buildall

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Jul 1, 2024

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link
Copy Markdown

doris-robot commented Jul 1, 2024

TPC-H: Total hot run time: 39773 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 0ca062bab4511a9bad2a41de8a36e0f66a9e439d, data reload: false

------ Round 1 ----------------------------------
q1	17617	4296	4269	4269
q2	2033	191	196	191
q3	10470	1244	1093	1093
q4	10186	763	747	747
q5	7471	2661	2597	2597
q6	229	142	141	141
q7	952	594	608	594
q8	9223	2085	2054	2054
q9	8823	6461	6454	6454
q10	8904	3686	3756	3686
q11	476	238	237	237
q12	491	240	237	237
q13	17781	2983	2979	2979
q14	261	224	235	224
q15	517	485	490	485
q16	513	366	374	366
q17	956	614	701	614
q18	7992	7417	7337	7337
q19	7945	1500	1565	1500
q20	673	321	327	321
q21	4909	3305	3944	3305
q22	400	342	347	342
Total cold run time: 118822 ms
Total hot run time: 39773 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4406	4246	4262	4246
q2	379	279	261	261
q3	3018	2899	2909	2899
q4	1938	1720	1669	1669
q5	5561	5467	5422	5422
q6	230	151	137	137
q7	2204	1868	1861	1861
q8	3221	3412	3463	3412
q9	8688	8693	8727	8693
q10	4052	3803	3774	3774
q11	620	518	498	498
q12	826	618	638	618
q13	15978	3174	3173	3173
q14	295	278	264	264
q15	519	495	478	478
q16	509	453	464	453
q17	1802	1521	1498	1498
q18	8155	7878	7764	7764
q19	1758	1732	1620	1620
q20	2166	1854	1843	1843
q21	10699	5006	4782	4782
q22	629	576	569	569
Total cold run time: 77653 ms
Total hot run time: 55934 ms

@doris-robot
Copy link
Copy Markdown

doris-robot commented Jul 1, 2024

TPC-DS: Total hot run time: 174485 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 0ca062bab4511a9bad2a41de8a36e0f66a9e439d, data reload: false

query1	918	374	390	374
query2	6437	2352	2382	2352
query3	6632	207	213	207
query4	20928	17444	17297	17297
query5	3660	487	468	468
query6	255	175	155	155
query7	4589	307	292	292
query8	306	298	294	294
query9	8639	2365	2353	2353
query10	569	315	289	289
query11	10581	10056	10159	10056
query12	112	87	85	85
query13	1646	375	364	364
query14	10259	7650	7649	7649
query15	241	184	192	184
query16	7982	295	278	278
query17	1994	585	557	557
query18	1971	289	286	286
query19	207	158	159	158
query20	91	84	82	82
query21	212	130	128	128
query22	4853	4056	3995	3995
query23	33815	33631	33471	33471
query24	10992	2848	2803	2803
query25	671	388	374	374
query26	708	161	156	156
query27	2342	325	321	321
query28	5956	2146	2119	2119
query29	899	653	647	647
query30	248	156	168	156
query31	979	766	757	757
query32	94	53	52	52
query33	718	290	285	285
query34	960	475	496	475
query35	788	659	646	646
query36	1118	998	980	980
query37	143	81	84	81
query38	3008	2931	2819	2819
query39	897	869	814	814
query40	205	133	128	128
query41	55	55	52	52
query42	113	100	107	100
query43	598	539	548	539
query44	1115	731	725	725
query45	195	181	163	163
query46	1078	760	714	714
query47	1809	1757	1785	1757
query48	377	289	295	289
query49	830	425	404	404
query50	761	396	395	395
query51	6855	6760	6828	6760
query52	102	91	89	89
query53	350	281	290	281
query54	899	448	438	438
query55	76	74	74	74
query56	279	269	266	266
query57	1107	1042	1059	1042
query58	245	231	239	231
query59	3498	3274	3279	3274
query60	301	269	301	269
query61	92	115	92	92
query62	593	444	457	444
query63	316	288	291	288
query64	8513	2218	1758	1758
query65	3184	3071	3124	3071
query66	737	339	320	320
query67	15252	15190	14729	14729
query68	5421	532	530	530
query69	685	439	347	347
query70	1113	1114	1145	1114
query71	452	270	287	270
query72	9020	5476	5573	5476
query73	778	324	326	324
query74	6051	5560	5619	5560
query75	4395	2634	2608	2608
query76	3645	966	961	961
query77	673	294	297	294
query78	10451	9879	10066	9879
query79	3225	517	518	517
query80	2385	475	480	475
query81	599	219	224	219
query82	1206	113	108	108
query83	307	168	172	168
query84	270	94	85	85
query85	1346	286	300	286
query86	461	327	330	327
query87	3371	3059	3100	3059
query88	4034	2376	2367	2367
query89	491	389	396	389
query90	1843	194	189	189
query91	127	101	102	101
query92	60	52	50	50
query93	3870	499	495	495
query94	1160	187	191	187
query95	405	311	319	311
query96	609	270	262	262
query97	3159	3004	2971	2971
query98	273	194	200	194
query99	1224	844	811	811
Total cold run time: 280062 ms
Total hot run time: 174485 ms

@doris-robot
Copy link
Copy Markdown

doris-robot commented Jul 1, 2024

ClickBench: Total hot run time: 30.36 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 0ca062bab4511a9bad2a41de8a36e0f66a9e439d, data reload: false

query1	0.04	0.04	0.03
query2	0.08	0.04	0.04
query3	0.23	0.04	0.04
query4	1.67	0.07	0.07
query5	0.47	0.48	0.49
query6	1.13	0.72	0.72
query7	0.02	0.01	0.02
query8	0.05	0.04	0.05
query9	0.55	0.48	0.49
query10	0.55	0.53	0.53
query11	0.15	0.11	0.12
query12	0.15	0.12	0.12
query13	0.58	0.59	0.62
query14	0.76	0.79	0.76
query15	0.85	0.81	0.81
query16	0.37	0.38	0.37
query17	0.99	0.97	0.98
query18	0.21	0.24	0.22
query19	1.81	1.76	1.68
query20	0.01	0.01	0.01
query21	15.43	0.73	0.65
query22	4.15	7.52	1.81
query23	18.23	1.44	1.26
query24	2.06	0.24	0.22
query25	0.14	0.11	0.08
query26	0.25	0.19	0.18
query27	0.08	0.08	0.07
query28	13.33	1.03	1.01
query29	12.64	3.31	3.28
query30	0.26	0.06	0.06
query31	2.85	0.39	0.39
query32	3.28	0.48	0.48
query33	2.92	2.87	2.97
query34	17.13	4.46	4.53
query35	4.45	4.40	4.45
query36	0.65	0.45	0.48
query37	0.18	0.16	0.16
query38	0.16	0.15	0.15
query39	0.04	0.04	0.03
query40	0.16	0.15	0.15
query41	0.09	0.04	0.05
query42	0.05	0.05	0.04
query43	0.05	0.04	0.04
Total cold run time: 109.25 s
Total hot run time: 30.36 s

@zhangstar333
Copy link
Copy Markdown
Contributor Author

zhangstar333 commented Jul 1, 2024

run buildall

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Jul 1, 2024

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link
Copy Markdown

doris-robot commented Jul 1, 2024

TPC-H: Total hot run time: 39386 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c9e6208c3d8ccf0ce1622323ddc9bea32c8b8c5b, data reload: false

------ Round 1 ----------------------------------
q1	17627	4337	4231	4231
q2	2010	189	187	187
q3	10479	1178	1068	1068
q4	10192	815	803	803
q5	7486	2631	2596	2596
q6	221	140	136	136
q7	938	590	598	590
q8	9219	2073	2038	2038
q9	8765	6502	6405	6405
q10	9043	3685	3694	3685
q11	447	230	236	230
q12	455	236	227	227
q13	17756	3008	2961	2961
q14	257	233	218	218
q15	511	464	487	464
q16	503	370	369	369
q17	1022	706	659	659
q18	7987	7480	7378	7378
q19	4033	1429	1462	1429
q20	653	328	328	328
q21	4980	3056	3190	3056
q22	391	330	328	328
Total cold run time: 114975 ms
Total hot run time: 39386 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4383	4255	4236	4236
q2	364	275	262	262
q3	2959	2702	2774	2702
q4	1956	1779	1747	1747
q5	5529	5670	5485	5485
q6	224	131	132	131
q7	2207	1804	1847	1804
q8	3279	3419	3414	3414
q9	8686	8680	8758	8680
q10	4188	3856	3908	3856
q11	573	480	487	480
q12	770	642	602	602
q13	16993	3207	3211	3207
q14	302	273	257	257
q15	544	480	501	480
q16	493	443	437	437
q17	1796	1529	1532	1529
q18	8133	7884	7778	7778
q19	1763	1454	1563	1454
q20	2045	1869	1876	1869
q21	8540	4941	4840	4840
q22	612	534	543	534
Total cold run time: 76339 ms
Total hot run time: 55784 ms

@doris-robot
Copy link
Copy Markdown

doris-robot commented Jul 1, 2024

TPC-DS: Total hot run time: 172990 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c9e6208c3d8ccf0ce1622323ddc9bea32c8b8c5b, data reload: false

query1	916	374	371	371
query2	6462	2388	2263	2263
query3	6644	206	213	206
query4	18947	17521	17136	17136
query5	3643	480	460	460
query6	249	160	157	157
query7	4596	298	286	286
query8	305	276	269	269
query9	8568	2395	2370	2370
query10	576	321	281	281
query11	10639	10033	9930	9930
query12	117	88	86	86
query13	1633	364	376	364
query14	10275	6946	7460	6946
query15	250	180	187	180
query16	7917	276	276	276
query17	1891	571	555	555
query18	2048	287	273	273
query19	195	157	159	157
query20	90	81	83	81
query21	209	144	133	133
query22	4411	4126	4120	4120
query23	33851	33456	33425	33425
query24	10652	2840	2809	2809
query25	613	395	396	395
query26	718	163	163	163
query27	2346	320	342	320
query28	5799	2131	2138	2131
query29	943	648	653	648
query30	255	165	166	165
query31	988	748	744	744
query32	100	55	60	55
query33	668	312	299	299
query34	888	489	499	489
query35	778	653	642	642
query36	1148	983	970	970
query37	138	82	90	82
query38	2959	2814	2772	2772
query39	914	867	834	834
query40	213	126	125	125
query41	54	57	138	57
query42	120	96	103	96
query43	585	541	534	534
query44	1078	732	720	720
query45	184	164	165	164
query46	1067	743	743	743
query47	1859	1805	1753	1753
query48	380	298	292	292
query49	848	406	401	401
query50	769	387	390	387
query51	6975	6850	6833	6833
query52	103	96	92	92
query53	376	293	278	278
query54	881	434	425	425
query55	72	72	73	72
query56	281	257	263	257
query57	1144	1057	1069	1057
query58	240	258	287	258
query59	3480	3300	2998	2998
query60	293	268	265	265
query61	91	92	96	92
query62	605	441	439	439
query63	316	291	283	283
query64	8515	2263	1723	1723
query65	3126	3060	3079	3060
query66	750	320	321	320
query67	15564	14950	14786	14786
query68	8506	536	543	536
query69	766	457	345	345
query70	1446	1133	1116	1116
query71	524	282	276	276
query72	8655	5437	5343	5343
query73	2270	323	323	323
query74	5960	5539	5491	5491
query75	5483	2687	2657	2657
query76	5600	984	937	937
query77	781	298	293	293
query78	10505	10002	9763	9763
query79	7087	530	509	509
query80	1763	460	479	460
query81	550	229	221	221
query82	289	104	106	104
query83	284	168	170	168
query84	271	86	84	84
query85	868	277	309	277
query86	348	315	322	315
query87	3342	3202	3078	3078
query88	4695	2363	2353	2353
query89	493	374	378	374
query90	1993	193	184	184
query91	124	102	96	96
query92	58	50	48	48
query93	5688	488	490	488
query94	1224	183	183	183
query95	405	315	310	310
query96	597	266	267	266
query97	3188	3019	3054	3019
query98	238	204	193	193
query99	1115	827	848	827
Total cold run time: 289424 ms
Total hot run time: 172990 ms

@doris-robot
Copy link
Copy Markdown

doris-robot commented Jul 1, 2024

ClickBench: Total hot run time: 30.54 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit c9e6208c3d8ccf0ce1622323ddc9bea32c8b8c5b, data reload: false

query1	0.04	0.03	0.04
query2	0.09	0.04	0.04
query3	0.23	0.05	0.04
query4	1.66	0.06	0.07
query5	0.51	0.53	0.49
query6	1.14	0.73	0.73
query7	0.02	0.02	0.01
query8	0.05	0.04	0.04
query9	0.55	0.50	0.50
query10	0.54	0.53	0.53
query11	0.16	0.11	0.11
query12	0.14	0.12	0.12
query13	0.59	0.58	0.58
query14	0.80	0.79	0.77
query15	0.82	0.80	0.81
query16	0.35	0.36	0.35
query17	0.96	1.02	0.98
query18	0.20	0.28	0.23
query19	1.83	1.72	1.72
query20	0.02	0.01	0.01
query21	15.72	0.78	0.66
query22	4.38	7.98	1.77
query23	18.26	1.32	1.31
query24	2.14	0.25	0.24
query25	0.15	0.08	0.09
query26	0.28	0.17	0.17
query27	0.08	0.09	0.08
query28	13.22	1.01	1.00
query29	12.68	3.33	3.31
query30	0.26	0.06	0.06
query31	2.86	0.39	0.38
query32	3.26	0.49	0.48
query33	2.80	2.99	2.90
query34	17.11	4.40	4.44
query35	4.49	4.48	4.53
query36	0.66	0.46	0.47
query37	0.18	0.16	0.17
query38	0.16	0.14	0.15
query39	0.04	0.03	0.03
query40	0.17	0.15	0.18
query41	0.10	0.05	0.05
query42	0.06	0.05	0.05
query43	0.04	0.04	0.04
Total cold run time: 109.8 s
Total hot run time: 30.54 s

@zhangstar333
Copy link
Copy Markdown
Contributor Author

zhangstar333 commented Jul 2, 2024

run buildall

@doris-robot
Copy link
Copy Markdown

doris-robot commented Jul 2, 2024

TPC-H: Total hot run time: 40007 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c9e6208c3d8ccf0ce1622323ddc9bea32c8b8c5b, data reload: false

------ Round 1 ----------------------------------
q1	17602	4391	4327	4327
q2	2021	193	194	193
q3	10453	1252	1084	1084
q4	10176	792	885	792
q5	7482	2673	2669	2669
q6	222	138	140	138
q7	950	619	598	598
q8	9231	2099	2104	2099
q9	9068	6511	6458	6458
q10	8871	3717	3743	3717
q11	450	239	236	236
q12	488	238	225	225
q13	17772	2975	2969	2969
q14	256	225	220	220
q15	523	485	476	476
q16	505	369	371	369
q17	965	704	716	704
q18	8093	7493	7460	7460
q19	6097	1491	1527	1491
q20	662	316	314	314
q21	4949	3213	3141	3141
q22	396	340	327	327
Total cold run time: 117232 ms
Total hot run time: 40007 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4380	4272	4246	4246
q2	376	266	261	261
q3	2969	2836	2867	2836
q4	1981	1667	1680	1667
q5	5684	5534	5491	5491
q6	230	135	133	133
q7	2240	1861	1859	1859
q8	3303	3451	3552	3451
q9	8768	8689	8843	8689
q10	4145	3868	3685	3685
q11	597	506	488	488
q12	858	647	642	642
q13	17135	3191	3193	3191
q14	329	286	278	278
q15	531	503	477	477
q16	488	440	429	429
q17	1819	1569	1527	1527
q18	8191	8140	7897	7897
q19	1785	1547	1637	1547
q20	2127	1871	1871	1871
q21	5302	5061	4883	4883
q22	619	582	553	553
Total cold run time: 73857 ms
Total hot run time: 56101 ms

@doris-robot
Copy link
Copy Markdown

doris-robot commented Jul 2, 2024

TPC-DS: Total hot run time: 173578 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c9e6208c3d8ccf0ce1622323ddc9bea32c8b8c5b, data reload: false

query1	915	393	375	375
query2	6453	2478	2262	2262
query3	6695	225	217	217
query4	19262	17548	17197	17197
query5	3660	478	473	473
query6	246	166	163	163
query7	4585	300	298	298
query8	327	309	276	276
query9	8650	2381	2365	2365
query10	580	323	289	289
query11	10557	9882	10036	9882
query12	116	83	81	81
query13	1654	373	373	373
query14	9555	7772	7027	7027
query15	224	191	189	189
query16	7827	259	271	259
query17	1901	538	524	524
query18	2000	275	273	273
query19	194	148	153	148
query20	90	80	79	79
query21	217	140	136	136
query22	4506	3930	3982	3930
query23	33950	33582	33618	33582
query24	10649	2883	2857	2857
query25	594	377	375	375
query26	708	159	160	159
query27	2254	328	331	328
query28	5899	2165	2159	2159
query29	902	620	626	620
query30	250	151	171	151
query31	984	815	731	731
query32	93	54	53	53
query33	657	286	307	286
query34	894	483	489	483
query35	770	650	614	614
query36	1140	939	997	939
query37	135	80	82	80
query38	2963	2862	2784	2784
query39	883	854	836	836
query40	203	130	128	128
query41	56	52	54	52
query42	103	100	107	100
query43	609	559	557	557
query44	1082	734	728	728
query45	194	166	165	165
query46	1083	709	743	709
query47	1821	1768	1734	1734
query48	368	305	308	305
query49	880	412	406	406
query50	769	389	405	389
query51	6978	6749	6750	6749
query52	104	95	98	95
query53	361	300	294	294
query54	864	448	438	438
query55	78	75	74	74
query56	290	269	257	257
query57	1145	1049	1063	1049
query58	253	240	244	240
query59	3459	3289	3461	3289
query60	294	274	276	274
query61	92	89	85	85
query62	593	428	429	428
query63	334	295	289	289
query64	8534	2261	1721	1721
query65	3142	3073	3124	3073
query66	738	327	328	327
query67	15310	15068	15028	15028
query68	4452	545	558	545
query69	455	322	319	319
query70	1191	1120	1170	1120
query71	367	281	285	281
query72	7151	5276	5792	5276
query73	742	330	327	327
query74	6007	5495	5482	5482
query75	3347	2683	2724	2683
query76	2095	990	930	930
query77	439	315	304	304
query78	10340	9919	9874	9874
query79	1604	520	526	520
query80	2378	467	463	463
query81	586	222	221	221
query82	1079	110	105	105
query83	373	180	179	179
query84	260	93	85	85
query85	1242	282	268	268
query86	432	339	303	303
query87	3260	3109	3098	3098
query88	3113	2382	2379	2379
query89	477	387	385	385
query90	1934	193	189	189
query91	127	100	98	98
query92	61	53	50	50
query93	1288	503	498	498
query94	1325	194	181	181
query95	424	315	317	315
query96	589	265	273	265
query97	3191	3049	2980	2980
query98	220	192	198	192
query99	1153	841	827	827
Total cold run time: 265378 ms
Total hot run time: 173578 ms

@doris-robot
Copy link
Copy Markdown

doris-robot commented Jul 2, 2024

ClickBench: Total hot run time: 30.93 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit c9e6208c3d8ccf0ce1622323ddc9bea32c8b8c5b, data reload: false

query1	0.04	0.03	0.02
query2	0.09	0.04	0.04
query3	0.23	0.06	0.06
query4	1.65	0.09	0.09
query5	0.50	0.48	0.48
query6	1.14	0.72	0.73
query7	0.02	0.01	0.02
query8	0.05	0.04	0.05
query9	0.56	0.49	0.49
query10	0.55	0.55	0.53
query11	0.14	0.12	0.11
query12	0.14	0.12	0.12
query13	0.60	0.59	0.58
query14	0.79	0.77	0.77
query15	0.83	0.81	0.82
query16	0.35	0.37	0.36
query17	1.03	1.07	1.02
query18	0.21	0.25	0.26
query19	1.86	1.72	1.73
query20	0.02	0.01	0.01
query21	15.43	0.77	0.67
query22	4.32	6.79	2.17
query23	18.26	1.39	1.28
query24	2.14	0.21	0.22
query25	0.14	0.10	0.08
query26	0.27	0.18	0.17
query27	0.09	0.09	0.08
query28	13.26	1.02	0.99
query29	12.66	3.27	3.26
query30	0.25	0.07	0.06
query31	2.86	0.38	0.37
query32	3.31	0.48	0.46
query33	2.85	2.96	2.95
query34	17.13	4.42	4.43
query35	4.45	4.47	4.46
query36	0.64	0.47	0.50
query37	0.19	0.15	0.16
query38	0.15	0.16	0.15
query39	0.05	0.03	0.03
query40	0.18	0.15	0.14
query41	0.09	0.07	0.05
query42	0.06	0.04	0.05
query43	0.04	0.04	0.04
Total cold run time: 109.62 s
Total hot run time: 30.93 s

HappenLee
HappenLee approved these changes Jul 5, 2024
View reviewed changes
Copy link
Copy Markdown
Contributor

@HappenLee HappenLee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

wangbo
wangbo approved these changes Jul 5, 2024
View reviewed changes
Copy link
Copy Markdown
Contributor

@wangbo wangbo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Jul 5, 2024

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added approved Indicates a PR has been approved by one committer. reviewed labels Jul 5, 2024
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Jul 5, 2024

PR approved by anyone and no changes requested.

@HappenLee HappenLee merged commit 63314b8 into apache:master Jul 5, 2024
dataroaring pushed a commit that referenced this pull request Jul 17, 2024
@zhangstar333 @dataroaring
[bug](function)fix json_replace check return type error (#37014)
f360524 
1. fix the return type dcheck error:
```
mysql [test]>select (json_replace(a, '$.fparam.nested_2', "qwe")) from json_table_2 limit 1;
ERROR 1105 (HY000): errCode = 2, detailMessage = (10.16.10.8)[INTERNAL_ERROR]Function json_replace get failed, expr is VectorizedFnCall[json_replace](arguments=a, String, String, String,return=Nullable(String)) and return type is Nullable(String).
```

2. improve the json_replace/json_insert/json_set function execute of not
convert const column, test about could faster 1s on 1000w table rows
@gavinchou gavinchou mentioned this pull request Aug 19, 2024
Closed
zhangstar333 added a commit to zhangstar333/incubator-doris that referenced this pull request Aug 26, 2024
@zhangstar333
[bug](function)fix json_replace check return type error (apache#37014)
843209f 
1. fix the return type dcheck error:
```
mysql [test]>select (json_replace(a, '$.fparam.nested_2', "qwe")) from json_table_2 limit 1;
ERROR 1105 (HY000): errCode = 2, detailMessage = (10.16.10.8)[INTERNAL_ERROR]Function json_replace get failed, expr is VectorizedFnCall[json_replace](arguments=a, String, String, String,return=Nullable(String)) and return type is Nullable(String).
```

2. improve the json_replace/json_insert/json_set function execute of not
convert const column, test about could faster 1s on 1000w table rows
yiguolei pushed a commit that referenced this pull request Aug 27, 2024
@zhangstar333
[bug](function)fix json_replace check return type error (#37014) (#39938
21bd4a4 
)

1. fix the return type dcheck error:
```
mysql [test]>select (json_replace(a, '$.fparam.nested_2', "qwe")) from json_table_2 limit 1;
ERROR 1105 (HY000): errCode = 2, detailMessage = (10.16.10.8)[INTERNAL_ERROR]Function json_replace get failed, expr is VectorizedFnCall[json_replace](arguments=a, String, String, String,return=Nullable(String)) and return type is Nullable(String).
```

2. improve the json_replace/json_insert/json_set function execute of not
convert const column, test about could faster 1s on 1000w table rows

## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->
@zhangstar333 zhangstar333 mentioned this pull request Aug 27, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@HappenLee HappenLee HappenLee approved these changes

@wangbo wangbo wangbo approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/2.1.6-merged dev/3.0.1-merged reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@zhangstar333 @doris-robot @HappenLee @wangbo @dataroaring