Skip to content

[improve](json)improve json support empty keys#36762

Merged
eldenmoon merged 8 commits intoapache:masterfrom
amorynan:improve-support-json-with-empty-keys
Jul 1, 2024
Merged

[improve](json)improve json support empty keys#36762
eldenmoon merged 8 commits intoapache:masterfrom
amorynan:improve-support-json-with-empty-keys

Conversation

@amorynan
Copy link
Copy Markdown
Contributor

@amorynan amorynan commented Jun 24, 2024

Proposed changes

support empty key for json
behavior reference MSYQL

Issue Number: close #xxx

@doris-robot
Copy link
Copy Markdown

doris-robot commented Jun 24, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Jun 24, 2024

clang-tidy review says "All clean, LGTM! 👍"

@amorynan
Copy link
Copy Markdown
Contributor Author

amorynan commented Jun 24, 2024

run buildall

@doris-robot
Copy link
Copy Markdown

doris-robot commented Jun 24, 2024

TPC-H: Total hot run time: 40254 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 3d68e6823af703e325e35bec7b7a324833c45323, data reload: false

------ Round 1 ----------------------------------
q1	17595	4505	4311	4311
q2	2002	191	192	191
q3	10483	1132	1091	1091
q4	10662	794	804	794
q5	7899	2769	2697	2697
q6	226	139	136	136
q7	959	630	611	611
q8	9297	2114	2123	2114
q9	9239	6522	6481	6481
q10	9055	3731	3688	3688
q11	449	249	244	244
q12	499	228	223	223
q13	17769	3012	2986	2986
q14	264	223	217	217
q15	516	493	488	488
q16	492	389	382	382
q17	987	668	686	668
q18	8062	7676	7484	7484
q19	7132	1495	1477	1477
q20	655	334	337	334
q21	4940	3293	3343	3293
q22	406	351	344	344
Total cold run time: 119588 ms
Total hot run time: 40254 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4465	4216	4228	4216
q2	374	265	273	265
q3	3003	2795	2699	2699
q4	1876	1580	1608	1580
q5	5330	5278	5341	5278
q6	219	127	130	127
q7	2177	1784	1730	1730
q8	3200	3348	3332	3332
q9	8404	8344	8391	8344
q10	3881	3652	3704	3652
q11	571	489	499	489
q12	793	640	625	625
q13	17641	2976	2984	2976
q14	294	280	261	261
q15	521	480	478	478
q16	467	423	414	414
q17	1781	1513	1489	1489
q18	7710	7383	7484	7383
q19	1698	1640	1676	1640
q20	2033	1772	1759	1759
q21	4823	4892	4813	4813
q22	615	568	567	567
Total cold run time: 71876 ms
Total hot run time: 54117 ms

@doris-robot
Copy link
Copy Markdown

doris-robot commented Jun 24, 2024

TPC-DS: Total hot run time: 172016 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 3d68e6823af703e325e35bec7b7a324833c45323, data reload: false

query1	919	377	389	377
query2	6432	2436	2494	2436
query3	6639	204	209	204
query4	19034	17310	17446	17310
query5	4239	478	498	478
query6	253	164	156	156
query7	4584	309	299	299
query8	316	305	296	296
query9	8604	2489	2452	2452
query10	594	296	267	267
query11	10606	9916	10041	9916
query12	145	94	84	84
query13	1647	381	392	381
query14	9647	7674	7865	7674
query15	260	205	194	194
query16	8095	286	277	277
query17	1877	572	547	547
query18	2086	289	278	278
query19	206	161	167	161
query20	93	85	82	82
query21	214	140	138	138
query22	4234	4015	3921	3921
query23	33796	33026	33186	33026
query24	11183	2891	2858	2858
query25	653	388	396	388
query26	1598	162	166	162
query27	2938	324	328	324
query28	7637	2111	2116	2111
query29	971	656	644	644
query30	295	156	153	153
query31	985	727	760	727
query32	100	59	56	56
query33	792	301	311	301
query34	902	508	476	476
query35	766	643	650	643
query36	1092	963	983	963
query37	155	76	72	72
query38	2869	2736	2736	2736
query39	888	824	819	819
query40	293	147	138	138
query41	58	58	54	54
query42	124	102	104	102
query43	614	572	574	572
query44	1236	765	753	753
query45	195	168	165	165
query46	1085	754	725	725
query47	1845	1754	1752	1752
query48	379	316	313	313
query49	1223	441	428	428
query50	788	410	415	410
query51	6903	6838	6940	6838
query52	117	93	106	93
query53	376	296	298	296
query54	929	468	464	464
query55	78	76	77	76
query56	303	298	291	291
query57	1146	1038	1036	1036
query58	284	264	284	264
query59	3468	3227	3354	3227
query60	326	289	406	289
query61	91	91	89	89
query62	647	442	465	442
query63	337	300	297	297
query64	9917	2275	1744	1744
query65	3186	3146	3130	3130
query66	1361	369	331	331
query67	15974	14977	14969	14969
query68	8900	564	584	564
query69	715	478	439	439
query70	1391	1099	1140	1099
query71	498	282	282	282
query72	8169	5363	2778	2778
query73	2205	339	341	339
query74	5979	5432	5407	5407
query75	4869	2672	2690	2672
query76	5065	993	959	959
query77	702	317	314	314
query78	10507	9720	9825	9720
query79	8548	529	532	529
query80	1087	486	478	478
query81	548	225	228	225
query82	636	109	105	105
query83	304	169	166	166
query84	271	90	86	86
query85	1238	290	274	274
query86	415	330	341	330
query87	3321	3097	3075	3075
query88	4128	2470	2450	2450
query89	554	396	404	396
query90	1890	203	187	187
query91	129	100	100	100
query92	58	52	49	49
query93	6928	532	517	517
query94	1165	193	192	192
query95	415	329	326	326
query96	603	275	267	267
query97	3240	2994	3034	2994
query98	225	209	203	203
query99	1249	847	819	819
Total cold run time: 297579 ms
Total hot run time: 172016 ms

@doris-robot
Copy link
Copy Markdown

doris-robot commented Jun 24, 2024

ClickBench: Total hot run time: 30.99 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 3d68e6823af703e325e35bec7b7a324833c45323, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.04	0.04
query3	0.22	0.05	0.05
query4	1.69	0.06	0.07
query5	0.50	0.48	0.53
query6	1.14	0.72	0.72
query7	0.02	0.01	0.01
query8	0.05	0.04	0.04
query9	0.55	0.48	0.49
query10	0.52	0.53	0.54
query11	0.14	0.10	0.11
query12	0.15	0.12	0.12
query13	0.60	0.58	0.58
query14	0.75	0.80	0.83
query15	0.84	0.82	0.81
query16	0.37	0.36	0.38
query17	1.03	0.97	0.95
query18	0.21	0.26	0.25
query19	1.86	1.72	1.69
query20	0.02	0.01	0.01
query21	15.43	0.66	0.66
query22	4.18	6.81	2.31
query23	18.29	1.44	1.25
query24	2.14	0.22	0.23
query25	0.14	0.09	0.08
query26	0.27	0.17	0.17
query27	0.08	0.08	0.07
query28	13.25	1.01	0.99
query29	12.64	3.30	3.26
query30	0.25	0.06	0.06
query31	2.86	0.39	0.39
query32	3.28	0.47	0.48
query33	2.92	2.92	2.95
query34	17.16	4.42	4.47
query35	4.57	4.55	4.60
query36	0.65	0.46	0.46
query37	0.19	0.15	0.16
query38	0.15	0.15	0.14
query39	0.04	0.04	0.04
query40	0.17	0.14	0.13
query41	0.09	0.05	0.05
query42	0.06	0.05	0.05
query43	0.04	0.04	0.04
Total cold run time: 109.61 s
Total hot run time: 30.99 s

eldenmoon
eldenmoon reviewed Jun 25, 2024
View reviewed changes
Comment thread be/src/util/jsonb_utils.h Outdated
string_to_json(iter->getKeyStr(), iter->klen());
} else {
os_.write(iter->getKeyId());
if (iter->getKeyId() == 0) {
Copy link
Copy Markdown
Member

@eldenmoon eldenmoon Jun 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is key == 0 special ? if special add comment to explain

Copy link
Copy Markdown
Contributor Author

@amorynan amorynan Jun 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, we do not use keyid before , just use keystr for jsonkey, and se I make keyid == 0 is special for empty key

Copy link
Copy Markdown
Member

@eldenmoon eldenmoon Jun 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe use sMaxKeyId for safe and readable, 0 is not too special

Copy link
Copy Markdown
Contributor Author

@amorynan amorynan Jun 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK I will try

eldenmoon
eldenmoon reviewed Jun 25, 2024
View reviewed changes
Comment thread be/src/util/jsonb_writer.h Outdated
os_->write(key, len);
size += len;
if (len == 0) {
JsonbKeyValue::keyid_type idx = 0;
Copy link
Copy Markdown
Member

@eldenmoon eldenmoon Jun 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use sMaxKeyId

Copy link
Copy Markdown
Member

@eldenmoon eldenmoon Jun 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add comment

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Jun 25, 2024

clang-tidy review says "All clean, LGTM! 👍"

@amorynan
Copy link
Copy Markdown
Contributor Author

amorynan commented Jun 25, 2024

run buildall

@doris-robot
Copy link
Copy Markdown

doris-robot commented Jun 25, 2024

TPC-H: Total hot run time: 40231 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 810065470efd35400292a8e88eaec12c5468150d, data reload: false

------ Round 1 ----------------------------------
q1	17600	4491	4315	4315
q2	2009	186	189	186
q3	10484	1096	1103	1096
q4	10202	833	807	807
q5	7473	2756	2675	2675
q6	225	137	140	137
q7	969	616	607	607
q8	9222	2103	2119	2103
q9	9064	6506	6464	6464
q10	9008	3692	3738	3692
q11	476	244	250	244
q12	428	236	240	236
q13	17765	2962	2991	2962
q14	272	227	223	223
q15	520	483	494	483
q16	531	387	390	387
q17	1006	736	719	719
q18	8094	7481	7487	7481
q19	5485	1538	1561	1538
q20	677	343	323	323
q21	4982	3209	3951	3209
q22	409	347	344	344
Total cold run time: 116901 ms
Total hot run time: 40231 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4442	4287	4272	4272
q2	370	270	263	263
q3	3022	2921	2878	2878
q4	2003	1717	1739	1717
q5	5508	5494	5492	5492
q6	225	133	136	133
q7	2237	1866	1884	1866
q8	3339	3431	3462	3431
q9	8837	8760	8850	8760
q10	4153	3883	3722	3722
q11	595	515	504	504
q12	823	649	624	624
q13	15962	3139	3121	3121
q14	304	266	283	266
q15	551	481	499	481
q16	483	435	440	435
q17	1840	1512	1514	1512
q18	8044	8013	7803	7803
q19	1840	1675	1656	1656
q20	2394	1893	1879	1879
q21	10168	5005	5028	5005
q22	641	562	549	549
Total cold run time: 77781 ms
Total hot run time: 56369 ms

@doris-robot
Copy link
Copy Markdown

doris-robot commented Jun 25, 2024

TPC-DS: Total hot run time: 174148 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 810065470efd35400292a8e88eaec12c5468150d, data reload: false

query1	941	405	369	369
query2	6428	2370	2290	2290
query3	6624	222	203	203
query4	19427	17246	17304	17246
query5	3661	510	462	462
query6	237	164	157	157
query7	4576	301	300	300
query8	334	301	288	288
query9	8915	2414	2381	2381
query10	579	295	268	268
query11	10615	10154	10083	10083
query12	107	85	85	85
query13	1627	371	380	371
query14	10109	7758	7024	7024
query15	247	185	190	185
query16	7721	281	277	277
query17	1890	569	537	537
query18	1955	289	310	289
query19	200	150	155	150
query20	90	85	86	85
query21	208	126	131	126
query22	4581	4238	4152	4152
query23	33843	33810	33563	33563
query24	11132	2869	2893	2869
query25	642	400	383	383
query26	1378	160	155	155
query27	2949	336	328	328
query28	7426	2173	2154	2154
query29	945	622	614	614
query30	260	155	158	155
query31	963	750	755	750
query32	90	52	56	52
query33	741	282	293	282
query34	1067	482	480	480
query35	774	647	620	620
query36	1145	975	955	955
query37	159	77	79	77
query38	2903	2820	2839	2820
query39	894	853	837	837
query40	227	131	125	125
query41	59	56	52	52
query42	108	110	101	101
query43	622	546	567	546
query44	1233	734	736	734
query45	192	163	164	163
query46	1085	728	737	728
query47	1848	1760	1742	1742
query48	389	307	293	293
query49	860	403	426	403
query50	766	391	393	391
query51	6786	6675	6738	6675
query52	97	101	91	91
query53	362	289	291	289
query54	907	435	431	431
query55	74	73	74	73
query56	285	256	268	256
query57	1162	1073	1062	1062
query58	264	247	244	244
query59	3628	3303	3108	3108
query60	293	264	287	264
query61	93	87	93	87
query62	601	445	481	445
query63	329	295	293	293
query64	10571	2387	1743	1743
query65	3172	3128	3124	3124
query66	1045	330	327	327
query67	15439	15011	14978	14978
query68	6255	550	549	549
query69	640	442	377	377
query70	1145	1082	1132	1082
query71	499	270	266	266
query72	7272	5740	5575	5575
query73	819	326	326	326
query74	5812	5505	5465	5465
query75	4024	2664	2637	2637
query76	4224	914	955	914
query77	685	317	299	299
query78	10262	9962	9758	9758
query79	1754	530	517	517
query80	1888	461	520	461
query81	554	217	218	217
query82	761	107	103	103
query83	277	169	170	169
query84	265	83	87	83
query85	1296	281	270	270
query86	455	328	336	328
query87	3293	3125	3088	3088
query88	3664	2385	2367	2367
query89	474	385	382	382
query90	1765	182	185	182
query91	126	99	100	99
query92	62	51	50	50
query93	2047	519	524	519
query94	1003	189	193	189
query95	406	308	312	308
query96	596	271	272	271
query97	3215	3074	3062	3062
query98	237	203	202	202
query99	1216	854	843	843
Total cold run time: 277871 ms
Total hot run time: 174148 ms

@doris-robot
Copy link
Copy Markdown

doris-robot commented Jun 25, 2024

ClickBench: Total hot run time: 31.33 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 810065470efd35400292a8e88eaec12c5468150d, data reload: false

query1	0.03	0.03	0.04
query2	0.07	0.04	0.04
query3	0.22	0.05	0.05
query4	1.70	0.07	0.07
query5	0.51	0.51	0.48
query6	1.13	0.73	0.72
query7	0.02	0.02	0.01
query8	0.04	0.04	0.04
query9	0.54	0.49	0.48
query10	0.54	0.53	0.54
query11	0.15	0.10	0.11
query12	0.16	0.12	0.12
query13	0.59	0.57	0.58
query14	0.74	0.80	0.78
query15	0.82	0.81	0.81
query16	0.38	0.37	0.38
query17	0.95	0.96	0.95
query18	0.20	0.28	0.25
query19	1.72	1.68	1.81
query20	0.01	0.00	0.00
query21	15.43	0.65	0.65
query22	4.17	5.96	2.92
query23	18.30	1.41	1.22
query24	2.13	0.22	0.22
query25	0.15	0.09	0.08
query26	0.25	0.18	0.17
query27	0.08	0.08	0.08
query28	13.21	1.03	1.00
query29	12.58	3.31	3.27
query30	0.25	0.06	0.06
query31	2.88	0.39	0.39
query32	3.27	0.47	0.48
query33	2.93	2.84	2.94
query34	17.17	4.41	4.41
query35	4.58	4.41	4.50
query36	0.66	0.46	0.46
query37	0.17	0.16	0.15
query38	0.15	0.15	0.15
query39	0.05	0.04	0.03
query40	0.16	0.14	0.14
query41	0.09	0.05	0.05
query42	0.06	0.05	0.04
query43	0.04	0.04	0.04
Total cold run time: 109.28 s
Total hot run time: 31.33 s

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Jun 27, 2024

clang-tidy review says "All clean, LGTM! 👍"

1 similar comment
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Jun 27, 2024

clang-tidy review says "All clean, LGTM! 👍"

@amorynan
Copy link
Copy Markdown
Contributor Author

amorynan commented Jun 27, 2024

run buildall

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Jun 27, 2024

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link
Copy Markdown

doris-robot commented Jun 27, 2024

TPC-H: Total hot run time: 40117 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 5d7320153ef06bba8be302cadebf45cf86270644, data reload: false

------ Round 1 ----------------------------------
q1	17610	4792	4280	4280
q2	2012	193	196	193
q3	10465	1261	1153	1153
q4	10193	848	800	800
q5	7905	2766	2684	2684
q6	225	136	136	136
q7	966	604	606	604
q8	9230	2116	2093	2093
q9	9086	6533	6515	6515
q10	8999	3724	3720	3720
q11	460	234	248	234
q12	425	239	233	233
q13	18783	3017	2977	2977
q14	256	230	223	223
q15	517	480	470	470
q16	537	383	378	378
q17	999	603	739	603
q18	8125	7477	7444	7444
q19	5618	1554	1478	1478
q20	668	329	323	323
q21	4964	3238	3868	3238
q22	403	338	343	338
Total cold run time: 118446 ms
Total hot run time: 40117 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4409	4242	4252	4242
q2	373	269	267	267
q3	2965	2750	2719	2719
q4	1903	1593	1604	1593
q5	5284	5295	5313	5295
q6	219	129	129	129
q7	2100	1775	1693	1693
q8	3199	3339	3331	3331
q9	8400	8360	8357	8357
q10	3853	3673	3623	3623
q11	589	508	483	483
q12	784	625	611	611
q13	17513	2988	2978	2978
q14	302	272	267	267
q15	521	480	481	480
q16	468	405	410	405
q17	1785	1485	1497	1485
q18	7612	7596	7423	7423
q19	1729	1481	1515	1481
q20	1982	1812	1767	1767
q21	5055	4838	4675	4675
q22	601	545	568	545
Total cold run time: 71646 ms
Total hot run time: 53849 ms

@doris-robot
Copy link
Copy Markdown

doris-robot commented Jun 27, 2024

TPC-DS: Total hot run time: 173351 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 5d7320153ef06bba8be302cadebf45cf86270644, data reload: false

query1	926	388	399	388
query2	7335	2508	2347	2347
query3	6643	214	221	214
query4	18993	17685	17433	17433
query5	4208	472	475	472
query6	270	171	167	167
query7	4601	304	293	293
query8	333	300	288	288
query9	8489	2338	2330	2330
query10	612	304	272	272
query11	10791	10223	10303	10223
query12	127	86	81	81
query13	1642	353	358	353
query14	10144	6892	6841	6841
query15	230	186	185	185
query16	7830	264	266	264
query17	1896	545	513	513
query18	1962	276	270	270
query19	189	154	154	154
query20	95	82	80	80
query21	207	136	128	128
query22	4846	4480	4312	4312
query23	33774	32955	33065	32955
query24	11798	2797	2800	2797
query25	660	366	370	366
query26	1789	151	150	150
query27	3073	317	315	315
query28	7704	2023	2010	2010
query29	1146	614	617	614
query30	279	154	155	154
query31	944	760	744	744
query32	95	53	52	52
query33	756	287	280	280
query34	1019	462	484	462
query35	740	647	622	622
query36	1106	928	919	919
query37	296	67	75	67
query38	2904	2772	2791	2772
query39	862	799	794	794
query40	281	151	121	121
query41	50	46	51	46
query42	118	96	106	96
query43	614	544	545	544
query44	1189	728	734	728
query45	193	164	159	159
query46	1078	706	742	706
query47	1870	1784	1800	1784
query48	383	297	306	297
query49	1201	415	403	403
query50	780	387	394	387
query51	6927	6797	6734	6734
query52	105	95	93	93
query53	362	292	297	292
query54	1018	439	439	439
query55	74	74	72	72
query56	313	253	264	253
query57	1169	1046	1079	1046
query58	249	253	263	253
query59	3559	3265	3432	3265
query60	316	296	310	296
query61	89	97	90	90
query62	636	449	450	449
query63	328	288	289	288
query64	9862	2210	1733	1733
query65	3149	3083	3181	3083
query66	1379	335	333	333
query67	15444	15058	14952	14952
query68	6167	538	545	538
query69	590	479	395	395
query70	1215	1134	1069	1069
query71	453	280	270	270
query72	7703	5428	5283	5283
query73	792	324	323	323
query74	5913	5582	5488	5488
query75	4050	2723	2669	2669
query76	3940	973	890	890
query77	660	292	292	292
query78	10329	9929	9788	9788
query79	2345	514	512	512
query80	2283	474	484	474
query81	532	221	224	221
query82	768	104	105	104
query83	288	171	169	169
query84	278	92	99	92
query85	1466	343	268	268
query86	464	324	312	312
query87	3290	3075	3082	3075
query88	4233	2458	2432	2432
query89	481	386	377	377
query90	1827	193	188	188
query91	134	101	97	97
query92	59	51	50	50
query93	3425	492	492	492
query94	1175	186	191	186
query95	392	310	315	310
query96	609	276	280	276
query97	3198	3045	3038	3038
query98	211	203	196	196
query99	1219	854	870	854
Total cold run time: 284539 ms
Total hot run time: 173351 ms

@doris-robot
Copy link
Copy Markdown

doris-robot commented Jun 27, 2024

ClickBench: Total hot run time: 30.72 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 5d7320153ef06bba8be302cadebf45cf86270644, data reload: false

query1	0.04	0.04	0.04
query2	0.09	0.04	0.04
query3	0.22	0.05	0.05
query4	1.68	0.07	0.07
query5	0.52	0.48	0.50
query6	1.13	0.74	0.74
query7	0.02	0.01	0.02
query8	0.05	0.04	0.05
query9	0.55	0.50	0.51
query10	0.54	0.54	0.56
query11	0.16	0.11	0.12
query12	0.14	0.11	0.12
query13	0.60	0.58	0.60
query14	0.79	0.76	0.77
query15	0.83	0.81	0.81
query16	0.36	0.36	0.34
query17	1.00	1.05	0.99
query18	0.23	0.25	0.23
query19	1.81	1.66	1.71
query20	0.02	0.01	0.01
query21	15.48	0.72	0.65
query22	3.65	7.34	2.25
query23	18.29	1.40	1.25
query24	2.12	0.22	0.22
query25	0.17	0.08	0.08
query26	0.27	0.18	0.17
query27	0.08	0.07	0.07
query28	13.28	1.01	1.01
query29	12.67	3.25	3.22
query30	0.25	0.06	0.05
query31	2.86	0.38	0.39
query32	3.30	0.47	0.48
query33	2.91	2.86	2.89
query34	17.13	4.40	4.43
query35	4.50	4.47	4.44
query36	0.67	0.48	0.48
query37	0.17	0.16	0.15
query38	0.15	0.15	0.15
query39	0.04	0.04	0.04
query40	0.18	0.14	0.15
query41	0.09	0.04	0.05
query42	0.05	0.05	0.05
query43	0.04	0.04	0.04
Total cold run time: 109.13 s
Total hot run time: 30.72 s

@amorynan
Copy link
Copy Markdown
Contributor Author

amorynan commented Jun 28, 2024

run buildall

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Jun 28, 2024

clang-tidy review says "All clean, LGTM! 👍"

eldenmoon
eldenmoon reviewed Jun 28, 2024
View reviewed changes
Comment thread be/src/util/jsonb_writer.h
@xiaokang xiaokang added dev/2.1.x usercase Important user case type label labels Jun 30, 2024
@amorynan amorynan dismissed stale reviews from xiaokang and eldenmoon via a9cb773 July 1, 2024 02:53
@amorynan
Copy link
Copy Markdown
Contributor Author

amorynan commented Jul 1, 2024

run buildall

@github-actions github-actions bot removed the approved Indicates a PR has been approved by one committer. label Jul 1, 2024
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Jul 1, 2024

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link
Copy Markdown

doris-robot commented Jul 1, 2024

TPC-H: Total hot run time: 39762 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a9cb77374ded417f79358e833459848a86dc7bf7, data reload: false

------ Round 1 ----------------------------------
q1	17614	4432	4309	4309
q2	2018	185	195	185
q3	10445	1231	1065	1065
q4	10186	800	843	800
q5	7487	2655	2646	2646
q6	226	139	137	137
q7	962	598	633	598
q8	9231	2089	2090	2089
q9	9099	6489	6492	6489
q10	8976	3677	3750	3677
q11	447	238	234	234
q12	425	241	231	231
q13	18880	2985	2965	2965
q14	270	226	209	209
q15	521	474	483	474
q16	522	379	369	369
q17	974	728	716	716
q18	8095	7495	7369	7369
q19	4155	1522	1367	1367
q20	657	333	328	328
q21	4919	3175	3968	3175
q22	386	330	337	330
Total cold run time: 116495 ms
Total hot run time: 39762 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4367	4311	4254	4254
q2	387	258	258	258
q3	2988	2856	2872	2856
q4	1981	1763	1683	1683
q5	5651	5508	5508	5508
q6	229	133	137	133
q7	2212	1836	1885	1836
q8	3245	3399	3429	3399
q9	8665	8679	8886	8679
q10	4119	3872	3666	3666
q11	593	498	499	498
q12	803	656	622	622
q13	17278	3148	3157	3148
q14	306	301	267	267
q15	540	483	481	481
q16	486	431	435	431
q17	1854	1569	1490	1490
q18	8134	8018	7780	7780
q19	1762	1654	1679	1654
q20	2178	1898	1854	1854
q21	5113	5146	4770	4770
q22	650	559	604	559
Total cold run time: 73541 ms
Total hot run time: 55826 ms

@doris-robot
Copy link
Copy Markdown

doris-robot commented Jul 1, 2024

TPC-DS: Total hot run time: 174348 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a9cb77374ded417f79358e833459848a86dc7bf7, data reload: false

query1	926	380	396	380
query2	6448	2360	2383	2360
query3	6634	216	217	216
query4	19259	17493	17140	17140
query5	3668	500	476	476
query6	256	165	159	159
query7	4591	290	294	290
query8	296	277	282	277
query9	8493	2386	2371	2371
query10	589	313	283	283
query11	10597	10121	10051	10051
query12	116	88	87	87
query13	1648	378	360	360
query14	10072	7834	7776	7776
query15	235	188	187	187
query16	7807	286	275	275
query17	1864	577	544	544
query18	1989	283	280	280
query19	206	157	158	157
query20	89	81	87	81
query21	212	135	128	128
query22	4264	4067	3913	3913
query23	34051	33842	33652	33652
query24	10540	2965	2942	2942
query25	617	406	410	406
query26	715	162	162	162
query27	2273	335	325	325
query28	6118	2150	2128	2128
query29	902	681	645	645
query30	244	162	161	161
query31	1008	751	835	751
query32	95	53	54	53
query33	657	300	293	293
query34	883	488	490	488
query35	746	685	678	678
query36	1147	993	959	959
query37	139	79	78	78
query38	2917	2854	2823	2823
query39	916	841	808	808
query40	206	127	121	121
query41	61	52	49	49
query42	123	103	101	101
query43	585	525	551	525
query44	1060	736	717	717
query45	192	166	158	158
query46	1067	744	722	722
query47	1837	1776	1760	1760
query48	355	309	300	300
query49	834	402	406	402
query50	762	382	391	382
query51	6861	6718	6830	6718
query52	105	91	93	91
query53	367	297	289	289
query54	837	449	435	435
query55	75	74	72	72
query56	296	265	258	258
query57	1118	1056	1074	1056
query58	247	256	236	236
query59	3276	3018	2981	2981
query60	326	282	272	272
query61	90	89	92	89
query62	591	465	442	442
query63	318	292	291	291
query64	8477	2278	1757	1757
query65	3150	3117	3088	3088
query66	733	321	332	321
query67	15594	14837	15024	14837
query68	4600	525	526	525
query69	668	434	342	342
query70	1158	1105	1104	1104
query71	431	281	276	276
query72	8653	5364	5329	5329
query73	765	369	330	330
query74	5980	5550	5577	5550
query75	4297	2618	2662	2618
query76	3284	998	916	916
query77	698	306	294	294
query78	10451	9819	9849	9819
query79	3050	506	522	506
query80	1381	470	471	470
query81	568	223	226	223
query82	1051	109	111	109
query83	442	176	176	176
query84	274	85	92	85
query85	1375	293	268	268
query86	441	311	321	311
query87	3366	3079	3095	3079
query88	4340	2398	2361	2361
query89	490	395	403	395
query90	1887	190	191	190
query91	133	97	100	97
query92	62	48	49	48
query93	4198	503	504	503
query94	1254	192	192	192
query95	414	316	320	316
query96	605	274	261	261
query97	3204	3031	3100	3031
query98	223	210	188	188
query99	1132	857	823	823
Total cold run time: 274645 ms
Total hot run time: 174348 ms

@doris-robot
Copy link
Copy Markdown

doris-robot commented Jul 1, 2024

ClickBench: Total hot run time: 30.66 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit a9cb77374ded417f79358e833459848a86dc7bf7, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.04	0.04
query3	0.23	0.05	0.05
query4	1.70	0.08	0.08
query5	0.50	0.49	0.50
query6	1.14	0.74	0.72
query7	0.02	0.01	0.01
query8	0.05	0.04	0.04
query9	0.56	0.49	0.49
query10	0.56	0.56	0.54
query11	0.15	0.12	0.11
query12	0.14	0.12	0.12
query13	0.59	0.59	0.61
query14	0.78	0.77	0.78
query15	0.85	0.82	0.82
query16	0.35	0.37	0.37
query17	0.98	1.01	0.97
query18	0.21	0.25	0.26
query19	1.75	1.66	1.70
query20	0.02	0.01	0.01
query21	15.43	0.74	0.66
query22	4.25	7.36	2.09
query23	18.31	1.39	1.28
query24	2.09	0.21	0.24
query25	0.16	0.09	0.09
query26	0.26	0.18	0.17
query27	0.08	0.07	0.08
query28	13.20	1.01	0.99
query29	12.61	3.28	3.31
query30	0.25	0.06	0.06
query31	2.87	0.40	0.38
query32	3.27	0.47	0.48
query33	2.88	2.89	2.86
query34	17.09	4.39	4.43
query35	4.44	4.47	4.44
query36	0.65	0.47	0.48
query37	0.18	0.16	0.15
query38	0.15	0.15	0.14
query39	0.04	0.03	0.03
query40	0.17	0.14	0.15
query41	0.10	0.05	0.05
query42	0.06	0.05	0.04
query43	0.04	0.05	0.04
Total cold run time: 109.28 s
Total hot run time: 30.66 s

@amorynan amorynan requested a review from xiaokang July 1, 2024 08:15
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Jul 1, 2024

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jul 1, 2024
eldenmoon
eldenmoon approved these changes Jul 1, 2024
View reviewed changes
Copy link
Copy Markdown
Member

@eldenmoon eldenmoon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@eldenmoon eldenmoon merged commit 6a9f003 into apache:master Jul 1, 2024
dataroaring pushed a commit that referenced this pull request Jul 2, 2024
@dataroaring
[improve](json)improve json support empty keys (#36762)
9126492 
support empty key for json 
behavior reference MSYQL
This was referenced Jul 5, 2024
Merged
Merged
xiaokang pushed a commit that referenced this pull request Jul 8, 2024
[Improve]Improve support json with empty keys 2.0 #36762 (#37369)
be6ba90
xiaokang pushed a commit that referenced this pull request Jul 8, 2024
[improve](json)improve json support empty keys #36762 (#37351)
0a103aa
@xiaokang xiaokang mentioned this pull request Jul 14, 2024
Closed
@yiguolei yiguolei mentioned this pull request Jul 19, 2024
Closed
1 task
mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024
@Baymine
[Improve]Improve support json with empty keys 2.0 apache#36762 (apach…
f592877 
…e#37369)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@xiaokang xiaokang xiaokang approved these changes

@eldenmoon eldenmoon eldenmoon approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/2.0.13-merged dev/2.1.5-merged dev/3.0.0-merged reviewed usercase Important user case type label

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@amorynan @doris-robot @xiaokang @eldenmoon @wm1581066 @dataroaring