[fix](Nereids) fold constant for string function process emoji character by mistake by LiBinfeng-01 · Pull Request #49087 · apache/doris

LiBinfeng-01 · 2025-03-14T09:04:05Z

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #40441

Problem Summary:

Release note

wrong calculation of emoji character length in some String function when do constant folding in FE. For example:

select STRLEFT('😊😉👍', 2);

should return 😊😉, but fe return 😊 only when folding constant

fixed functions:

left
strleft
right
strright
locate
character_length
split_by_string
overlay
replace_empty

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

hello-stephen · 2025-03-14T09:04:11Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

LiBinfeng-01 · 2025-03-14T09:04:18Z

run buildall

LiBinfeng-01 · 2025-03-14T09:45:56Z

run buildall

doris-robot · 2025-03-14T10:13:48Z

TPC-H: Total hot run time: 32628 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ae2cca205e9c0edb345edf71858a000e2d5f7ea0, data reload: false

------ Round 1 ----------------------------------
q1	17579	5342	5051	5051
q2	2045	304	170	170
q3	10415	1309	776	776
q4	10208	1049	554	554
q5	7564	2439	2350	2350
q6	194	162	133	133
q7	944	767	626	626
q8	9306	1278	1162	1162
q9	5022	4658	4617	4617
q10	6823	2305	1868	1868
q11	496	269	253	253
q12	352	386	218	218
q13	17767	3688	3101	3101
q14	227	222	218	218
q15	546	491	481	481
q16	610	611	585	585
q17	579	880	364	364
q18	6874	6484	6369	6369
q19	1425	958	565	565
q20	330	341	199	199
q21	2902	2224	1984	1984
q22	1035	1033	984	984
Total cold run time: 103243 ms
Total hot run time: 32628 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5155	5149	5113	5113
q2	248	327	235	235
q3	2178	2687	2309	2309
q4	1481	1816	1413	1413
q5	4198	4149	4197	4149
q6	208	164	130	130
q7	1885	1923	1790	1790
q8	2696	2633	2588	2588
q9	7309	7219	7052	7052
q10	3033	3258	2800	2800
q11	631	502	483	483
q12	702	783	650	650
q13	3435	4043	3347	3347
q14	281	306	267	267
q15	543	512	492	492
q16	648	693	648	648
q17	1143	1644	1308	1308
q18	7757	7623	7431	7431
q19	806	869	1030	869
q20	2015	2060	1981	1981
q21	5552	4983	4851	4851
q22	1136	1073	1019	1019
Total cold run time: 53040 ms
Total hot run time: 50925 ms

doris-robot · 2025-03-14T10:26:01Z

TPC-DS: Total hot run time: 192255 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ae2cca205e9c0edb345edf71858a000e2d5f7ea0, data reload: false

query1	1394	1025	1016	1016
query2	6345	1958	1934	1934
query3	10998	4547	4522	4522
query4	56202	25204	23525	23525
query5	5228	508	484	484
query6	392	199	189	189
query7	4963	504	306	306
query8	319	255	230	230
query9	5798	2646	2637	2637
query10	402	314	245	245
query11	15317	15115	14870	14870
query12	160	107	105	105
query13	1039	522	392	392
query14	10202	6613	6861	6613
query15	215	205	197	197
query16	7031	662	487	487
query17	1087	746	600	600
query18	1567	405	320	320
query19	198	200	185	185
query20	127	130	125	125
query21	208	127	106	106
query22	4467	4435	4258	4258
query23	34042	33253	33297	33253
query24	6096	2483	2509	2483
query25	460	466	401	401
query26	664	278	158	158
query27	1857	485	345	345
query28	2749	2505	2467	2467
query29	593	572	448	448
query30	275	223	201	201
query31	875	894	801	801
query32	73	68	61	61
query33	459	368	306	306
query34	765	844	534	534
query35	807	842	739	739
query36	951	1006	913	913
query37	127	100	77	77
query38	4112	4252	4149	4149
query39	1468	1469	1442	1442
query40	208	120	107	107
query41	57	58	51	51
query42	120	105	100	100
query43	499	525	493	493
query44	1323	830	816	816
query45	179	174	163	163
query46	834	1047	697	697
query47	1816	1858	1766	1766
query48	384	414	299	299
query49	678	510	424	424
query50	727	781	418	418
query51	4282	4311	4171	4171
query52	106	108	99	99
query53	239	266	193	193
query54	489	508	449	449
query55	88	78	88	78
query56	291	290	256	256
query57	1191	1233	1110	1110
query58	252	260	239	239
query59	2880	2825	2714	2714
query60	301	279	281	279
query61	124	121	136	121
query62	749	747	683	683
query63	235	193	187	187
query64	1424	1038	703	703
query65	4581	4443	4520	4443
query66	725	405	321	321
query67	15930	15351	15216	15216
query68	7141	872	500	500
query69	541	301	259	259
query70	1095	1134	1090	1090
query71	469	301	265	265
query72	5811	3608	3735	3608
query73	1201	754	350	350
query74	9016	9199	8671	8671
query75	3418	3172	2725	2725
query76	3786	1186	760	760
query77	567	449	274	274
query78	10053	10082	9307	9307
query79	2388	871	583	583
query80	670	527	466	466
query81	482	253	219	219
query82	317	124	95	95
query83	173	167	148	148
query84	291	95	73	73
query85	991	362	313	313
query86	437	287	274	274
query87	4350	4612	4341	4341
query88	3579	2296	2291	2291
query89	408	313	284	284
query90	1853	209	208	208
query91	142	140	111	111
query92	70	63	53	53
query93	2301	1051	583	583
query94	625	417	299	299
query95	357	274	271	271
query96	485	581	277	277
query97	3341	3419	3327	3327
query98	220	202	201	201
query99	1306	1387	1309	1309
Total cold run time: 298628 ms
Total hot run time: 192255 ms

doris-robot · 2025-03-14T10:31:20Z

ClickBench: Total hot run time: 30.54 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit ae2cca205e9c0edb345edf71858a000e2d5f7ea0, data reload: false

query1	0.03	0.04	0.04
query2	0.06	0.04	0.03
query3	0.24	0.06	0.07
query4	1.61	0.10	0.11
query5	0.58	0.55	0.53
query6	1.21	0.73	0.72
query7	0.03	0.01	0.01
query8	0.04	0.03	0.03
query9	0.58	0.52	0.51
query10	0.59	0.58	0.58
query11	0.16	0.11	0.11
query12	0.14	0.12	0.11
query13	0.63	0.60	0.61
query14	2.77	2.69	2.80
query15	0.94	0.85	0.86
query16	0.39	0.39	0.38
query17	1.04	1.05	1.00
query18	0.21	0.19	0.20
query19	1.93	1.88	1.99
query20	0.02	0.01	0.01
query21	15.38	0.92	0.54
query22	0.76	1.19	0.64
query23	14.94	1.40	0.65
query24	7.60	1.42	0.44
query25	0.51	0.27	0.13
query26	0.64	0.16	0.13
query27	0.06	0.06	0.05
query28	9.39	0.88	0.44
query29	12.66	3.98	3.28
query30	0.25	0.09	0.06
query31	2.82	0.58	0.39
query32	3.23	0.55	0.47
query33	3.00	3.04	3.10
query34	15.74	5.16	4.49
query35	4.52	4.61	4.53
query36	0.66	0.50	0.49
query37	0.10	0.07	0.06
query38	0.05	0.04	0.04
query39	0.03	0.03	0.03
query40	0.17	0.15	0.13
query41	0.09	0.03	0.03
query42	0.03	0.02	0.02
query43	0.04	0.04	0.03
Total cold run time: 105.87 s
Total hot run time: 30.54 s

LiBinfeng-01 · 2025-03-17T07:34:48Z

run buildall

LiBinfeng-01 · 2025-03-17T08:38:21Z

run buildall

LiBinfeng-01 · 2025-03-18T03:35:52Z

run buildall

doris-robot · 2025-03-18T04:15:02Z

TPC-H: Total hot run time: 32319 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 938981ec446397d4c18b5a17e17509ccfa579d1c, data reload: false

------ Round 1 ----------------------------------
q1	24427	5092	5068	5068
q2	2056	343	192	192
q3	10318	1233	675	675
q4	10215	999	522	522
q5	7518	2379	2289	2289
q6	186	162	131	131
q7	924	749	629	629
q8	9297	1221	1101	1101
q9	5193	4686	4618	4618
q10	6797	2282	1890	1890
q11	483	269	259	259
q12	347	347	215	215
q13	17774	3739	3242	3242
q14	231	230	224	224
q15	543	487	479	479
q16	628	631	586	586
q17	587	865	353	353
q18	7062	6449	6199	6199
q19	1225	943	551	551
q20	330	342	201	201
q21	2719	2075	1932	1932
q22	1021	1002	963	963
Total cold run time: 109881 ms
Total hot run time: 32319 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5214	5134	5133	5133
q2	228	319	229	229
q3	2158	2648	2287	2287
q4	1411	1810	1335	1335
q5	4230	4072	4339	4072
q6	210	162	125	125
q7	1990	1916	1766	1766
q8	2602	2590	2518	2518
q9	7229	7124	7217	7124
q10	3000	3229	2657	2657
q11	568	489	490	489
q12	710	819	607	607
q13	3541	3913	3301	3301
q14	280	288	275	275
q15	520	477	457	457
q16	658	703	677	677
q17	1117	1585	1345	1345
q18	7689	7620	7581	7581
q19	770	799	872	799
q20	1976	2033	1901	1901
q21	5462	4784	4904	4784
q22	1081	1061	1031	1031
Total cold run time: 52644 ms
Total hot run time: 50493 ms

doris-robot · 2025-03-18T04:27:11Z

TPC-DS: Total hot run time: 191392 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 938981ec446397d4c18b5a17e17509ccfa579d1c, data reload: false

query1	1404	1029	1029	1029
query2	6069	1901	1864	1864
query3	11021	4581	4423	4423
query4	54047	24795	23056	23056
query5	5236	529	491	491
query6	404	201	195	195
query7	5210	495	293	293
query8	332	248	235	235
query9	7015	2530	2542	2530
query10	394	313	262	262
query11	15267	15027	14949	14949
query12	161	107	104	104
query13	1227	509	376	376
query14	10038	6363	6244	6244
query15	193	195	177	177
query16	7034	647	453	453
query17	1037	717	578	578
query18	1507	399	315	315
query19	193	185	171	171
query20	131	123	127	123
query21	204	129	110	110
query22	4370	4584	4513	4513
query23	33891	33337	33236	33236
query24	6570	2439	2472	2439
query25	439	472	405	405
query26	679	279	150	150
query27	2124	499	339	339
query28	3036	2436	2402	2402
query29	558	564	427	427
query30	265	219	189	189
query31	899	865	810	810
query32	72	66	62	62
query33	444	383	327	327
query34	754	882	498	498
query35	830	845	783	783
query36	948	980	898	898
query37	122	103	78	78
query38	4144	4377	4166	4166
query39	1486	1447	1453	1447
query40	221	113	103	103
query41	52	52	49	49
query42	122	107	104	104
query43	489	517	474	474
query44	1331	807	814	807
query45	176	176	166	166
query46	847	1048	657	657
query47	1915	1848	1786	1786
query48	397	435	317	317
query49	706	524	443	443
query50	704	746	436	436
query51	4277	4394	4276	4276
query52	111	111	95	95
query53	236	267	193	193
query54	491	523	415	415
query55	83	81	83	81
query56	272	252	272	252
query57	1182	1209	1123	1123
query58	244	252	245	245
query59	2670	2893	2650	2650
query60	287	282	260	260
query61	128	122	125	122
query62	719	755	672	672
query63	225	198	193	193
query64	1440	1050	669	669
query65	4616	4540	4462	4462
query66	790	400	312	312
query67	15834	15719	15527	15527
query68	6934	865	496	496
query69	547	317	268	268
query70	1207	1133	1115	1115
query71	457	302	298	298
query72	5704	3582	3685	3582
query73	1083	743	348	348
query74	9206	9007	8674	8674
query75	3514	3158	2712	2712
query76	3985	1208	751	751
query77	636	377	274	274
query78	9905	10123	9339	9339
query79	1912	831	573	573
query80	595	584	440	440
query81	481	247	215	215
query82	310	129	93	93
query83	174	167	153	153
query84	285	87	73	73
query85	809	355	303	303
query86	405	305	274	274
query87	4387	4564	4315	4315
query88	3521	2225	2212	2212
query89	399	315	277	277
query90	1899	206	206	206
query91	142	141	109	109
query92	77	61	54	54
query93	1605	1057	583	583
query94	653	394	293	293
query95	348	270	265	265
query96	473	559	273	273
query97	3351	3396	3297	3297
query98	221	207	204	204
query99	1351	1400	1287	1287
Total cold run time: 296896 ms
Total hot run time: 191392 ms

doris-robot · 2025-03-18T04:32:30Z

ClickBench: Total hot run time: 30.77 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 938981ec446397d4c18b5a17e17509ccfa579d1c, data reload: false

query1	0.04	0.03	0.03
query2	0.12	0.11	0.10
query3	0.25	0.19	0.20
query4	1.59	0.19	0.20
query5	0.61	0.58	0.58
query6	1.20	0.72	0.72
query7	0.02	0.01	0.02
query8	0.04	0.03	0.04
query9	0.58	0.52	0.56
query10	0.57	0.60	0.58
query11	0.15	0.11	0.11
query12	0.14	0.11	0.11
query13	0.61	0.61	0.60
query14	2.67	2.71	2.69
query15	0.93	0.86	0.84
query16	0.39	0.40	0.37
query17	1.00	1.02	1.03
query18	0.21	0.19	0.19
query19	1.96	1.91	1.83
query20	0.01	0.01	0.01
query21	15.38	0.90	0.56
query22	0.74	1.32	0.71
query23	14.77	1.40	0.66
query24	7.60	2.03	0.37
query25	0.34	0.38	0.08
query26	0.66	0.16	0.14
query27	0.06	0.04	0.04
query28	9.23	0.77	0.43
query29	12.95	4.10	3.30
query30	0.25	0.08	0.07
query31	2.83	0.60	0.38
query32	3.22	0.54	0.46
query33	3.02	3.04	3.01
query34	15.81	5.12	4.52
query35	4.54	4.54	4.53
query36	0.66	0.49	0.49
query37	0.08	0.07	0.06
query38	0.05	0.04	0.03
query39	0.03	0.03	0.02
query40	0.18	0.13	0.12
query41	0.09	0.03	0.02
query42	0.03	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 105.64 s
Total hot run time: 30.77 s

LiBinfeng-01 · 2025-03-18T08:08:28Z

run buildall

LiBinfeng-01 · 2025-03-18T08:14:25Z

run buildall

LiBinfeng-01 · 2025-03-20T09:07:08Z

run buildall

doris-robot · 2025-03-20T09:28:54Z

TPC-H: Total hot run time: 32783 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit eada04c15dd6159dc39ae469d60fd8d1e1057839, data reload: false

------ Round 1 ----------------------------------
q1	24441	5211	5142	5142
q2	2048	313	173	173
q3	10476	1299	731	731
q4	10270	1041	561	561
q5	8901	2483	2397	2397
q6	193	170	138	138
q7	950	782	616	616
q8	9327	1328	1109	1109
q9	5194	4578	4834	4578
q10	6817	2316	1910	1910
q11	520	272	266	266
q12	349	358	222	222
q13	17773	3698	3159	3159
q14	229	240	224	224
q15	536	489	479	479
q16	631	603	566	566
q17	587	885	363	363
q18	7229	6439	6441	6439
q19	1352	971	571	571
q20	336	339	196	196
q21	3019	2175	1971	1971
q22	1055	1048	972	972
Total cold run time: 112233 ms
Total hot run time: 32783 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5236	5195	5193	5193
q2	241	326	226	226
q3	2193	2725	2284	2284
q4	1457	1831	1376	1376
q5	4275	4439	4473	4439
q6	223	172	138	138
q7	2048	1938	1733	1733
q8	2636	2718	2639	2639
q9	7230	7206	7125	7125
q10	3002	3219	2740	2740
q11	590	496	498	496
q12	683	759	622	622
q13	3583	3917	3256	3256
q14	276	291	288	288
q15	518	480	485	480
q16	654	686	652	652
q17	1160	1825	1307	1307
q18	7863	7780	7510	7510
q19	814	795	840	795
q20	1982	2014	1862	1862
q21	5347	4862	4724	4724
q22	1086	1025	1015	1015
Total cold run time: 53097 ms
Total hot run time: 50900 ms

doris-robot · 2025-03-20T09:40:30Z

TPC-DS: Total hot run time: 186488 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit eada04c15dd6159dc39ae469d60fd8d1e1057839, data reload: false

query1	1031	495	495	495
query2	6543	1920	1872	1872
query3	6812	223	220	220
query4	26331	23756	23631	23631
query5	4884	684	515	515
query6	310	204	196	196
query7	4620	519	296	296
query8	309	253	247	247
query9	8672	2620	2630	2620
query10	478	316	251	251
query11	15594	15374	14874	14874
query12	174	109	105	105
query13	1639	550	417	417
query14	10145	6333	6724	6333
query15	207	182	172	172
query16	7420	663	494	494
query17	1123	695	545	545
query18	1910	400	303	303
query19	186	188	151	151
query20	123	122	121	121
query21	205	123	105	105
query22	4315	4115	4197	4115
query23	33903	33022	33003	33003
query24	7860	2378	2390	2378
query25	577	492	432	432
query26	1239	289	161	161
query27	2521	498	338	338
query28	4263	2452	2399	2399
query29	777	592	465	465
query30	337	220	193	193
query31	956	876	782	782
query32	75	63	72	63
query33	568	369	328	328
query34	796	858	488	488
query35	785	792	751	751
query36	974	994	911	911
query37	128	100	76	76
query38	4236	4256	4231	4231
query39	1457	1418	1543	1418
query40	210	115	100	100
query41	62	59	50	50
query42	119	103	118	103
query43	496	516	490	490
query44	1359	815	799	799
query45	190	169	168	168
query46	873	1047	628	628
query47	1749	1805	1712	1712
query48	373	422	305	305
query49	802	516	409	409
query50	752	773	425	425
query51	4232	4386	4201	4201
query52	101	112	99	99
query53	237	266	191	191
query54	492	503	406	406
query55	85	82	81	81
query56	271	282	253	253
query57	1154	1142	1056	1056
query58	251	246	235	235
query59	2623	2738	2598	2598
query60	280	286	259	259
query61	142	155	127	127
query62	806	730	654	654
query63	234	186	192	186
query64	4309	1037	698	698
query65	4499	4368	4359	4359
query66	1113	407	307	307
query67	15867	15290	15199	15199
query68	8473	890	501	501
query69	480	299	258	258
query70	1269	1116	1137	1116
query71	459	297	260	260
query72	5700	3663	3800	3663
query73	787	753	346	346
query74	9256	8905	8999	8905
query75	3747	3204	2714	2714
query76	3655	1199	777	777
query77	783	386	288	288
query78	9986	10237	9338	9338
query79	1492	909	580	580
query80	637	566	454	454
query81	486	269	219	219
query82	609	134	100	100
query83	181	169	163	163
query84	241	91	74	74
query85	773	352	309	309
query86	346	316	298	298
query87	4399	4480	4396	4396
query88	2888	2238	2262	2238
query89	399	334	291	291
query90	1926	225	218	218
query91	143	142	109	109
query92	77	59	58	58
query93	1151	1093	594	594
query94	677	412	307	307
query95	355	284	268	268
query96	484	628	281	281
query97	3323	3385	3290	3290
query98	214	212	200	200
query99	1370	1403	1282	1282
Total cold run time: 274930 ms
Total hot run time: 186488 ms

doris-robot · 2025-03-20T09:45:51Z

ClickBench: Total hot run time: 31.58 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit eada04c15dd6159dc39ae469d60fd8d1e1057839, data reload: false

query1	0.04	0.03	0.03
query2	0.12	0.11	0.10
query3	0.25	0.19	0.19
query4	1.60	0.20	0.11
query5	0.56	0.55	0.55
query6	1.20	0.72	0.71
query7	0.02	0.02	0.01
query8	0.05	0.04	0.03
query9	0.59	0.52	0.51
query10	0.61	0.58	0.57
query11	0.16	0.10	0.11
query12	0.15	0.11	0.12
query13	0.62	0.59	0.60
query14	2.67	2.71	2.69
query15	0.92	0.86	0.85
query16	0.38	0.39	0.39
query17	1.03	1.02	1.03
query18	0.22	0.19	0.19
query19	2.03	1.86	1.86
query20	0.02	0.01	0.01
query21	15.36	0.93	0.54
query22	0.77	1.20	0.77
query23	14.79	1.43	0.67
query24	6.77	2.01	1.23
query25	0.49	0.21	0.06
query26	0.52	0.16	0.13
query27	0.06	0.05	0.05
query28	9.66	0.79	0.42
query29	12.54	4.00	3.35
query30	0.26	0.09	0.07
query31	2.87	0.58	0.38
query32	3.23	0.56	0.48
query33	3.03	3.03	3.08
query34	15.75	5.16	4.48
query35	4.60	4.51	4.55
query36	0.67	0.49	0.49
query37	0.08	0.06	0.06
query38	0.05	0.04	0.03
query39	0.02	0.02	0.02
query40	0.18	0.14	0.12
query41	0.08	0.02	0.02
query42	0.04	0.02	0.02
query43	0.04	0.04	0.03
Total cold run time: 105.1 s
Total hot run time: 31.58 s

github-actions · 2025-03-20T11:53:56Z

PR approved by at least one committer and no changes requested.

github-actions · 2025-03-20T11:53:58Z

PR approved by anyone and no changes requested.

aytrack

LGTM

…ter by mistake (apache#49087) Related PR: apache#40441 Problem Summary: wrong calculation of emoji character length in some String function when do constant folding in FE. For example: select STRLEFT('😊😉👍', 2); should return 😊😉, but fe return 😊 only when folding constant fixed functions: - left - strleft - right - strright - locate - character_length - split_by_string - overlay - replace_empty

…emoji character by mistake #49087 (#49344) pick: #49087 Related PR: #40441 Problem Summary: wrong calculation of emoji character length in some String function when do constant folding in FE. For example: select STRLEFT('😊😉👍', 2); should return 😊😉, but fe return 😊 only when folding constant fixed functions: - left - strleft - right - strright - locate - character_length - split_by_string - overlay - replace_empty

…emoji character by mistake #49087 (#49346) pick: #49087 Related PR: #40441 Problem Summary: wrong calculation of emoji character length in some String function when do constant folding in FE. For example: select STRLEFT('😊😉👍', 2); should return 😊😉, but fe return 😊 only when folding constant fixed functions: - left - strleft - right - strright - locate - character_length - split_by_string - overlay - replace_empty

…ter by mistake (apache#49087) ### What problem does this PR solve? Related PR: apache#40441 Problem Summary: wrong calculation of emoji character length in some String function when do constant folding in FE. For example: select STRLEFT('😊😉👍', 2); should return 😊😉, but fe return 😊 only when folding constant fixed functions: - left - strleft - right - strright - locate - character_length - split_by_string - overlay - replace_empty

morrySnow changed the title ~~[fix](Nereids) fix emoji character error~~ [fix](Nereids) fold constant for string function process emoji character my mistake Mar 14, 2025

LiBinfeng-01 force-pushed the fix_emoji branch from 785f6f4 to ae2cca2 Compare March 14, 2025 09:45

morrySnow reviewed Mar 14, 2025

View reviewed changes

Comment thread ...n/java/org/apache/doris/nereids/trees/expressions/functions/executable/StringArithmetic.java Outdated

Comment thread ...n/java/org/apache/doris/nereids/trees/expressions/functions/executable/StringArithmetic.java

morrySnow added p0_w dev/2.1.x dev/3.0.x labels Mar 17, 2025

morrySnow reviewed Mar 17, 2025

View reviewed changes

LiBinfeng-01 force-pushed the fix_emoji branch from fba1144 to 938981e Compare March 18, 2025 03:35

morrySnow reviewed Mar 18, 2025

View reviewed changes

morrySnow changed the title ~~[fix](Nereids) fold constant for string function process emoji character my mistake~~ [fix](Nereids) fold constant for string function process emoji character by mistake Mar 18, 2025

LiBinfeng added 6 commits March 19, 2025 11:26

[fix](Nereids) fix emoji character error

d061bc5

fix regression bugs

d760b46

fix comments

7947fb0

fix out of boundary problems

e1aa954

fix comments

a5d3615

fix comments and add some uts

e5be7c1

fix comments3

eada04c

morrySnow approved these changes Mar 20, 2025

View reviewed changes

github-actions Bot added the approved Indicates a PR has been approved by one committer. label Mar 20, 2025

github-actions Bot added the reviewed label Mar 20, 2025

aytrack approved these changes Mar 20, 2025

View reviewed changes

morrySnow merged commit 095a2ae into apache:master Mar 21, 2025
27 of 28 checks passed

github-actions Bot added dev/3.0.x-conflict dev/2.1.x-conflict labels Mar 21, 2025

LiBinfeng-01 mentioned this pull request Mar 21, 2025

branch-2.1: [fix](Nereids) fold constant for string function process emoji character by mistake #49087 #49344

Merged

16 tasks

LiBinfeng-01 mentioned this pull request Mar 21, 2025

branch-3.0: [fix](Nereids) fold constant for string function process emoji character by mistake #49087 #49346

Merged

16 tasks

yiguolei added dev/2.1.9-merged and removed dev/2.1.x dev/2.1.x-conflict labels Mar 21, 2025

dataroaring added dev/3.0.5-merged and removed dev/3.0.x dev/3.0.x-conflict labels Mar 27, 2025

gavinchou mentioned this pull request Apr 23, 2025

Release Notes 3.0.5 #50333

Closed

Conversation

LiBinfeng-01 commented Mar 14, 2025 • edited by morrySnow Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

hello-stephen commented Mar 14, 2025

Uh oh!

LiBinfeng-01 commented Mar 14, 2025

Uh oh!

LiBinfeng-01 commented Mar 14, 2025

Uh oh!

Uh oh!

Uh oh!

doris-robot commented Mar 14, 2025

Uh oh!

doris-robot commented Mar 14, 2025

Uh oh!

doris-robot commented Mar 14, 2025

Uh oh!

LiBinfeng-01 commented Mar 17, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

LiBinfeng-01 commented Mar 17, 2025

Uh oh!

LiBinfeng-01 commented Mar 18, 2025

Uh oh!

doris-robot commented Mar 18, 2025

Uh oh!

doris-robot commented Mar 18, 2025

Uh oh!

doris-robot commented Mar 18, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

LiBinfeng-01 commented Mar 18, 2025

Uh oh!

LiBinfeng-01 commented Mar 18, 2025

Uh oh!

LiBinfeng-01 commented Mar 20, 2025

Uh oh!

doris-robot commented Mar 20, 2025

Uh oh!

doris-robot commented Mar 20, 2025

Uh oh!

doris-robot commented Mar 20, 2025

Uh oh!

github-actions Bot commented Mar 20, 2025

Uh oh!

github-actions Bot commented Mar 20, 2025

Uh oh!

aytrack left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

LiBinfeng-01 commented Mar 14, 2025 •

edited by morrySnow

Loading