[fix](update null map) Fix update_null_map by zhiqiang-hhhh · Pull Request #38787 · apache/doris

zhiqiang-hhhh · 2024-08-02T09:09:38Z

When is_single == true and r[0]==false, we will get unstable result.

doris-robot · 2024-08-02T09:09:42Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

zhiqiang-hhhh · 2024-08-02T09:10:53Z

run buildall

github-actions · 2024-08-02T09:16:12Z

clang-tidy review says "All clean, LGTM! 👍"

zclllyybb

LGTM

github-actions · 2024-08-02T09:17:25Z

PR approved by anyone and no changes requested.

doris-robot · 2024-08-02T10:08:36Z

TPC-H: Total hot run time: 42207 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 1ef213414848bbf418df6e9ba85f911c3dbdfeea, data reload: false

------ Round 1 ----------------------------------
q1	18142	4262	4263	4262
q2	2387	205	196	196
q3	10455	1367	1366	1366
q4	10178	890	938	890
q5	7578	2979	3001	2979
q6	223	138	141	138
q7	1086	615	613	613
q8	9437	1970	1991	1970
q9	8554	6639	6634	6634
q10	8728	3840	3863	3840
q11	425	278	248	248
q12	427	228	229	228
q13	17759	2941	2941	2941
q14	275	245	246	245
q15	525	487	491	487
q16	490	399	391	391
q17	989	932	920	920
q18	8091	7356	7244	7244
q19	1385	1228	1229	1228
q20	578	322	331	322
q21	5330	4785	4803	4785
q22	359	280	280	280
Total cold run time: 113401 ms
Total hot run time: 42207 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4100	4043	4089	4043
q2	332	233	220	220
q3	3017	2996	2985	2985
q4	1914	1902	1881	1881
q5	5263	5248	5263	5248
q6	221	131	131	131
q7	2043	1625	1679	1625
q8	3213	3304	3279	3279
q9	8302	8327	8329	8327
q10	3770	3839	3837	3837
q11	552	465	447	447
q12	733	561	567	561
q13	16754	2945	2920	2920
q14	284	273	256	256
q15	513	472	475	472
q16	438	398	409	398
q17	1733	1716	1722	1716
q18	7789	7388	7189	7189
q19	1729	1676	1671	1671
q20	1972	1770	1726	1726
q21	5524	5176	5251	5176
q22	506	451	478	451
Total cold run time: 70702 ms
Total hot run time: 54559 ms

doris-robot · 2024-08-02T10:19:31Z

TPC-DS: Total hot run time: 169547 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 1ef213414848bbf418df6e9ba85f911c3dbdfeea, data reload: false

query1	926	382	370	370
query2	6489	1650	1672	1650
query3	6709	215	226	215
query4	19697	17457	17524	17457
query5	4270	507	518	507
query6	274	165	176	165
query7	4601	307	314	307
query8	267	193	208	193
query9	8562	2366	2348	2348
query10	446	282	260	260
query11	10671	10010	10247	10010
query12	144	95	90	90
query13	1636	383	380	380
query14	9880	6879	7588	6879
query15	205	173	170	170
query16	7101	449	478	449
query17	980	567	565	565
query18	1764	291	289	289
query19	212	160	152	152
query20	96	86	91	86
query21	207	100	104	100
query22	4059	4043	3956	3956
query23	33889	33136	32985	32985
query24	10349	3096	3023	3023
query25	700	419	413	413
query26	1775	157	156	156
query27	2893	274	281	274
query28	6908	1963	1960	1960
query29	1280	448	439	439
query30	290	155	153	153
query31	941	772	769	769
query32	106	55	60	55
query33	724	324	338	324
query34	934	501	491	491
query35	878	738	733	733
query36	1001	851	885	851
query37	300	82	82	82
query38	2866	2798	2757	2757
query39	874	833	805	805
query40	289	113	116	113
query41	48	53	48	48
query42	131	111	110	110
query43	480	406	409	406
query44	1155	747	749	747
query45	213	181	182	181
query46	1095	790	801	790
query47	1818	1745	1739	1739
query48	379	305	304	304
query49	1192	452	452	452
query50	905	446	438	438
query51	6692	6736	6640	6640
query52	104	99	94	94
query53	260	182	186	182
query54	679	463	472	463
query55	79	79	78	78
query56	296	267	282	267
query57	1158	1063	1039	1039
query58	284	288	283	283
query59	2533	2471	2490	2471
query60	326	293	287	287
query61	130	115	114	114
query62	1049	662	690	662
query63	221	186	182	182
query64	5908	1926	1900	1900
query65	3174	3113	3094	3094
query66	1402	344	337	337
query67	15527	14921	14833	14833
query68	6295	580	602	580
query69	767	360	320	320
query70	1122	1085	1056	1056
query71	554	286	280	280
query72	8148	2671	2514	2514
query73	945	344	329	329
query74	6074	5675	5707	5675
query75	4409	2721	2724	2721
query76	4445	1267	1294	1267
query77	747	308	307	307
query78	9439	8965	8890	8890
query79	4060	542	572	542
query80	930	523	509	509
query81	557	226	227	226
query82	739	133	130	130
query83	345	173	180	173
query84	269	78	81	78
query85	1417	320	306	306
query86	426	293	279	279
query87	3333	3131	3095	3095
query88	3589	2525	2524	2524
query89	484	294	289	289
query90	2080	197	193	193
query91	127	102	98	98
query92	65	48	50	48
query93	4727	612	610	610
query94	964	293	306	293
query95	396	272	275	272
query96	624	295	290	290
query97	3247	3093	3080	3080
query98	219	209	194	194
query99	1604	1257	1292	1257
Total cold run time: 276578 ms
Total hot run time: 169547 ms

doris-robot · 2024-08-02T10:25:01Z

ClickBench: Total hot run time: 29.96 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 1ef213414848bbf418df6e9ba85f911c3dbdfeea, data reload: false

query1	0.05	0.04	0.03
query2	0.08	0.03	0.04
query3	0.22	0.05	0.05
query4	1.67	0.07	0.07
query5	0.49	0.48	0.48
query6	1.13	0.72	0.72
query7	0.02	0.01	0.02
query8	0.05	0.05	0.05
query9	0.61	0.50	0.52
query10	0.56	0.56	0.56
query11	0.16	0.12	0.12
query12	0.15	0.12	0.12
query13	0.63	0.60	0.59
query14	0.77	0.80	0.78
query15	0.90	0.87	0.85
query16	0.35	0.35	0.34
query17	0.98	1.00	1.00
query18	0.22	0.21	0.22
query19	1.86	1.75	1.77
query20	0.01	0.01	0.01
query21	15.40	0.76	0.67
query22	4.16	8.19	1.16
query23	17.83	1.38	1.37
query24	2.27	0.23	0.23
query25	0.18	0.08	0.07
query26	0.31	0.21	0.21
query27	0.46	0.23	0.22
query28	13.17	1.01	0.97
query29	12.50	3.31	3.28
query30	0.26	0.05	0.05
query31	2.84	0.41	0.40
query32	3.22	0.50	0.48
query33	2.90	3.02	2.97
query34	15.45	4.23	4.25
query35	4.29	4.31	4.29
query36	0.68	0.47	0.48
query37	0.21	0.17	0.17
query38	0.17	0.16	0.17
query39	0.05	0.03	0.03
query40	0.16	0.12	0.14
query41	0.10	0.05	0.05
query42	0.06	0.05	0.05
query43	0.05	0.04	0.04
Total cold run time: 107.63 s
Total hot run time: 29.96 s

HappenLee

LGTM

github-actions · 2024-08-03T06:53:58Z

PR approved by at least one committer and no changes requested.

## Proposed changes Issue Number: close #xxx This pr make `convert_tz` use default nullity processing logic rather than `update_null_map` manually to get correct result. The bug of `update_null_map` will be fixed by #38787 before: ```sql mysql> select convert_tz(dt, '+00:00', IF(property_value IS NULL, '+00:00', property_value)) from cvt_tz; +--------------------------------------------------------------------------------+ | convert_tz(dt, '+00:00', if(property_value IS NULL, '+00:00', property_value)) | +--------------------------------------------------------------------------------+ | 2024-04-18 23:20:00 | | | | | | 2024-04-18 23:20:00 | | | | | | 2024-04-18 23:20:00 | | | | 2024-04-18 23:20:00 | | | | 2024-04-18 23:20:00 | | 2024-04-18 23:20:00 | | | | | | 2024-04-18 23:20:00 | | | | 2024-04-18 23:20:00 | | 2024-04-18 23:20:00 | | | | 2024-04-18 23:20:00 | +--------------------------------------------------------------------------------+ ``` after: ```sql mysql> select convert_tz(dt, '+00:00', IF(property_value IS NULL, '+00:00', property_value)) from cvt_tz; +--------------------------------------------------------------------------------+ | convert_tz(dt, '+00:00', if(property_value IS NULL, '+00:00', property_value)) | +--------------------------------------------------------------------------------+ | 2024-04-18 23:20:00 | | 2024-04-18 23:20:00 | | 2024-04-18 23:20:00 | | 2024-04-18 23:20:00 | | 2024-04-18 23:20:00 | | 2024-04-18 23:20:00 | | 2024-04-18 23:20:00 | | 2024-04-18 23:20:00 | | 2024-04-18 23:20:00 | | 2024-04-18 23:20:00 | | 2024-04-18 23:20:00 | | 2024-04-18 23:20:00 | | 2024-04-18 23:20:00 | | 2024-04-18 23:20:00 | | 2024-04-18 23:20:00 | | 2024-04-18 23:20:00 | | 2024-04-18 23:20:00 | | 2024-04-18 23:20:00 | | 2024-04-18 23:20:00 | | 2024-04-18 23:20:00 | +--------------------------------------------------------------------------------+ ```

zhiqiang-hhhh · 2024-08-06T02:08:26Z

run cloud_p0

When `is_single == true` and `r[0]==false`, we will get unstable result.

cherry pick from #38787

When `is_single == true` and `r[0]==false`, we will get unstable result.

zhiqiang-hhhh added 2 commits August 2, 2024 17:07

FIX

e401743

FIX

1ef2134

github-actions bot added the doing label Aug 2, 2024

zclllyybb approved these changes Aug 2, 2024

View reviewed changes

github-actions bot added the reviewed label Aug 2, 2024

HappenLee approved these changes Aug 3, 2024

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Aug 3, 2024

yiguolei added dev/2.0.x dev/2.1.x labels Aug 5, 2024

zclllyybb mentioned this pull request Aug 5, 2024

[fix](function) Fix wrong nullable processing for convert_tz #38764

Merged

yiguolei merged commit 39c9e35 into apache:master Aug 6, 2024

zhiqiang-hhhh added a commit to zhiqiang-hhhh/doris that referenced this pull request Aug 6, 2024

[fix](update null map) Fix update_null_map (apache#38787)

51d1c73

When `is_single == true` and `r[0]==false`, we will get unstable result.

zhiqiang-hhhh mentioned this pull request Aug 6, 2024

[fix](update null map) Fix update_null_map #38787 #38920

Merged

zhiqiang-hhhh added a commit to zhiqiang-hhhh/doris that referenced this pull request Aug 6, 2024

[fix](update null map) Fix update_null_map (apache#38787)

81c9271

When `is_single == true` and `r[0]==false`, we will get unstable result.

zhiqiang-hhhh mentioned this pull request Aug 6, 2024

[fix](update null map) Fix update_null_map #38787 #38921

Closed

zhiqiang-hhhh deleted the fix-update-null branch August 6, 2024 03:09

yiguolei pushed a commit that referenced this pull request Aug 7, 2024

[fix](update null map) Fix update_null_map #38787 (#38920)

e400859

cherry pick from #38787

yiguolei added dev/2.1.6-merged and removed dev/2.1.x labels Aug 7, 2024

dataroaring pushed a commit that referenced this pull request Aug 7, 2024

[fix](update null map) Fix update_null_map (#38787)

a961ddb

When `is_single == true` and `r[0]==false`, we will get unstable result.

dataroaring added the dev/3.0.1-merged label Aug 7, 2024

gavinchou mentioned this pull request Aug 19, 2024

Release Note 3.0.1 #39570

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[fix](update null map) Fix update_null_map#38787

[fix](update null map) Fix update_null_map#38787
yiguolei merged 2 commits intoapache:masterfrom
zhiqiang-hhhh:fix-update-null

zhiqiang-hhhh commented Aug 2, 2024

Uh oh!

doris-robot commented Aug 2, 2024

Uh oh!

zhiqiang-hhhh commented Aug 2, 2024

Uh oh!

github-actions bot commented Aug 2, 2024

Uh oh!

zclllyybb left a comment

Uh oh!

github-actions bot commented Aug 2, 2024

Uh oh!

doris-robot commented Aug 2, 2024

Uh oh!

doris-robot commented Aug 2, 2024

Uh oh!

doris-robot commented Aug 2, 2024

Uh oh!

HappenLee left a comment

Uh oh!

github-actions bot commented Aug 3, 2024

Uh oh!

zhiqiang-hhhh commented Aug 6, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Conversation

zhiqiang-hhhh commented Aug 2, 2024

Uh oh!

doris-robot commented Aug 2, 2024

Uh oh!

zhiqiang-hhhh commented Aug 2, 2024

Uh oh!

github-actions bot commented Aug 2, 2024

Uh oh!

zclllyybb left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Aug 2, 2024

Uh oh!

doris-robot commented Aug 2, 2024

Uh oh!

doris-robot commented Aug 2, 2024

Uh oh!

doris-robot commented Aug 2, 2024

Uh oh!

HappenLee left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Aug 3, 2024

Uh oh!

zhiqiang-hhhh commented Aug 6, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants