Skip to content

[fix](inverted index) Incorrect handling of NULL literal#37921

Merged
airborne12 merged 1 commit intoapache:masterfrom
zzzxl1993:202407161429
Jul 17, 2024
Merged

[fix](inverted index) Incorrect handling of NULL literal#37921
airborne12 merged 1 commit intoapache:masterfrom
zzzxl1993:202407161429

Conversation

@zzzxl1993
Copy link
Copy Markdown
Contributor

@zzzxl1993 zzzxl1993 commented Jul 16, 2024 •
edited
Loading

Proposed changes

  1. Index does not handle NULL literal for now

@doris-robot
Copy link
Copy Markdown

doris-robot commented Jul 16, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@zzzxl1993
Copy link
Copy Markdown
Contributor Author

zzzxl1993 commented Jul 16, 2024

run buildall

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Jul 16, 2024

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link
Copy Markdown

doris-robot commented Jul 16, 2024

TPC-H: Total hot run time: 40143 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 3391c51634a8828c2817653e1bac74c2cb4b1cb7, data reload: false

------ Round 1 ----------------------------------
q1	17600	4447	4301	4301
q2	2016	206	184	184
q3	10441	1178	1102	1102
q4	10187	852	850	850
q5	7621	2760	2763	2760
q6	224	141	137	137
q7	955	598	593	593
q8	9213	2089	2098	2089
q9	8755	6597	6586	6586
q10	8902	3765	3799	3765
q11	466	238	241	238
q12	396	226	226	226
q13	17771	2980	3007	2980
q14	280	231	238	231
q15	527	487	476	476
q16	506	372	382	372
q17	966	719	665	665
q18	8076	7637	7482	7482
q19	6687	1538	1365	1365
q20	666	325	317	317
q21	4910	3149	3201	3149
q22	346	281	275	275
Total cold run time: 117511 ms
Total hot run time: 40143 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4413	4272	4264	4264
q2	360	267	260	260
q3	3017	2936	2905	2905
q4	1999	1710	1761	1710
q5	5659	5549	5506	5506
q6	224	146	131	131
q7	2178	1911	1880	1880
q8	3248	3419	3432	3419
q9	8758	8852	8838	8838
q10	4073	3831	3839	3831
q11	607	498	518	498
q12	844	651	632	632
q13	17257	3203	3189	3189
q14	307	295	281	281
q15	532	484	478	478
q16	495	429	435	429
q17	1831	1554	1511	1511
q18	8210	7963	7965	7963
q19	1718	1429	1492	1429
q20	2974	1869	1879	1869
q21	10124	4735	4745	4735
q22	590	541	504	504
Total cold run time: 79418 ms
Total hot run time: 56262 ms

@zzzxl1993
Copy link
Copy Markdown
Contributor Author

zzzxl1993 commented Jul 16, 2024

run buildall

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Jul 16, 2024

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link
Copy Markdown

doris-robot commented Jul 16, 2024

TPC-H: Total hot run time: 39494 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a60f2ee639518bd408b2de49d9feb0d019af3429, data reload: false

------ Round 1 ----------------------------------
q1	17757	5529	4351	4351
q2	2399	188	183	183
q3	10478	1177	1066	1066
q4	10192	838	719	719
q5	7517	2643	2590	2590
q6	219	134	133	133
q7	956	597	597	597
q8	9211	2077	2061	2061
q9	8752	6535	6539	6535
q10	8728	3786	3731	3731
q11	464	236	238	236
q12	392	226	222	222
q13	17768	2963	2966	2963
q14	289	227	246	227
q15	523	487	486	486
q16	493	407	379	379
q17	971	723	618	618
q18	8098	7447	7371	7371
q19	3843	1406	1436	1406
q20	705	323	325	323
q21	4945	3020	3268	3020
q22	344	282	277	277
Total cold run time: 115044 ms
Total hot run time: 39494 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4375	4212	4260	4212
q2	368	276	257	257
q3	3004	2799	2761	2761
q4	1889	1555	1611	1555
q5	5286	5294	5274	5274
q6	227	127	130	127
q7	2094	1764	1676	1676
q8	3154	3354	3267	3267
q9	8345	8337	8314	8314
q10	3909	3676	3681	3676
q11	590	488	492	488
q12	759	580	595	580
q13	16626	2922	2954	2922
q14	304	278	285	278
q15	509	485	470	470
q16	462	426	423	423
q17	1744	1464	1445	1445
q18	7608	7622	7367	7367
q19	2164	1590	1608	1590
q20	1973	1795	1774	1774
q21	4805	4691	4601	4601
q22	598	500	498	498
Total cold run time: 70793 ms
Total hot run time: 53555 ms

@doris-robot
Copy link
Copy Markdown

doris-robot commented Jul 16, 2024

TPC-DS: Total hot run time: 172426 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a60f2ee639518bd408b2de49d9feb0d019af3429, data reload: false

query1	911	378	365	365
query2	6452	1891	1704	1704
query3	6669	208	220	208
query4	27598	17207	17233	17207
query5	4190	476	491	476
query6	266	179	165	165
query7	4586	288	280	280
query8	245	207	210	207
query9	8594	2406	2382	2382
query10	460	296	273	273
query11	12097	10041	9942	9942
query12	133	85	84	84
query13	1643	367	368	367
query14	10064	7523	7485	7485
query15	221	168	172	168
query16	7845	326	317	317
query17	1779	572	542	542
query18	1981	305	283	283
query19	200	153	155	153
query20	94	79	90	79
query21	211	129	131	129
query22	4317	4068	4091	4068
query23	33644	33120	32777	32777
query24	12184	2897	2871	2871
query25	693	392	391	391
query26	1865	149	151	149
query27	2973	269	277	269
query28	7748	1981	1974	1974
query29	1223	640	618	618
query30	297	151	149	149
query31	964	749	764	749
query32	102	56	57	56
query33	791	321	301	301
query34	975	485	491	485
query35	673	580	585	580
query36	1102	976	942	942
query37	291	84	85	84
query38	2915	2765	2741	2741
query39	852	840	813	813
query40	294	124	123	123
query41	50	46	47	46
query42	126	96	101	96
query43	493	467	460	460
query44	1244	738	725	725
query45	195	170	159	159
query46	1092	748	688	688
query47	1847	1799	1821	1799
query48	379	291	287	287
query49	1209	446	416	416
query50	770	390	388	388
query51	6868	6857	6774	6774
query52	112	95	96	95
query53	368	284	290	284
query54	980	541	433	433
query55	74	74	76	74
query56	276	283	268	268
query57	1159	1050	1040	1040
query58	248	237	242	237
query59	2752	2753	2613	2613
query60	315	294	295	294
query61	93	91	94	91
query62	830	647	661	647
query63	324	285	291	285
query64	10422	2216	7418	2216
query65	3147	3110	3083	3083
query66	1369	358	342	342
query67	15621	14954	15091	14954
query68	6459	555	563	555
query69	726	482	350	350
query70	1213	1146	1092	1092
query71	534	287	272	272
query72	8876	5499	5813	5499
query73	767	325	326	325
query74	6176	5697	5671	5671
query75	5063	2696	2721	2696
query76	4798	963	880	880
query77	762	291	298	291
query78	9588	9742	8931	8931
query79	8238	514	516	514
query80	1006	517	479	479
query81	594	222	223	222
query82	756	133	139	133
query83	339	172	170	170
query84	273	86	88	86
query85	1470	317	288	288
query86	455	286	337	286
query87	3331	3170	3081	3081
query88	4253	2368	2385	2368
query89	525	408	369	369
query90	2016	199	199	199
query91	134	101	103	101
query92	60	50	50	50
query93	6624	506	498	498
query94	1300	211	211	211
query95	408	322	319	319
query96	623	276	269	269
query97	3226	2993	3060	2993
query98	215	199	195	195
query99	1553	1300	1283	1283
Total cold run time: 304866 ms
Total hot run time: 172426 ms

@doris-robot
Copy link
Copy Markdown

doris-robot commented Jul 16, 2024

ClickBench: Total hot run time: 30.71 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit a60f2ee639518bd408b2de49d9feb0d019af3429, data reload: false

query1	0.04	0.04	0.03
query2	0.08	0.04	0.04
query3	0.22	0.05	0.06
query4	1.68	0.08	0.08
query5	0.48	0.49	0.50
query6	1.13	0.73	0.72
query7	0.02	0.02	0.02
query8	0.05	0.04	0.04
query9	0.54	0.49	0.50
query10	0.55	0.54	0.53
query11	0.15	0.12	0.11
query12	0.15	0.12	0.12
query13	0.60	0.57	0.58
query14	0.76	0.81	0.77
query15	0.84	0.82	0.81
query16	0.37	0.38	0.37
query17	1.04	1.01	1.04
query18	0.24	0.23	0.22
query19	1.92	1.72	1.68
query20	0.01	0.01	0.01
query21	15.41	0.73	0.64
query22	4.10	7.60	1.96
query23	18.32	1.41	1.28
query24	2.08	0.24	0.22
query25	0.15	0.09	0.09
query26	0.30	0.20	0.21
query27	0.45	0.23	0.23
query28	13.27	1.02	0.99
query29	12.67	3.32	3.27
query30	0.25	0.06	0.06
query31	2.86	0.40	0.40
query32	3.27	0.48	0.47
query33	2.88	2.92	2.90
query34	17.08	4.33	4.38
query35	4.44	4.47	4.46
query36	0.65	0.47	0.48
query37	0.19	0.16	0.15
query38	0.16	0.15	0.14
query39	0.05	0.04	0.03
query40	0.15	0.12	0.13
query41	0.09	0.05	0.05
query42	0.06	0.05	0.05
query43	0.04	0.04	0.05
Total cold run time: 109.79 s
Total hot run time: 30.71 s

airborne12
airborne12 approved these changes Jul 17, 2024
View reviewed changes
Copy link
Copy Markdown
Member

@airborne12 airborne12 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jul 17, 2024
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Jul 17, 2024

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Jul 17, 2024

PR approved by anyone and no changes requested.

@airborne12 airborne12 merged commit 4000dec into apache:master Jul 17, 2024
zzzxl1993 added a commit to zzzxl1993/doris that referenced this pull request Jul 21, 2024
@zzzxl1993
[fix](inverted index) Incorrect handling of NULL literal (apache#37921)
d1ea692 
## Proposed changes

1. Index does not handle NULL literal for now
@zzzxl1993 zzzxl1993 mentioned this pull request Jul 21, 2024
Merged
yiguolei pushed a commit that referenced this pull request Jul 25, 2024
@zzzxl1993
[fix](inverted index) fixed in_list condition not indexed on pipelinex (
e2bb86e 
#38178)

## Proposed changes

#36565
#37842
#37921
#37386

<!--Describe your changes.-->
zzzxl1993 added a commit to zzzxl1993/doris that referenced this pull request Aug 1, 2024
@zzzxl1993
[fix](inverted index) Incorrect handling of NULL literal (apache#37921)
9cd92de 
1. Index does not handle NULL literal for now
@zzzxl1993 zzzxl1993 mentioned this pull request Aug 1, 2024
Merged
zzzxl1993 added a commit to zzzxl1993/doris that referenced this pull request Aug 1, 2024
@zzzxl1993
[fix](inverted index) Incorrect handling of NULL literal (apache#37921)
e34d238 
1. Index does not handle NULL literal for now
xiaokang pushed a commit that referenced this pull request Aug 1, 2024
@zzzxl1993
[fix](inverted index) Incorrect handling of NULL literal #37921 (#38727)
7288154
zzzxl1993 added a commit to zzzxl1993/doris that referenced this pull request Aug 1, 2024
@zzzxl1993
[fix](inverted index) Incorrect handling of NULL literal (apache#37921)
05f8bb3 
1. Index does not handle NULL literal for now
This was referenced Aug 1, 2024
Merged
Merged
xiaokang pushed a commit that referenced this pull request Aug 1, 2024
@zzzxl1993
[fix](inverted index) Incorrect handling of NULL literal #37921 #37842 (
01921de 
#38741)
xiaokang pushed a commit that referenced this pull request Aug 2, 2024
@zzzxl1993
[fix](inverted index) in_list support inverted index #37921 #37842 (#…
373aa35 
…38738)
mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024
@zzzxl1993 @Baymine
[fix](inverted index) in_list support inverted index apache#37921 apa…
471af2a 
…che#37842 (apache#38738)
@gavinchou gavinchou mentioned this pull request Aug 19, 2024
Closed
mongo360 pushed a commit to mongo360/doris that referenced this pull request Dec 11, 2024
@zzzxl1993
[fix](inverted index) Incorrect handling of NULL literal apache#37921 (…
92d8b23 
…apache#38727)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@airborne12 airborne12 airborne12 approved these changes

@qidaye qidaye qidaye approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/2.0.14-merged dev/2.1.6-merged dev/3.0.1-merged p0_w reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

7 participants

@zzzxl1993 @doris-robot @airborne12 @qidaye @xiaokang @yiguolei @dataroaring