[fix] (inverted index) Fix match function without inverted index by csun5285 · Pull Request #38989 · apache/doris

csun5285 · 2024-08-07T01:54:23Z

Proposed changes

BUG

The properties of the inverted index were not passed from the FE to the BE, resulting in inconsistencies between non-indexed and indexed match queries.

FIX

Fix match function without inverted index

Issue Number: close #xxx

doris-robot · 2024-08-07T01:54:27Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

csun5285 · 2024-08-07T01:55:29Z

run buildall

github-actions · 2024-08-07T02:01:42Z

clang-tidy review says "All clean, LGTM! 👍"

github-actions · 2024-08-07T02:01:49Z

clang-tidy review says "All clean, LGTM! 👍"

doris-robot · 2024-08-07T03:44:24Z

TPC-H: Total hot run time: 41918 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 18a280f5d4a61117e434f96efec0deea13f9a0c3, data reload: false

------ Round 1 ----------------------------------
q1	17791	4181	4081	4081
q2	2024	203	198	198
q3	10560	1334	1353	1334
q4	10260	805	911	805
q5	7644	3047	2991	2991
q6	225	137	142	137
q7	1061	626	619	619
q8	9432	1936	1973	1936
q9	8524	6613	6610	6610
q10	8707	3871	3865	3865
q11	439	258	250	250
q12	420	240	231	231
q13	17763	2979	2948	2948
q14	271	241	241	241
q15	526	498	496	496
q16	531	391	390	390
q17	991	960	941	941
q18	8035	7332	7256	7256
q19	1807	1242	1219	1219
q20	567	325	345	325
q21	5426	4764	4788	4764
q22	356	285	281	281
Total cold run time: 113360 ms
Total hot run time: 41918 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4140	3998	4069	3998
q2	338	233	218	218
q3	3000	3049	3174	3049
q4	2017	2010	1949	1949
q5	5673	5474	5462	5462
q6	219	132	170	132
q7	2140	1806	1827	1806
q8	3311	3417	3358	3358
q9	8677	8717	8750	8717
q10	3956	4003	3982	3982
q11	555	475	461	461
q12	772	590	581	581
q13	16388	3149	3135	3135
q14	286	265	266	265
q15	521	489	484	484
q16	471	428	406	406
q17	1777	1756	1749	1749
q18	8151	7721	7611	7611
q19	1755	1765	1745	1745
q20	2138	1835	1853	1835
q21	5803	5593	5178	5178
q22	546	474	470	470
Total cold run time: 72634 ms
Total hot run time: 56591 ms

doris-robot · 2024-08-07T03:54:58Z

TPC-DS: Total hot run time: 168551 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 18a280f5d4a61117e434f96efec0deea13f9a0c3, data reload: false

query1	924	378	370	370
query2	6455	1730	1774	1730
query3	6665	211	220	211
query4	20254	17453	17097	17097
query5	3643	516	502	502
query6	283	186	170	170
query7	4593	294	292	292
query8	263	197	192	192
query9	8529	2348	2341	2341
query10	426	259	257	257
query11	10424	9931	9992	9931
query12	119	95	85	85
query13	1610	371	372	371
query14	9407	6875	7343	6875
query15	217	157	167	157
query16	6897	434	457	434
query17	942	584	540	540
query18	1809	277	278	277
query19	183	154	152	152
query20	94	84	85	84
query21	222	98	94	94
query22	4245	4149	3946	3946
query23	33652	33587	33373	33373
query24	9367	3069	3072	3069
query25	679	445	398	398
query26	1214	152	152	152
query27	2714	278	292	278
query28	7451	2020	2023	2020
query29	950	445	419	419
query30	236	156	153	153
query31	951	783	766	766
query32	112	56	58	56
query33	693	306	323	306
query34	910	482	502	482
query35	894	762	751	751
query36	1051	928	909	909
query37	188	84	83	83
query38	2913	2862	2839	2839
query39	918	811	807	807
query40	206	117	116	116
query41	49	45	48	45
query42	124	103	103	103
query43	494	442	429	429
query44	1191	736	743	736
query45	215	172	174	172
query46	1101	819	772	772
query47	1815	1731	1711	1711
query48	359	292	287	287
query49	867	415	417	415
query50	881	429	421	421
query51	6795	6678	6670	6670
query52	105	88	87	87
query53	260	181	183	181
query54	613	441	449	441
query55	76	73	75	73
query56	265	249	252	249
query57	1147	1029	1042	1029
query58	256	260	266	260
query59	2599	2314	2481	2314
query60	298	261	279	261
query61	97	96	94	94
query62	909	658	646	646
query63	216	185	181	181
query64	4736	1913	1845	1845
query65	3202	3110	3072	3072
query66	968	323	325	323
query67	15220	14702	14807	14702
query68	4338	561	584	561
query69	452	308	300	300
query70	1122	1091	1074	1074
query71	378	276	275	275
query72	7120	2652	2449	2449
query73	757	325	321	321
query74	6008	5679	5668	5668
query75	3378	2689	2724	2689
query76	2067	1214	1236	1214
query77	473	302	315	302
query78	9459	8833	8908	8833
query79	1878	531	536	531
query80	1186	521	496	496
query81	577	227	227	227
query82	1060	131	132	131
query83	246	169	166	166
query84	263	78	80	78
query85	1099	320	304	304
query86	394	280	303	280
query87	3229	3056	3067	3056
query88	2892	2386	2395	2386
query89	389	294	299	294
query90	1682	188	187	187
query91	126	114	102	102
query92	59	48	49	48
query93	1525	640	633	633
query94	759	299	293	293
query95	438	257	260	257
query96	593	276	277	276
query97	3241	3062	3076	3062
query98	215	198	202	198
query99	1615	1256	1271	1256
Total cold run time: 257179 ms
Total hot run time: 168551 ms

doris-robot · 2024-08-07T04:00:29Z

ClickBench: Total hot run time: 29.92 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 18a280f5d4a61117e434f96efec0deea13f9a0c3, data reload: false

query1	0.05	0.04	0.04
query2	0.07	0.04	0.04
query3	0.23	0.05	0.05
query4	1.68	0.07	0.07
query5	0.48	0.48	0.48
query6	1.17	0.72	0.71
query7	0.02	0.01	0.02
query8	0.06	0.04	0.04
query9	0.56	0.51	0.52
query10	0.56	0.55	0.56
query11	0.15	0.11	0.11
query12	0.14	0.12	0.13
query13	0.62	0.60	0.60
query14	0.77	0.80	0.80
query15	0.90	0.86	0.86
query16	0.36	0.35	0.34
query17	0.97	0.99	1.00
query18	0.22	0.22	0.21
query19	1.82	1.75	1.76
query20	0.01	0.01	0.03
query21	15.40	0.74	0.65
query22	4.79	7.00	1.17
query23	18.02	1.34	1.34
query24	2.24	0.22	0.22
query25	0.19	0.08	0.08
query26	0.31	0.22	0.22
query27	0.46	0.24	0.23
query28	13.16	0.99	0.97
query29	12.53	3.36	3.30
query30	0.25	0.06	0.06
query31	2.88	0.40	0.40
query32	3.25	0.48	0.48
query33	2.96	2.98	2.96
query34	15.44	4.22	4.23
query35	4.28	4.32	4.26
query36	0.69	0.48	0.47
query37	0.19	0.17	0.17
query38	0.16	0.15	0.15
query39	0.04	0.03	0.04
query40	0.16	0.13	0.13
query41	0.09	0.04	0.05
query42	0.05	0.05	0.05
query43	0.05	0.05	0.03
Total cold run time: 108.43 s
Total hot run time: 29.92 s

csun5285 · 2024-08-07T09:09:12Z

run buildall

csun5285 · 2024-08-07T09:14:06Z

run buildall

github-actions · 2024-08-07T09:15:25Z

clang-tidy review says "All clean, LGTM! 👍"

github-actions · 2024-08-07T09:20:32Z

clang-tidy review says "All clean, LGTM! 👍"

doris-robot · 2024-08-07T10:28:45Z

TPC-H: Total hot run time: 42198 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit cf5fd4c1262f35856d8cbf2f8afd462834ce6d0f, data reload: false

------ Round 1 ----------------------------------
q1	18063	4265	4178	4178
q2	2488	208	206	206
q3	11311	1342	1398	1342
q4	11694	862	1019	862
q5	8610	3007	3005	3005
q6	223	140	143	140
q7	1075	644	640	640
q8	9442	1887	1952	1887
q9	8551	6595	6625	6595
q10	8729	3880	3849	3849
q11	431	254	260	254
q12	420	240	236	236
q13	17761	2968	2970	2968
q14	284	254	247	247
q15	541	489	492	489
q16	494	401	387	387
q17	982	932	954	932
q18	8112	7409	7268	7268
q19	1438	1234	1230	1230
q20	565	330	353	330
q21	5413	4868	4882	4868
q22	368	290	285	285
Total cold run time: 116995 ms
Total hot run time: 42198 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4060	4017	4029	4017
q2	329	237	223	223
q3	3019	3015	3015	3015
q4	1994	2007	1976	1976
q5	5315	5299	5264	5264
q6	219	132	137	132
q7	2072	1662	1724	1662
q8	3212	3286	3287	3286
q9	8337	8305	8327	8305
q10	3803	3854	3887	3854
q11	558	458	459	458
q12	743	543	544	543
q13	16415	2945	2986	2945
q14	284	253	261	253
q15	521	481	483	481
q16	450	412	410	410
q17	1760	1733	1730	1730
q18	7887	7608	7192	7192
q19	1675	1683	1712	1683
q20	1983	1756	1759	1756
q21	5546	5395	5386	5386
q22	564	468	443	443
Total cold run time: 70746 ms
Total hot run time: 55014 ms

doris-robot · 2024-08-07T10:39:24Z

TPC-DS: Total hot run time: 168636 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit cf5fd4c1262f35856d8cbf2f8afd462834ce6d0f, data reload: false

query1	908	372	369	369
query2	6494	1679	1666	1666
query3	6672	213	219	213
query4	19598	17656	17286	17286
query5	4282	521	515	515
query6	292	175	171	171
query7	4616	308	302	302
query8	274	204	229	204
query9	8516	2377	2358	2358
query10	456	278	261	261
query11	10465	10079	9932	9932
query12	136	86	85	85
query13	1637	380	375	375
query14	9388	7608	7100	7100
query15	199	167	161	161
query16	7099	481	503	481
query17	944	576	546	546
query18	1909	283	275	275
query19	188	142	144	142
query20	91	84	86	84
query21	205	105	100	100
query22	4330	4110	4085	4085
query23	33501	33012	32629	32629
query24	10395	3048	3054	3048
query25	702	397	388	388
query26	1747	151	153	151
query27	2899	274	280	274
query28	6920	1967	1957	1957
query29	1309	421	413	413
query30	293	150	152	150
query31	919	745	754	745
query32	106	53	55	53
query33	702	326	323	323
query34	887	481	484	481
query35	847	729	744	729
query36	984	883	860	860
query37	302	79	78	78
query38	2879	2787	2763	2763
query39	878	834	817	817
query40	285	147	112	112
query41	47	43	43	43
query42	122	99	99	99
query43	462	424	423	423
query44	1186	734	724	724
query45	204	179	176	176
query46	1097	812	796	796
query47	1811	1698	1722	1698
query48	370	294	293	293
query49	1201	415	422	415
query50	904	436	432	432
query51	6750	6727	6720	6720
query52	103	90	90	90
query53	250	184	182	182
query54	662	452	452	452
query55	76	75	76	75
query56	283	260	271	260
query57	1140	1082	1065	1065
query58	281	277	266	266
query59	2556	2399	2284	2284
query60	340	273	281	273
query61	102	97	109	97
query62	885	661	660	660
query63	219	184	179	179
query64	5901	1925	1880	1880
query65	3162	3114	3131	3114
query66	1453	384	361	361
query67	15468	14736	14788	14736
query68	4365	563	567	563
query69	592	364	301	301
query70	1098	1082	1068	1068
query71	399	276	277	276
query72	7051	2742	2509	2509
query73	773	325	335	325
query74	6026	5692	5618	5618
query75	3346	2728	2750	2728
query76	2318	1223	1287	1223
query77	477	321	319	319
query78	9380	8865	8868	8865
query79	1463	541	540	540
query80	1217	509	536	509
query81	580	230	224	224
query82	998	132	132	132
query83	244	170	172	170
query84	274	80	81	80
query85	1304	410	310	310
query86	403	299	292	292
query87	3270	3082	3163	3082
query88	2884	2428	2393	2393
query89	383	291	287	287
query90	1764	190	190	190
query91	128	100	105	100
query92	61	50	52	50
query93	1367	632	639	632
query94	875	300	288	288
query95	384	273	272	272
query96	596	273	282	273
query97	3224	3056	3078	3056
query98	214	194	196	194
query99	1643	1309	1292	1292
Total cold run time: 261390 ms
Total hot run time: 168636 ms

doris-robot · 2024-08-07T10:44:53Z

ClickBench: Total hot run time: 30.05 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit cf5fd4c1262f35856d8cbf2f8afd462834ce6d0f, data reload: false

query1	0.05	0.05	0.04
query2	0.07	0.04	0.04
query3	0.23	0.05	0.05
query4	1.67	0.06	0.08
query5	0.48	0.48	0.49
query6	1.13	0.71	0.71
query7	0.02	0.01	0.02
query8	0.04	0.04	0.04
query9	0.59	0.52	0.50
query10	0.58	0.57	0.56
query11	0.15	0.11	0.11
query12	0.15	0.13	0.14
query13	0.62	0.61	0.61
query14	0.77	0.80	0.79
query15	0.91	0.86	0.84
query16	0.36	0.36	0.36
query17	1.00	1.02	1.01
query18	0.23	0.22	0.24
query19	1.83	1.76	1.80
query20	0.01	0.01	0.01
query21	15.39	0.75	0.65
query22	4.11	6.95	1.41
query23	18.16	1.28	1.26
query24	2.28	0.22	0.21
query25	0.18	0.08	0.07
query26	0.33	0.22	0.21
query27	0.47	0.23	0.24
query28	13.18	0.99	0.98
query29	12.57	3.29	3.24
query30	0.26	0.06	0.06
query31	2.85	0.41	0.41
query32	3.23	0.50	0.48
query33	2.96	2.95	2.98
query34	15.44	4.30	4.24
query35	4.28	4.24	4.29
query36	0.68	0.47	0.48
query37	0.19	0.17	0.17
query38	0.16	0.15	0.14
query39	0.04	0.04	0.03
query40	0.16	0.13	0.14
query41	0.10	0.05	0.05
query42	0.06	0.05	0.06
query43	0.05	0.04	0.05
Total cold run time: 108.02 s
Total hot run time: 30.05 s

zzzxl1993

LGTM

github-actions · 2024-08-07T11:25:14Z

PR approved by anyone and no changes requested.

airborne12

LGTM

github-actions · 2024-08-08T02:50:36Z

PR approved by at least one committer and no changes requested.

) ## Proposed changes ### BUG The properties of the inverted index were not passed from the FE to the BE, resulting in inconsistencies between non-indexed and indexed match queries. ### FIX Fix match function without inverted index

) (#39220) ## Proposed changes pick from #38989

) (#39221) ## Proposed changes pick from #38989

…che#38989) ## Proposed changes ### BUG The properties of the inverted index were not passed from the FE to the BE, resulting in inconsistencies between non-indexed and indexed match queries. ### FIX Fix match function without inverted index

) ## Proposed changes ### BUG The properties of the inverted index were not passed from the FE to the BE, resulting in inconsistencies between non-indexed and indexed match queries. ### FIX Fix match function without inverted index

…che#38989) (apache#39221) ## Proposed changes pick from apache#38989

github-actions Bot added doing meta-change labels Aug 7, 2024

csun5285 added 5 commits August 7, 2024 17:11

fix

edfb621

fix

dc509b8

fix

2e41d28

remove unused code

11f7b1c

fix p0

cf5fd4c

csun5285 force-pushed the fix_match_phrase branch from 92e7cc6 to cf5fd4c Compare August 7, 2024 09:13

zzzxl1993 approved these changes Aug 7, 2024

View reviewed changes

github-actions Bot added the reviewed label Aug 7, 2024

airborne12 approved these changes Aug 8, 2024

View reviewed changes

github-actions Bot added the approved Indicates a PR has been approved by one committer. label Aug 8, 2024

airborne12 added dev/2.0.x dev/2.1.x labels Aug 8, 2024

airborne12 added the dev/3.0.x label Aug 8, 2024

airborne12 merged commit 3f74137 into apache:master Aug 8, 2024

dataroaring added dev/3.0.2-merged and removed dev/3.0.x labels Aug 12, 2024

This was referenced Aug 12, 2024

[fix] (inverted index) Fix match function without inverted index (#38989) #39220

Merged

[fix] (inverted index) Fix match function without inverted index (#38989) #39221

Merged

airborne12 pushed a commit that referenced this pull request Aug 13, 2024

[fix] (inverted index) Fix match function without inverted index (#38989

60eeec3

) (#39220) ## Proposed changes pick from #38989

airborne12 added dev/2.1.6-merged and removed dev/2.1.x labels Aug 13, 2024

airborne12 pushed a commit that referenced this pull request Aug 13, 2024

[fix] (inverted index) Fix match function without inverted index (#38989

a432408

) (#39221) ## Proposed changes pick from #38989

airborne12 added dev/2.0.15-merged and removed dev/2.0.x doing labels Aug 13, 2024

GoGoWen pushed a commit to GoGoWen/incubator-doris that referenced this pull request Aug 27, 2024

[fix] (inverted index) Fix match function without inverted index (apa…

8c2df08

…che#38989) (apache#39221) ## Proposed changes pick from apache#38989

gavinchou mentioned this pull request Oct 13, 2024

Release Note 3.0.2 #41558

Closed

Conversation

csun5285 commented Aug 7, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Proposed changes

BUG

FIX

Uh oh!

doris-robot commented Aug 7, 2024

Uh oh!

csun5285 commented Aug 7, 2024

Uh oh!

github-actions Bot commented Aug 7, 2024

Uh oh!

github-actions Bot commented Aug 7, 2024

Uh oh!

doris-robot commented Aug 7, 2024

Uh oh!

doris-robot commented Aug 7, 2024

Uh oh!

doris-robot commented Aug 7, 2024

Uh oh!

csun5285 commented Aug 7, 2024

Uh oh!

csun5285 commented Aug 7, 2024

Uh oh!

github-actions Bot commented Aug 7, 2024

Uh oh!

github-actions Bot commented Aug 7, 2024

Uh oh!

doris-robot commented Aug 7, 2024

Uh oh!

doris-robot commented Aug 7, 2024

Uh oh!

doris-robot commented Aug 7, 2024

Uh oh!

zzzxl1993 left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Aug 7, 2024

Uh oh!

airborne12 left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Aug 8, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

csun5285 commented Aug 7, 2024 •

edited

Loading