Skip to content

[fix](segcompaction) fix convert delete bitmap core#38800

Merged
dataroaring merged 1 commit intoapache:masterfrom
zhannngchen:fix-segcompaction-delete-bitmap-core
Aug 3, 2024
Merged

[fix](segcompaction) fix convert delete bitmap core#38800
dataroaring merged 1 commit intoapache:masterfrom
zhannngchen:fix-segcompaction-delete-bitmap-core

Conversation

@zhannngchen
Copy link
Copy Markdown
Contributor

@zhannngchen zhannngchen commented Aug 2, 2024

Proposed changes

Issue Number: close #xxx

introduced by #38369

@doris-robot
Copy link
Copy Markdown

doris-robot commented Aug 2, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@github-actions github-actions bot added the doing label Aug 2, 2024
@zhannngchen zhannngchen force-pushed the fix-segcompaction-delete-bitmap-core branch from a7b449b to 078982b Compare August 2, 2024 11:51
@zhannngchen
Copy link
Copy Markdown
Contributor Author

zhannngchen commented Aug 2, 2024

run buildall

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Aug 2, 2024

clang-tidy review says "All clean, LGTM! 👍"

1 similar comment
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Aug 2, 2024

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link
Copy Markdown

doris-robot commented Aug 2, 2024

TPC-H: Total hot run time: 41123 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 078982b7bf49c19a1bc10ea3416550595201480e, data reload: false

------ Round 1 ----------------------------------
q1	17607	4125	4314	4125
q2	2021	196	192	192
q3	10478	1239	1358	1239
q4	10171	853	851	851
q5	7641	2857	2944	2857
q6	218	134	135	134
q7	1052	607	599	599
q8	9439	1781	1938	1781
q9	8487	6577	6589	6577
q10	8730	3822	3798	3798
q11	425	244	244	244
q12	437	222	220	220
q13	17768	2911	2934	2911
q14	267	236	241	236
q15	527	486	495	486
q16	536	384	384	384
q17	946	918	905	905
q18	8042	7220	7167	7167
q19	2621	1134	1201	1134
q20	557	332	323	323
q21	5226	4702	4682	4682
q22	357	279	278	278
Total cold run time: 113553 ms
Total hot run time: 41123 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4136	3980	4012	3980
q2	347	227	218	218
q3	2988	2976	3133	2976
q4	1995	1989	1963	1963
q5	5556	5452	5402	5402
q6	217	129	129	129
q7	2131	1797	1846	1797
q8	3281	3346	3338	3338
q9	8638	8670	8794	8670
q10	3946	4090	3899	3899
q11	542	462	456	456
q12	781	590	589	589
q13	16344	3102	3091	3091
q14	307	275	274	274
q15	531	489	493	489
q16	469	435	416	416
q17	1748	1704	1709	1704
q18	8202	7715	7768	7715
q19	1717	1699	1687	1687
q20	2051	1849	1831	1831
q21	5699	5514	5307	5307
q22	527	471	473	471
Total cold run time: 72153 ms
Total hot run time: 56402 ms

@doris-robot
Copy link
Copy Markdown

doris-robot commented Aug 2, 2024

TPC-DS: Total hot run time: 169112 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 078982b7bf49c19a1bc10ea3416550595201480e, data reload: false

query1	906	373	368	368
query2	6476	1649	1647	1647
query3	6661	212	221	212
query4	19872	17573	17429	17429
query5	3622	507	511	507
query6	267	174	168	168
query7	4604	310	294	294
query8	263	203	207	203
query9	8512	2327	2299	2299
query10	431	307	261	261
query11	10566	10155	9964	9964
query12	125	89	90	89
query13	1645	383	384	383
query14	10380	6707	6840	6707
query15	201	158	166	158
query16	6790	434	398	398
query17	946	550	559	550
query18	1744	276	273	273
query19	191	146	139	139
query20	94	92	84	84
query21	204	98	98	98
query22	4151	4127	4235	4127
query23	33785	33545	33280	33280
query24	10086	3080	3128	3080
query25	702	401	399	399
query26	1681	150	149	149
query27	2883	279	281	279
query28	7440	1990	1978	1978
query29	1194	446	440	440
query30	237	161	154	154
query31	947	780	769	769
query32	106	60	56	56
query33	668	313	318	313
query34	942	498	510	498
query35	857	757	762	757
query36	1066	891	878	878
query37	215	84	87	84
query38	3092	2905	2877	2877
query39	952	804	821	804
query40	265	111	116	111
query41	46	44	71	44
query42	118	115	99	99
query43	466	399	413	399
query44	1176	735	743	735
query45	210	180	182	180
query46	1093	840	793	793
query47	1789	1688	1687	1687
query48	369	291	291	291
query49	948	421	427	421
query50	888	436	432	432
query51	6843	6772	6607	6607
query52	100	94	84	84
query53	249	183	178	178
query54	613	457	478	457
query55	74	74	74	74
query56	302	262	272	262
query57	1165	1037	1027	1027
query58	273	271	286	271
query59	2590	2366	2320	2320
query60	289	279	274	274
query61	98	91	108	91
query62	899	658	672	658
query63	212	186	181	181
query64	5751	1871	1872	1871
query65	3159	3081	3093	3081
query66	1295	346	332	332
query67	15248	14831	14814	14814
query68	4321	561	575	561
query69	442	304	302	302
query70	1118	1056	1057	1056
query71	408	279	282	279
query72	7082	2720	2467	2467
query73	762	330	338	330
query74	6084	5639	5652	5639
query75	3369	2746	2745	2745
query76	2716	1205	1276	1205
query77	424	317	312	312
query78	9461	8915	8882	8882
query79	1529	530	532	530
query80	1315	511	497	497
query81	561	227	226	226
query82	1049	132	134	132
query83	253	172	171	171
query84	273	80	77	77
query85	1275	315	295	295
query86	415	301	302	301
query87	3254	3114	3106	3106
query88	3006	2534	2474	2474
query89	399	299	291	291
query90	1764	196	195	195
query91	125	100	105	100
query92	58	51	51	51
query93	1419	617	610	610
query94	784	303	293	293
query95	388	265	268	265
query96	608	283	282	282
query97	3218	3035	3098	3035
query98	217	192	202	192
query99	1667	1309	1284	1284
Total cold run time: 261796 ms
Total hot run time: 169112 ms

@doris-robot
Copy link
Copy Markdown

doris-robot commented Aug 2, 2024

ClickBench: Total hot run time: 29.7 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 078982b7bf49c19a1bc10ea3416550595201480e, data reload: false

query1	0.05	0.04	0.03
query2	0.08	0.04	0.04
query3	0.23	0.06	0.05
query4	1.64	0.08	0.08
query5	0.49	0.49	0.47
query6	1.15	0.73	0.72
query7	0.02	0.02	0.01
query8	0.05	0.04	0.04
query9	0.57	0.50	0.51
query10	0.60	0.57	0.55
query11	0.16	0.12	0.12
query12	0.16	0.13	0.13
query13	0.61	0.62	0.59
query14	0.77	0.79	0.80
query15	0.89	0.88	0.84
query16	0.35	0.34	0.35
query17	1.00	1.00	0.96
query18	0.21	0.21	0.20
query19	1.83	1.71	1.73
query20	0.01	0.01	0.01
query21	15.39	0.75	0.66
query22	4.09	8.30	1.17
query23	17.64	1.33	1.29
query24	2.27	0.23	0.22
query25	0.19	0.08	0.08
query26	0.32	0.22	0.21
query27	0.47	0.23	0.22
query28	13.16	1.00	0.97
query29	12.53	3.31	3.24
query30	0.26	0.06	0.06
query31	2.85	0.41	0.40
query32	3.22	0.49	0.48
query33	2.97	2.92	2.95
query34	15.46	4.26	4.23
query35	4.31	4.30	4.28
query36	0.69	0.48	0.48
query37	0.19	0.16	0.17
query38	0.16	0.16	0.16
query39	0.04	0.03	0.04
query40	0.16	0.14	0.13
query41	0.10	0.05	0.05
query42	0.06	0.04	0.05
query43	0.05	0.04	0.04
Total cold run time: 107.45 s
Total hot run time: 29.7 s

dataroaring
dataroaring approved these changes Aug 2, 2024
View reviewed changes
Copy link
Copy Markdown
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Aug 2, 2024
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Aug 2, 2024

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Aug 2, 2024

PR approved by anyone and no changes requested.

@dataroaring dataroaring merged commit d664a0a into apache:master Aug 3, 2024
dataroaring pushed a commit that referenced this pull request Aug 11, 2024
@zhannngchen @dataroaring
[fix](segcompaction) fix convert delete bitmap core (#38800)
3bc2e7e 
## Proposed changes

Issue Number: close #xxx

introduced by #38369
dataroaring pushed a commit that referenced this pull request Aug 16, 2024
@zhannngchen @dataroaring
[fix](segcompaction) fix convert delete bitmap core (#38800)
5911aec 
## Proposed changes

Issue Number: close #xxx

introduced by #38369
zhannngchen added a commit to zhannngchen/incubator-doris that referenced this pull request Aug 21, 2024
@zhannngchen
[fix](segcompaction) fix convert delete bitmap core (apache#38800)
e2d072b 
## Proposed changes

Issue Number: close #xxx

introduced by apache#38369
@zhannngchen zhannngchen mentioned this pull request Aug 21, 2024
Merged
yiguolei pushed a commit that referenced this pull request Aug 21, 2024
@zhannngchen
[fix](merge-on-write) segcompaction should process delete bitmap if n…
0e694f1 
…ecessary (#38369) (#39707)

## Proposed changes

Issue Number: close #xxx

cherry-pick #38369 and #38800
zhannngchen added a commit to zhannngchen/incubator-doris that referenced this pull request Aug 22, 2024
@zhannngchen
[fix](merge-on-write) segcompaction should process delete bitmap if n…
28143ff 
…ecessary (apache#38369) (apache#39707)

Issue Number: close #xxx

cherry-pick apache#38369 and apache#38800
@zhannngchen zhannngchen mentioned this pull request Aug 22, 2024
Merged
zhannngchen added a commit that referenced this pull request Aug 22, 2024
@zhannngchen
[fix](merge-on-write) segcompaction should process delete bitmap if n…
c313446 
…ecessary (#38369) (#39749)

cherry-pick #38369 and #38800
mongo360 pushed a commit to mongo360/doris that referenced this pull request Dec 11, 2024
@zhannngchen @Baymine
[fix](merge-on-write) segcompaction should process delete bitmap if n…
e2660a0 
…ecessary (apache#38369) (apache#39749)

cherry-pick apache#38369 and apache#38800
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@bobhan1 bobhan1 bobhan1 approved these changes

@dataroaring dataroaring dataroaring approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/2.0.15-merged dev/2.1.6-merged dev/3.0.2-merged doing reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@zhannngchen @doris-robot @bobhan1 @dataroaring @yiguolei