Skip to content

[Improment]Limit remote scan IO#39012

Merged
yiguolei merged 1 commit intoapache:masterfrom
wangbo:0805_limit_prefetch_io
Aug 8, 2024
Merged

[Improment]Limit remote scan IO#39012
yiguolei merged 1 commit intoapache:masterfrom
wangbo:0805_limit_prefetch_io

Conversation

@wangbo
Copy link
Copy Markdown
Contributor

@wangbo wangbo commented Aug 7, 2024

Proposed changes

Issue Number: close #xxx

@doris-robot
Copy link
Copy Markdown

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@github-actions github-actions bot added the doing label Aug 7, 2024
Copy link
Copy Markdown
Contributor

@yiguolei yiguolei left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Aug 7, 2024

clang-tidy review says "All clean, LGTM! 👍"

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Aug 7, 2024
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Aug 7, 2024

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Aug 7, 2024

PR approved by anyone and no changes requested.

@yiguolei
Copy link
Copy Markdown
Contributor

yiguolei commented Aug 7, 2024

run buildall

@yiguolei yiguolei force-pushed the 0805_limit_prefetch_io branch from 68452c2 to 9c7be9e Compare August 7, 2024 08:13
@yiguolei
Copy link
Copy Markdown
Contributor

yiguolei commented Aug 7, 2024

run buildall

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Aug 7, 2024

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link
Copy Markdown

TPC-H: Total hot run time: 42053 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 9c7be9e590f04de184b950d76645c62e4d95c5a2, data reload: false

------ Round 1 ----------------------------------
q1	17626	4183	4173	4173
q2	2024	225	217	217
q3	10400	1390	1393	1390
q4	10165	788	946	788
q5	7647	2945	3001	2945
q6	224	138	143	138
q7	1090	624	622	622
q8	9450	1829	1974	1829
q9	8538	6653	6678	6653
q10	8786	3861	3876	3861
q11	436	254	257	254
q12	409	230	224	224
q13	17771	2919	2920	2919
q14	275	245	241	241
q15	528	492	501	492
q16	525	383	387	383
q17	996	950	957	950
q18	8024	7372	7374	7372
q19	1794	1233	1227	1227
q20	572	332	354	332
q21	5372	4782	4754	4754
q22	354	289	291	289
Total cold run time: 113006 ms
Total hot run time: 42053 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4131	4042	4072	4042
q2	331	228	225	225
q3	3004	3056	3143	3056
q4	2042	2082	2026	2026
q5	5635	5483	5438	5438
q6	224	134	135	134
q7	2144	1846	1797	1797
q8	3345	3426	3390	3390
q9	8694	8736	8726	8726
q10	3984	3975	4023	3975
q11	566	469	467	467
q12	787	617	599	599
q13	16432	3097	3137	3097
q14	300	284	264	264
q15	539	476	506	476
q16	467	425	426	425
q17	1805	1763	1746	1746
q18	8246	7731	7756	7731
q19	1919	1777	1752	1752
q20	2077	1839	1845	1839
q21	5743	5430	5398	5398
q22	524	469	472	469
Total cold run time: 72939 ms
Total hot run time: 57072 ms

@doris-robot
Copy link
Copy Markdown

TPC-DS: Total hot run time: 170011 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 9c7be9e590f04de184b950d76645c62e4d95c5a2, data reload: false

query1	915	378	371	371
query2	6447	1768	1759	1759
query3	6654	210	223	210
query4	20311	17518	17518	17518
query5	3635	514	517	514
query6	272	164	164	164
query7	4593	304	302	302
query8	250	197	203	197
query9	8518	2433	2415	2415
query10	452	286	275	275
query11	10522	10091	10046	10046
query12	122	91	89	89
query13	1642	385	368	368
query14	8601	7505	6810	6810
query15	201	163	175	163
query16	6984	485	424	424
query17	971	575	567	567
query18	1842	289	285	285
query19	193	148	150	148
query20	97	90	88	88
query21	206	102	101	101
query22	4273	3994	4103	3994
query23	33816	33453	33375	33375
query24	10235	3126	3127	3126
query25	719	428	420	420
query26	1701	155	161	155
query27	3058	288	283	283
query28	7444	2062	2027	2027
query29	1329	442	439	439
query30	237	154	152	152
query31	943	799	754	754
query32	106	55	58	55
query33	694	331	323	323
query34	932	513	506	506
query35	875	768	765	765
query36	1052	904	912	904
query37	300	81	88	81
query38	3014	2908	2879	2879
query39	883	838	804	804
query40	250	114	112	112
query41	46	44	44	44
query42	124	103	107	103
query43	480	405	419	405
query44	1197	731	724	724
query45	215	176	176	176
query46	1092	816	792	792
query47	1822	1731	1713	1713
query48	366	297	293	293
query49	938	427	428	427
query50	899	433	437	433
query51	6825	6702	6645	6645
query52	104	94	91	91
query53	247	181	180	180
query54	623	486	463	463
query55	80	75	76	75
query56	279	249	263	249
query57	1132	1025	1027	1025
query58	286	272	276	272
query59	2618	2400	2383	2383
query60	294	280	269	269
query61	97	95	94	94
query62	869	664	674	664
query63	222	184	182	182
query64	5684	1940	1872	1872
query65	3161	3107	3088	3088
query66	1300	332	329	329
query67	15324	14997	14699	14699
query68	4415	563	570	563
query69	514	303	336	303
query70	1122	1087	1094	1087
query71	449	285	289	285
query72	7852	2696	2509	2509
query73	761	338	339	338
query74	6043	5723	5663	5663
query75	3780	2782	2739	2739
query76	2670	1178	1239	1178
query77	670	312	317	312
query78	9521	8971	8879	8879
query79	2689	531	524	524
query80	1162	513	505	505
query81	558	222	232	222
query82	929	140	132	132
query83	321	233	177	177
query84	265	82	82	82
query85	1329	321	297	297
query86	468	280	286	280
query87	3334	3123	3118	3118
query88	3989	2529	2524	2524
query89	384	294	293	293
query90	1846	198	200	198
query91	128	102	106	102
query92	60	48	50	48
query93	2122	618	628	618
query94	890	297	285	285
query95	383	278	269	269
query96	629	286	282	282
query97	3243	3074	3083	3074
query98	236	210	197	197
query99	1635	1270	1271	1270
Total cold run time: 266010 ms
Total hot run time: 170011 ms

@doris-robot
Copy link
Copy Markdown

ClickBench: Total hot run time: 29.95 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 9c7be9e590f04de184b950d76645c62e4d95c5a2, data reload: false

query1	0.05	0.04	0.04
query2	0.08	0.04	0.04
query3	0.23	0.05	0.05
query4	1.67	0.07	0.06
query5	0.48	0.48	0.49
query6	1.15	0.71	0.72
query7	0.02	0.01	0.02
query8	0.06	0.05	0.05
query9	0.59	0.50	0.51
query10	0.57	0.57	0.58
query11	0.16	0.12	0.12
query12	0.15	0.13	0.13
query13	0.63	0.61	0.60
query14	0.77	0.81	0.78
query15	0.92	0.88	0.88
query16	0.36	0.35	0.35
query17	0.99	0.98	0.99
query18	0.22	0.21	0.22
query19	1.83	1.71	1.71
query20	0.01	0.01	0.01
query21	15.39	0.77	0.66
query22	3.83	8.24	1.25
query23	18.13	1.34	1.27
query24	2.24	0.23	0.22
query25	0.17	0.08	0.08
query26	0.33	0.22	0.21
query27	0.47	0.24	0.23
query28	13.16	1.01	0.97
query29	12.56	3.28	3.27
query30	0.25	0.06	0.06
query31	2.88	0.40	0.41
query32	3.24	0.48	0.48
query33	2.94	2.96	2.99
query34	15.50	4.27	4.24
query35	4.29	4.27	4.27
query36	0.67	0.49	0.48
query37	0.20	0.16	0.17
query38	0.17	0.15	0.15
query39	0.04	0.04	0.04
query40	0.17	0.14	0.13
query41	0.10	0.05	0.05
query42	0.05	0.05	0.05
query43	0.05	0.05	0.04
Total cold run time: 107.77 s
Total hot run time: 29.95 s

@yiguolei yiguolei merged commit d1bc525 into apache:master Aug 8, 2024
dataroaring pushed a commit that referenced this pull request Aug 11, 2024
## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->
wyxxxcat pushed a commit to wyxxxcat/doris that referenced this pull request Aug 14, 2024
## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->
dataroaring pushed a commit that referenced this pull request Aug 16, 2024
## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->
wangbo added a commit to wangbo/incubator-doris that referenced this pull request Aug 23, 2024
Issue Number: close #xxx

<!--Describe your changes.-->
wangbo added a commit to wangbo/incubator-doris that referenced this pull request Aug 23, 2024
Issue Number: close #xxx

<!--Describe your changes.-->
wangbo added a commit to wangbo/incubator-doris that referenced this pull request Aug 23, 2024
Issue Number: close #xxx

<!--Describe your changes.-->
wangbo added a commit to wangbo/incubator-doris that referenced this pull request Aug 23, 2024
Issue Number: close #xxx

<!--Describe your changes.-->
@gavinchou gavinchou mentioned this pull request Oct 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. dev/3.0.2-merged doing reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants