[test](mtmv) Add inject statistic when mv rewrite regression test to make sure rewrite result stable by seawinde · Pull Request #43785 · apache/doris

seawinde · 2024-11-12T13:09:09Z

What problem does this PR solve?

The result of successful rewriting by the cbo optimizer depends on the statistics.
The priority of the optimizer consumption statistics in descending order is

the injected statistics
the statistics reported by be
and the statistics analyzed actively.

When the pipeline runs the case, the statistics reported by be may not be timely. Therefore, the outcome that leads to the cbo optimizer's successful selection of overwrites is not very certain, so the statistics are currently injected manually in the test cases

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

doris-robot · 2024-11-12T13:09:14Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

seawinde · 2024-11-12T13:09:19Z

run buildall

seawinde · 2024-11-13T13:40:45Z

run buildall

seawinde · 2024-11-13T23:15:34Z

run buildall

seawinde · 2024-11-14T00:33:31Z

run buildall

seawinde · 2024-11-14T06:24:42Z

run buildall

seawinde · 2024-11-14T23:50:48Z

run buildall

seawinde · 2024-11-19T07:35:34Z

run p0 5

github-actions · 2024-11-22T08:56:55Z

PR approved by at least one committer and no changes requested.

github-actions · 2024-11-22T08:56:56Z

PR approved by anyone and no changes requested.

…make sure rewrite result stable

seawinde · 2024-11-27T09:27:02Z

run buildall

doris-robot · 2024-11-27T10:05:38Z

TPC-H: Total hot run time: 39871 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit dde934637e7ae243c6e2bbb327b5a418c7f5f3d7, data reload: false

------ Round 1 ----------------------------------
q1	17570	7711	7251	7251
q2	2042	172	164	164
q3	10725	1075	1177	1075
q4	10750	719	773	719
q5	7610	2701	2666	2666
q6	247	149	148	148
q7	982	630	613	613
q8	9530	1840	1917	1840
q9	6956	6439	6437	6437
q10	7026	2319	2354	2319
q11	466	259	255	255
q12	419	209	207	207
q13	20884	3025	3026	3025
q14	254	207	218	207
q15	579	519	515	515
q16	680	582	579	579
q17	972	504	607	504
q18	7313	6688	6684	6684
q19	1346	1051	978	978
q20	461	178	178	178
q21	3950	3291	3161	3161
q22	387	346	351	346
Total cold run time: 111149 ms
Total hot run time: 39871 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7388	7357	7351	7351
q2	327	232	228	228
q3	3142	3127	3096	3096
q4	2167	1892	1869	1869
q5	5839	5783	5766	5766
q6	223	143	145	143
q7	2239	1809	1821	1809
q8	3355	3503	3484	3484
q9	8854	8837	8811	8811
q10	3594	3531	3497	3497
q11	598	500	510	500
q12	841	625	638	625
q13	12501	3292	3189	3189
q14	309	285	269	269
q15	575	537	519	519
q16	697	663	660	660
q17	1821	1616	1584	1584
q18	8318	7791	7636	7636
q19	1745	1349	1584	1349
q20	2132	1928	1882	1882
q21	5409	5312	5266	5266
q22	654	562	566	562
Total cold run time: 72728 ms
Total hot run time: 60095 ms

doris-robot · 2024-11-27T10:18:21Z

TPC-DS: Total hot run time: 197014 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit dde934637e7ae243c6e2bbb327b5a418c7f5f3d7, data reload: false

query1	1247	949	930	930
query2	6229	2121	2095	2095
query3	10957	4009	3996	3996
query4	67888	29082	24008	24008
query5	4949	460	446	446
query6	427	188	178	178
query7	5588	289	282	282
query8	294	222	212	212
query9	9142	2681	2693	2681
query10	459	246	252	246
query11	17475	15250	15691	15250
query12	162	100	105	100
query13	1529	414	399	399
query14	10991	7079	7139	7079
query15	217	185	200	185
query16	7219	442	467	442
query17	1243	596	600	596
query18	1831	311	301	301
query19	225	155	158	155
query20	127	117	116	116
query21	208	113	107	107
query22	4619	4803	4483	4483
query23	34771	34627	34344	34344
query24	5457	2465	2441	2441
query25	487	377	427	377
query26	645	150	147	147
query27	1981	279	282	279
query28	4395	2468	2450	2450
query29	684	423	435	423
query30	215	150	143	143
query31	1007	820	881	820
query32	67	57	58	57
query33	446	298	293	293
query34	947	523	522	522
query35	892	756	749	749
query36	1115	963	986	963
query37	123	77	75	75
query38	4421	4379	4485	4379
query39	1558	1494	1472	1472
query40	199	98	95	95
query41	44	41	47	41
query42	110	100	99	99
query43	553	520	508	508
query44	1175	815	798	798
query45	187	163	170	163
query46	1140	707	726	707
query47	2104	1938	1962	1938
query48	428	324	347	324
query49	728	380	391	380
query50	843	390	402	390
query51	7480	7318	7100	7100
query52	99	85	90	85
query53	256	176	179	176
query54	497	393	401	393
query55	76	78	76	76
query56	249	236	248	236
query57	1303	1206	1171	1171
query58	213	206	214	206
query59	3244	3137	3078	3078
query60	258	239	248	239
query61	109	106	103	103
query62	771	657	645	645
query63	215	198	185	185
query64	1357	704	644	644
query65	3297	3190	3215	3190
query66	703	294	290	290
query67	16037	15647	15663	15647
query68	3996	581	556	556
query69	432	248	268	248
query70	1175	1152	1090	1090
query71	356	258	259	258
query72	6400	3997	4070	3997
query73	761	366	355	355
query74	10299	9079	9037	9037
query75	3397	2665	2644	2644
query76	2127	1063	964	964
query77	459	274	283	274
query78	10451	9490	9310	9310
query79	1908	598	585	585
query80	1326	484	427	427
query81	523	241	228	228
query82	1316	123	116	116
query83	225	155	158	155
query84	284	71	67	67
query85	997	304	293	293
query86	412	311	315	311
query87	4729	4650	4640	4640
query88	3630	2228	2295	2228
query89	419	296	292	292
query90	1953	190	184	184
query91	137	102	105	102
query92	60	50	56	50
query93	2535	529	522	522
query94	813	290	279	279
query95	358	249	246	246
query96	633	283	287	283
query97	2874	2708	2645	2645
query98	208	193	194	193
query99	1591	1307	1311	1307
Total cold run time: 322696 ms
Total hot run time: 197014 ms

doris-robot · 2024-11-27T10:23:51Z

ClickBench: Total hot run time: 32.7 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit dde934637e7ae243c6e2bbb327b5a418c7f5f3d7, data reload: false

query1	0.03	0.03	0.03
query2	0.06	0.03	0.04
query3	0.24	0.06	0.07
query4	1.64	0.10	0.10
query5	0.43	0.41	0.40
query6	1.15	0.65	0.65
query7	0.02	0.02	0.02
query8	0.04	0.04	0.03
query9	0.57	0.53	0.49
query10	0.55	0.58	0.56
query11	0.14	0.09	0.10
query12	0.14	0.12	0.11
query13	0.61	0.61	0.60
query14	2.83	2.85	2.86
query15	0.90	0.82	0.84
query16	0.38	0.38	0.39
query17	1.06	0.96	1.08
query18	0.22	0.22	0.20
query19	1.92	1.74	2.03
query20	0.01	0.01	0.02
query21	15.39	0.58	0.58
query22	3.15	1.65	1.86
query23	17.00	0.83	0.79
query24	3.40	1.33	1.79
query25	0.26	0.19	0.09
query26	0.63	0.13	0.14
query27	0.04	0.04	0.04
query28	9.46	1.12	1.09
query29	12.60	3.26	3.22
query30	0.25	0.07	0.06
query31	2.86	0.39	0.37
query32	3.27	0.47	0.46
query33	2.99	3.06	3.06
query34	17.15	4.44	4.44
query35	4.52	4.55	4.48
query36	0.68	0.48	0.48
query37	0.10	0.06	0.06
query38	0.04	0.03	0.04
query39	0.04	0.02	0.02
query40	0.15	0.12	0.13
query41	0.07	0.02	0.02
query42	0.04	0.02	0.02
query43	0.03	0.04	0.03
Total cold run time: 107.06 s
Total hot run time: 32.7 s

seawinde · 2024-11-27T11:06:15Z

run buildall

doris-robot · 2024-11-27T11:25:13Z

TPC-H: Total hot run time: 40071 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ed3daf98afd168e98dcf264c244311c7db56c757, data reload: false

------ Round 1 ----------------------------------
q1	17614	7441	7306	7306
q2	2041	178	175	175
q3	10542	1129	1184	1129
q4	10218	784	757	757
q5	7609	2762	2704	2704
q6	240	149	151	149
q7	981	636	620	620
q8	9260	1855	1907	1855
q9	6570	6387	6322	6322
q10	6997	2300	2312	2300
q11	455	259	259	259
q12	407	223	223	223
q13	17758	3035	3108	3035
q14	249	206	220	206
q15	554	524	524	524
q16	651	585	583	583
q17	977	574	543	543
q18	7628	6711	6703	6703
q19	1336	1043	1055	1043
q20	480	195	189	189
q21	3980	3189	3127	3127
q22	384	319	322	319
Total cold run time: 106931 ms
Total hot run time: 40071 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7269	7357	7234	7234
q2	320	230	235	230
q3	2880	2730	3111	2730
q4	2085	1822	1735	1735
q5	5531	5695	5624	5624
q6	223	142	143	142
q7	2251	1784	1787	1784
q8	3382	3555	3489	3489
q9	8905	8913	8870	8870
q10	3609	3537	3540	3537
q11	602	501	488	488
q12	839	645	587	587
q13	13133	3267	3267	3267
q14	293	278	264	264
q15	570	532	523	523
q16	680	648	621	621
q17	1868	1623	1566	1566
q18	7919	7426	7274	7274
q19	1661	1423	1482	1423
q20	2029	1806	1822	1806
q21	5274	5263	5236	5236
q22	620	554	590	554
Total cold run time: 71943 ms
Total hot run time: 58984 ms

doris-robot · 2024-11-27T11:37:20Z

TPC-DS: Total hot run time: 191174 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ed3daf98afd168e98dcf264c244311c7db56c757, data reload: false

query1	962	389	378	378
query2	6515	2176	2082	2082
query3	6711	209	210	209
query4	34272	23674	23525	23525
query5	4323	456	444	444
query6	295	184	171	171
query7	4636	302	304	302
query8	303	227	228	227
query9	9471	2712	2699	2699
query10	468	253	254	253
query11	18203	15314	15156	15156
query12	147	101	100	100
query13	1639	435	401	401
query14	9655	7516	6793	6793
query15	302	167	183	167
query16	8198	421	488	421
query17	1800	567	555	555
query18	2142	293	292	292
query19	362	147	140	140
query20	113	107	119	107
query21	204	102	102	102
query22	4617	4218	4185	4185
query23	35057	34320	34043	34043
query24	10389	2426	2424	2424
query25	635	386	388	386
query26	1400	153	154	153
query27	2883	281	276	276
query28	8024	2424	2466	2424
query29	865	403	419	403
query30	297	150	153	150
query31	1038	785	812	785
query32	97	56	58	56
query33	779	285	321	285
query34	947	496	515	496
query35	905	743	766	743
query36	1107	922	958	922
query37	149	77	75	75
query38	4451	4295	4248	4248
query39	1460	1402	1414	1402
query40	275	99	98	98
query41	47	47	44	44
query42	123	104	103	103
query43	546	490	495	490
query44	1234	800	793	793
query45	182	179	160	160
query46	1142	714	697	697
query47	1971	1847	1867	1847
query48	396	323	315	315
query49	1147	389	386	386
query50	813	388	388	388
query51	7356	7088	7201	7088
query52	101	97	85	85
query53	256	179	180	179
query54	1304	405	415	405
query55	80	78	84	78
query56	265	255	229	229
query57	1362	1170	1144	1144
query58	246	216	215	215
query59	3248	3195	3051	3051
query60	279	263	257	257
query61	136	126	132	126
query62	883	681	688	681
query63	212	190	184	184
query64	5414	757	785	757
query65	3310	3187	3251	3187
query66	1468	315	330	315
query67	16043	15552	15659	15552
query68	4509	564	562	562
query69	420	251	259	251
query70	1174	1122	1107	1107
query71	316	250	246	246
query72	6422	4084	4040	4040
query73	779	448	367	367
query74	10123	9060	9066	9060
query75	3437	2653	2685	2653
query76	2869	1107	1137	1107
query77	385	279	277	277
query78	10502	9501	9376	9376
query79	2506	605	624	605
query80	1146	429	438	429
query81	559	226	230	226
query82	635	120	118	118
query83	239	147	148	147
query84	241	74	74	74
query85	1737	302	295	295
query86	485	307	303	303
query87	4817	4770	4689	4689
query88	3924	2256	2209	2209
query89	401	303	300	300
query90	1983	189	190	189
query91	143	105	108	105
query92	66	60	54	54
query93	1762	545	535	535
query94	849	288	295	288
query95	363	249	248	248
query96	630	286	287	286
query97	2894	2676	2676	2676
query98	229	195	196	195
query99	1589	1341	1323	1323
Total cold run time: 303794 ms
Total hot run time: 191174 ms

doris-robot · 2024-11-27T11:42:49Z

ClickBench: Total hot run time: 32.63 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit ed3daf98afd168e98dcf264c244311c7db56c757, data reload: false

query1	0.03	0.03	0.02
query2	0.06	0.03	0.03
query3	0.23	0.08	0.08
query4	1.61	0.10	0.10
query5	0.42	0.40	0.41
query6	1.14	0.66	0.64
query7	0.02	0.01	0.01
query8	0.05	0.04	0.04
query9	0.56	0.53	0.50
query10	0.56	0.56	0.56
query11	0.15	0.11	0.11
query12	0.14	0.10	0.12
query13	0.62	0.61	0.60
query14	2.70	2.83	2.83
query15	0.91	0.84	0.82
query16	0.41	0.38	0.38
query17	1.06	1.05	1.05
query18	0.21	0.20	0.21
query19	1.85	1.82	1.99
query20	0.02	0.01	0.02
query21	15.36	0.60	0.58
query22	2.31	2.19	1.85
query23	16.90	0.91	0.94
query24	3.16	0.78	1.14
query25	0.18	0.20	0.11
query26	0.39	0.14	0.14
query27	0.05	0.06	0.04
query28	10.65	1.11	1.08
query29	12.57	3.27	3.20
query30	0.27	0.06	0.06
query31	2.87	0.38	0.37
query32	3.27	0.46	0.46
query33	3.14	3.06	3.01
query34	17.14	4.48	4.48
query35	4.46	4.54	4.48
query36	0.67	0.51	0.48
query37	0.10	0.06	0.06
query38	0.05	0.04	0.03
query39	0.04	0.02	0.02
query40	0.16	0.12	0.12
query41	0.08	0.03	0.02
query42	0.04	0.03	0.02
query43	0.03	0.03	0.03
Total cold run time: 106.64 s
Total hot run time: 32.63 s

zfr9527

LGTM

github-actions · 2024-11-29T06:28:29Z

PR approved by at least one committer and no changes requested.

…make sure rewrite result stable (apache#43785) The result of successful rewriting by the cbo optimizer depends on the statistics. The priority of the optimizer consumption statistics in descending order is 1. the injected statistics 2. the statistics reported by be 3. and the statistics analyzed actively. When the pipeline runs the case, the statistics reported by be may not be timely. Therefore, the outcome that leads to the cbo optimizer's successful selection of overwrites is not very certain, so the statistics are currently injected manually in the test cases

…make sure rewrite result stable (#43785) (#44826) pick from master #43785

…make sure rewrite result stable (apache#43785) The result of successful rewriting by the cbo optimizer depends on the statistics. The priority of the optimizer consumption statistics in descending order is 1. the injected statistics 2. the statistics reported by be 3. and the statistics analyzed actively. When the pipeline runs the case, the statistics reported by be may not be timely. Therefore, the outcome that leads to the cbo optimizer's successful selection of overwrites is not very certain, so the statistics are currently injected manually in the test cases

seawinde force-pushed the fix_mv_test_unstable_inject_statistic branch from 8c5650e to e0149ac Compare November 14, 2024 06:23

seawinde force-pushed the fix_mv_test_unstable_inject_statistic branch from e0149ac to 91409b6 Compare November 14, 2024 23:50

morrySnow added dev/2.1.x dev/3.0.x labels Nov 19, 2024

morrySnow previously approved these changes Nov 22, 2024

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Nov 22, 2024

github-actions bot added the reviewed label Nov 22, 2024

seawinde added 6 commits November 27, 2024 15:50

[test](mtmv) Add inject statistic when mv rewrite regression test to …

d197a32

…make sure rewrite result stable

fix regression test

f51a916

fix regression case

4ccd6c6

fix licence

9df64bc

fix regression test

767509e

fix test fail

dde9346

seawinde dismissed morrySnow’s stale review via dde9346 November 27, 2024 09:26

seawinde force-pushed the fix_mv_test_unstable_inject_statistic branch from 91409b6 to dde9346 Compare November 27, 2024 09:26

github-actions bot removed the approved Indicates a PR has been approved by one committer. label Nov 27, 2024

fix test fail

ed3daf9

zfr9527 approved these changes Nov 28, 2024

View reviewed changes

morrySnow approved these changes Nov 29, 2024

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Nov 29, 2024

morrySnow merged commit 9daa3b7 into apache:master Nov 29, 2024

github-actions bot added dev/3.0.x-conflict dev/2.1.x-conflict labels Nov 29, 2024

seawinde mentioned this pull request Dec 1, 2024

[test](mtmv) Add inject statistic when mv rewrite regression test to make sure rewrite result stable (#43785) #44826

Merged

16 tasks

morrySnow pushed a commit that referenced this pull request Dec 6, 2024

[test](mtmv) Add inject statistic when mv rewrite regression test to …

5980ab0

…make sure rewrite result stable (#43785) (#44826) pick from master #43785

morrySnow added dev/3.0.4-merged and removed dev/3.0.x dev/3.0.x-conflict labels Dec 6, 2024

seawinde mentioned this pull request Dec 6, 2024

Pick some pr to 21 #43010 #43030 #43785 #44779 #44786 #44857 #45129

Merged

16 tasks

yiguolei pushed a commit that referenced this pull request Dec 9, 2024

Pick some pr to 21 #43010 #43030 #43785 #44779 #44786 #44857 (#45129)

1662e47

yiguolei added dev/2.1.8-merged and removed dev/2.1.x dev/2.1.x-conflict labels Dec 9, 2024

Conversation

seawinde commented Nov 12, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

doris-robot commented Nov 12, 2024

Uh oh!

seawinde commented Nov 12, 2024

Uh oh!

seawinde commented Nov 13, 2024

Uh oh!

seawinde commented Nov 13, 2024

Uh oh!

seawinde commented Nov 14, 2024

Uh oh!

seawinde commented Nov 14, 2024

Uh oh!

seawinde commented Nov 14, 2024

Uh oh!

seawinde commented Nov 19, 2024

Uh oh!

github-actions bot commented Nov 22, 2024

Uh oh!

github-actions bot commented Nov 22, 2024

Uh oh!

seawinde commented Nov 27, 2024

Uh oh!

doris-robot commented Nov 27, 2024

Uh oh!

doris-robot commented Nov 27, 2024

Uh oh!

doris-robot commented Nov 27, 2024

Uh oh!

seawinde commented Nov 27, 2024

Uh oh!

doris-robot commented Nov 27, 2024

Uh oh!

doris-robot commented Nov 27, 2024

Uh oh!

doris-robot commented Nov 27, 2024

Uh oh!

zfr9527 left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Nov 29, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

seawinde commented Nov 12, 2024 •

edited

Loading