Revert "[fix] (compaction) fix compaction score in time series policy(#40242)" by csun5285 · Pull Request #44525 · apache/doris

csun5285 · 2024-11-25T03:27:15Z

revert #40242

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

doris-robot · 2024-11-25T03:27:20Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

csun5285 · 2024-11-25T03:27:30Z

run buildall

github-actions · 2024-11-25T03:31:51Z

clang-tidy review says "All clean, LGTM! 👍"

doris-robot · 2024-11-25T05:09:25Z

TPC-H: Total hot run time: 40067 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 13047a2b76ad3c75a47a9f098c66ebc1e183e90b, data reload: false

------ Round 1 ----------------------------------
q1	17684	7520	7312	7312
q2	2048	175	170	170
q3	10980	1125	1166	1125
q4	10333	798	718	718
q5	7605	2777	2785	2777
q6	238	153	147	147
q7	968	624	607	607
q8	9242	1858	1949	1858
q9	6519	6411	6421	6411
q10	7051	2335	2327	2327
q11	467	251	267	251
q12	413	216	216	216
q13	17781	3059	3000	3000
q14	233	209	213	209
q15	579	547	531	531
q16	677	600	578	578
q17	996	525	629	525
q18	7318	6632	6619	6619
q19	1329	989	1039	989
q20	487	177	175	175
q21	4004	3212	3350	3212
q22	391	316	310	310
Total cold run time: 107343 ms
Total hot run time: 40067 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7217	7608	7288	7288
q2	326	229	225	225
q3	2898	2898	2982	2898
q4	2098	1810	1854	1810
q5	5690	5685	5731	5685
q6	223	137	139	137
q7	2248	1841	1814	1814
q8	3382	3554	3594	3554
q9	8773	8906	8860	8860
q10	3622	3590	3560	3560
q11	596	505	495	495
q12	809	625	618	618
q13	11976	3189	3219	3189
q14	296	277	266	266
q15	600	527	528	527
q16	689	654	650	650
q17	1868	1648	1629	1629
q18	8316	7748	7702	7702
q19	1734	1654	1522	1522
q20	2137	1892	1868	1868
q21	5474	5360	5484	5360
q22	706	567	568	567
Total cold run time: 71678 ms
Total hot run time: 60224 ms

doris-robot · 2024-11-25T05:22:08Z

TPC-DS: Total hot run time: 197041 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 13047a2b76ad3c75a47a9f098c66ebc1e183e90b, data reload: false

query1	1246	964	937	937
query2	6222	2118	2053	2053
query3	10779	4056	3951	3951
query4	67574	29092	23509	23509
query5	4981	466	444	444
query6	403	191	197	191
query7	5515	306	288	288
query8	322	234	233	233
query9	8837	2646	2644	2644
query10	438	258	264	258
query11	17111	15242	15940	15242
query12	169	113	100	100
query13	1527	438	447	438
query14	10798	7696	7785	7696
query15	202	181	174	174
query16	7088	450	458	450
query17	1449	600	567	567
query18	1832	303	308	303
query19	202	158	156	156
query20	123	141	111	111
query21	197	107	107	107
query22	4667	4589	4441	4441
query23	34631	34420	34467	34420
query24	5509	2498	2549	2498
query25	506	395	403	395
query26	647	151	159	151
query27	1934	286	292	286
query28	4815	2464	2479	2464
query29	678	418	426	418
query30	219	150	149	149
query31	1063	835	854	835
query32	69	55	56	55
query33	482	307	302	302
query34	932	531	526	526
query35	834	751	738	738
query36	1062	969	957	957
query37	125	81	75	75
query38	4426	4373	4572	4373
query39	1550	1472	1476	1472
query40	203	98	103	98
query41	49	44	46	44
query42	107	99	115	99
query43	547	496	504	496
query44	1228	854	835	835
query45	218	176	170	170
query46	1137	728	697	697
query47	2007	1900	1924	1900
query48	429	323	340	323
query49	762	406	395	395
query50	851	410	418	410
query51	7406	7298	7071	7071
query52	102	88	84	84
query53	253	182	176	176
query54	507	390	391	390
query55	78	74	75	74
query56	263	254	243	243
query57	1315	1177	1144	1144
query58	217	248	213	213
query59	3185	2949	3063	2949
query60	273	249	249	249
query61	115	110	110	110
query62	779	682	678	678
query63	221	191	193	191
query64	1431	691	623	623
query65	3269	3201	3231	3201
query66	715	302	301	301
query67	16007	15538	15574	15538
query68	4016	560	549	549
query69	438	252	255	252
query70	1196	1108	1111	1108
query71	361	258	254	254
query72	6446	4171	4064	4064
query73	757	354	355	354
query74	10276	8986	8941	8941
query75	3397	2700	2690	2690
query76	1855	1037	1137	1037
query77	474	279	271	271
query78	11029	9508	9375	9375
query79	2225	592	629	592
query80	1180	461	468	461
query81	532	226	231	226
query82	1120	118	119	118
query83	244	143	144	143
query84	285	71	73	71
query85	1010	304	301	301
query86	432	303	305	303
query87	4806	4582	4589	4582
query88	3794	2210	2123	2123
query89	415	288	317	288
query90	2019	187	186	186
query91	136	102	107	102
query92	72	50	51	50
query93	2821	543	537	537
query94	868	295	297	295
query95	359	245	250	245
query96	636	276	273	273
query97	2866	2697	2660	2660
query98	220	197	193	193
query99	1605	1344	1288	1288
Total cold run time: 322728 ms
Total hot run time: 197041 ms

doris-robot · 2024-11-25T05:27:37Z

ClickBench: Total hot run time: 31.58 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 13047a2b76ad3c75a47a9f098c66ebc1e183e90b, data reload: false

query1	0.03	0.04	0.02
query2	0.07	0.03	0.03
query3	0.23	0.07	0.07
query4	1.62	0.10	0.11
query5	0.41	0.41	0.42
query6	1.19	0.65	0.64
query7	0.02	0.02	0.02
query8	0.04	0.03	0.03
query9	0.57	0.51	0.52
query10	0.54	0.55	0.55
query11	0.14	0.11	0.11
query12	0.13	0.11	0.11
query13	0.61	0.61	0.60
query14	2.84	2.80	2.70
query15	0.92	0.83	0.82
query16	0.41	0.38	0.39
query17	1.08	1.05	1.05
query18	0.22	0.20	0.20
query19	1.88	1.72	1.93
query20	0.02	0.01	0.01
query21	15.38	0.58	0.55
query22	2.94	2.61	1.64
query23	17.13	0.94	0.73
query24	3.15	0.47	1.65
query25	0.17	0.08	0.06
query26	0.53	0.14	0.14
query27	0.04	0.05	0.05
query28	10.42	1.12	1.07
query29	12.53	3.19	3.16
query30	0.25	0.06	0.06
query31	2.87	0.38	0.38
query32	3.29	0.46	0.46
query33	3.01	2.97	3.09
query34	16.88	4.47	4.51
query35	4.52	4.52	4.50
query36	0.67	0.49	0.49
query37	0.09	0.06	0.05
query38	0.04	0.04	0.04
query39	0.04	0.02	0.02
query40	0.16	0.12	0.12
query41	0.08	0.02	0.02
query42	0.04	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 107.24 s
Total hot run time: 31.58 s

xiaokang

LGTM

github-actions · 2024-12-09T00:50:12Z

PR approved by at least one committer and no changes requested.

github-actions · 2024-12-09T00:50:15Z

PR approved by anyone and no changes requested.

csun5285 · 2024-12-18T06:39:27Z

run be ut

csun5285 · 2024-12-18T08:30:11Z

run buildall

github-actions · 2024-12-18T08:34:45Z

clang-tidy review says "All clean, LGTM! 👍"

doris-robot · 2024-12-18T09:03:58Z

TeamCity be ut coverage result:
Function Coverage: 38.87% (10133/26066)
Line Coverage: 29.79% (85176/285969)
Region Coverage: 28.84% (43696/151524)
Branch Coverage: 25.38% (22202/87482)
Coverage Report: http://coverage.selectdb-in.cc/coverage/215e55273958061298f2ff658506a2caa9613e93_215e55273958061298f2ff658506a2caa9613e93/report/index.html

doris-robot · 2024-12-18T09:08:05Z

TPC-H: Total hot run time: 39742 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 215e55273958061298f2ff658506a2caa9613e93, data reload: false

------ Round 1 ----------------------------------
q1	17608	7536	7247	7247
q2	2045	182	180	180
q3	10543	1085	1191	1085
q4	10560	747	708	708
q5	7597	2678	2705	2678
q6	242	147	145	145
q7	986	635	629	629
q8	9243	1838	1878	1838
q9	6576	6410	6404	6404
q10	7037	2267	2341	2267
q11	466	269	263	263
q12	421	237	235	235
q13	17781	2920	2937	2920
q14	246	210	208	208
q15	575	550	496	496
q16	657	611	587	587
q17	979	553	570	553
q18	7358	6880	6701	6701
q19	1349	1040	987	987
q20	467	186	181	181
q21	4164	3119	3258	3119
q22	382	319	311	311
Total cold run time: 107282 ms
Total hot run time: 39742 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7222	7228	7201	7201
q2	325	224	238	224
q3	2887	2781	2929	2781
q4	2095	1842	1840	1840
q5	5732	5669	5655	5655
q6	224	147	142	142
q7	2256	1799	1822	1799
q8	3390	3532	3577	3532
q9	8867	8994	8980	8980
q10	3609	3554	3566	3554
q11	601	498	511	498
q12	835	596	575	575
q13	12187	3167	3166	3166
q14	319	277	266	266
q15	555	500	515	500
q16	692	645	663	645
q17	1847	1635	1590	1590
q18	8383	7835	7597	7597
q19	1764	1485	1548	1485
q20	2082	1902	1854	1854
q21	5605	5532	5349	5349
q22	656	592	569	569
Total cold run time: 72133 ms
Total hot run time: 59802 ms

…apache#40242)" (apache#44491) revert apache#40242

csun5285 · 2024-12-18T09:10:04Z

run buildall

github-actions · 2024-12-18T09:10:40Z

`sh-checker report`

To get the full details, please check in the job output.

shellcheck errors


'shellcheck ' returned error 1 finding the following syntactical issues:

----------

In gensrc/script/gen_build_version.sh line 38:
if [[ ${build_version_hotfix} > 0 ]]; then
                              ^-- SC2071 (error): > is for string comparisons. Use -gt instead.


In gensrc/script/gen_build_version.sh line 228:
if [[ ${build_version_hotfix} > 0 ]]; then
                              ^-- SC2071 (error): > is for string comparisons. Use -gt instead.

For more information:
  https://www.shellcheck.net/wiki/SC2071 -- > is for string comparisons. Use ...
----------

You can address the above issues in one of three ways:
1. Manually correct the issue in the offending shell script;
2. Disable specific issues by adding the comment:
  # shellcheck disable=NNNN
above the line that contains the issue, where NNNN is the error code;
3. Add '-e NNNN' to the SHELLCHECK_OPTS setting in your .yml action file.

shfmt errors

'shfmt ' found no issues.

github-actions · 2024-12-18T09:14:43Z

clang-tidy review says "All clean, LGTM! 👍"

doris-robot · 2024-12-18T09:37:16Z

TeamCity be ut coverage result:
Function Coverage: 38.87% (10133/26066)
Line Coverage: 29.80% (85206/285969)
Region Coverage: 28.84% (43694/151524)
Branch Coverage: 25.38% (22200/87482)
Coverage Report: http://coverage.selectdb-in.cc/coverage/b66a1465d5482f4e3c14e896282794924d23a580_b66a1465d5482f4e3c14e896282794924d23a580/report/index.html

doris-robot · 2024-12-18T09:40:56Z

TPC-H: Total hot run time: 39668 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit b66a1465d5482f4e3c14e896282794924d23a580, data reload: false

------ Round 1 ----------------------------------
q1	17595	7435	7258	7258
q2	2039	177	163	163
q3	10712	1112	1205	1112
q4	10561	736	712	712
q5	7600	2753	2660	2660
q6	244	149	147	147
q7	977	629	615	615
q8	9232	1838	1905	1838
q9	6617	6497	6485	6485
q10	7011	2296	2313	2296
q11	472	265	251	251
q12	423	226	232	226
q13	17775	2892	2987	2892
q14	238	214	208	208
q15	548	528	502	502
q16	638	592	591	591
q17	977	613	521	521
q18	7221	6683	6549	6549
q19	1347	1013	995	995
q20	497	184	181	181
q21	4060	3265	3152	3152
q22	380	317	314	314
Total cold run time: 107164 ms
Total hot run time: 39668 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7219	7238	7215	7215
q2	332	227	233	227
q3	2871	2850	2972	2850
q4	2092	1917	1836	1836
q5	5679	5645	5649	5645
q6	222	145	146	145
q7	2263	1813	1824	1813
q8	3426	3571	3492	3492
q9	8900	8998	8975	8975
q10	3608	3531	3546	3531
q11	599	507	515	507
q12	825	588	582	582
q13	12224	3222	3105	3105
q14	314	306	274	274
q15	551	503	500	500
q16	676	632	663	632
q17	1863	1622	1597	1597
q18	8181	7789	7653	7653
q19	1716	1492	1587	1492
q20	2109	1853	1833	1833
q21	5641	5501	5491	5491
q22	624	601	574	574
Total cold run time: 71935 ms
Total hot run time: 59969 ms

doris-robot · 2024-12-18T09:53:19Z

TPC-DS: Total hot run time: 197677 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit b66a1465d5482f4e3c14e896282794924d23a580, data reload: false

query1	1311	947	945	945
query2	6230	2247	2232	2232
query3	11141	4481	4883	4481
query4	33372	23575	23411	23411
query5	4801	472	449	449
query6	282	185	174	174
query7	3984	299	306	299
query8	294	237	239	237
query9	9729	2721	2704	2704
query10	497	259	254	254
query11	18106	15212	15214	15212
query12	156	106	104	104
query13	1593	434	418	418
query14	10243	7827	7946	7827
query15	269	194	213	194
query16	7982	512	507	507
query17	1742	613	595	595
query18	2114	321	338	321
query19	383	180	163	163
query20	127	125	119	119
query21	205	113	147	113
query22	4837	4471	4542	4471
query23	35028	34547	33977	33977
query24	10583	2583	2534	2534
query25	577	385	404	385
query26	727	168	160	160
query27	2284	334	335	334
query28	6371	2468	2470	2468
query29	749	435	424	424
query30	231	148	149	148
query31	1047	840	832	832
query32	98	60	59	59
query33	766	312	329	312
query34	1094	547	531	531
query35	899	796	762	762
query36	1135	969	1002	969
query37	122	75	82	75
query38	4437	4200	4150	4150
query39	1502	1465	1462	1462
query40	210	108	107	107
query41	45	46	44	44
query42	126	106	102	102
query43	538	508	482	482
query44	1311	861	858	858
query45	196	173	168	168
query46	1182	750	778	750
query47	2018	1929	1937	1929
query48	436	336	318	318
query49	852	396	402	396
query50	837	414	397	397
query51	7328	7125	7173	7125
query52	105	89	94	89
query53	279	191	184	184
query54	1153	431	435	431
query55	99	85	78	78
query56	271	257	248	248
query57	1302	1203	1156	1156
query58	248	227	228	227
query59	3274	3097	3049	3049
query60	284	262	259	259
query61	106	102	121	102
query62	882	716	704	704
query63	218	203	201	201
query64	3786	691	650	650
query65	3291	3212	3358	3212
query66	761	363	312	312
query67	16486	15545	15448	15448
query68	5797	533	534	533
query69	509	260	268	260
query70	1232	1097	1070	1070
query71	485	268	274	268
query72	7011	4159	4064	4064
query73	801	359	367	359
query74	10175	8883	8886	8883
query75	3938	2639	2637	2637
query76	3851	1124	1126	1124
query77	553	298	275	275
query78	10293	9441	9419	9419
query79	2096	603	601	601
query80	1352	438	430	430
query81	514	241	237	237
query82	596	118	118	118
query83	188	147	155	147
query84	284	73	68	68
query85	1308	305	307	305
query86	383	275	289	275
query87	4482	4432	4566	4432
query88	3971	2196	2178	2178
query89	428	293	287	287
query90	2108	191	192	191
query91	140	106	103	103
query92	62	49	50	49
query93	4331	545	523	523
query94	825	279	293	279
query95	345	251	253	251
query96	652	273	281	273
query97	2910	2694	2698	2694
query98	220	196	200	196
query99	1607	1361	1310	1310
Total cold run time: 308119 ms
Total hot run time: 197677 ms

doris-robot · 2024-12-18T09:58:49Z

ClickBench: Total hot run time: 32.47 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit b66a1465d5482f4e3c14e896282794924d23a580, data reload: false

query1	0.03	0.03	0.03
query2	0.06	0.03	0.03
query3	0.23	0.07	0.07
query4	1.61	0.10	0.11
query5	0.42	0.41	0.43
query6	1.15	0.65	0.65
query7	0.02	0.01	0.02
query8	0.04	0.03	0.03
query9	0.59	0.51	0.51
query10	0.55	0.59	0.55
query11	0.14	0.11	0.10
query12	0.14	0.11	0.11
query13	0.61	0.61	0.60
query14	2.83	2.80	2.87
query15	0.90	0.83	0.84
query16	0.40	0.39	0.39
query17	1.03	1.06	1.00
query18	0.23	0.21	0.21
query19	1.93	1.79	2.00
query20	0.01	0.01	0.01
query21	15.37	0.61	0.60
query22	2.69	2.65	1.62
query23	16.98	1.04	0.87
query24	3.13	1.52	0.75
query25	0.19	0.15	0.18
query26	0.35	0.13	0.14
query27	0.04	0.04	0.04
query28	10.44	1.11	1.07
query29	12.57	3.31	3.27
query30	0.24	0.06	0.06
query31	2.85	0.40	0.38
query32	3.24	0.47	0.46
query33	3.08	3.25	3.09
query34	17.02	4.48	4.54
query35	4.51	4.49	4.51
query36	0.68	0.49	0.49
query37	0.09	0.06	0.06
query38	0.04	0.03	0.03
query39	0.03	0.02	0.02
query40	0.17	0.12	0.12
query41	0.08	0.02	0.02
query42	0.04	0.02	0.02
query43	0.04	0.03	0.02
Total cold run time: 106.79 s
Total hot run time: 32.47 s

zzzxl1993

LGTM

airborne12

LGTM

github-actions · 2024-12-18T11:08:47Z

PR approved by at least one committer and no changes requested.

xiaokang previously approved these changes Dec 9, 2024

View reviewed changes

github-actions bot added approved Indicates a PR has been approved by one committer. reviewed labels Dec 9, 2024

csun5285 dismissed xiaokang’s stale review via 215e552 December 18, 2024 08:29

github-actions bot removed the approved Indicates a PR has been approved by one committer. label Dec 18, 2024

csun5285 added 2 commits December 18, 2024 17:09

Revert "[fix] (compaction) fix compaction score in time series policy (…

ef6962c

…apache#40242)" (apache#44491) revert apache#40242

fix be ut

b66a146

csun5285 force-pushed the revert_fix_compaction_score_master branch from 215e552 to b66a146 Compare December 18, 2024 09:09

zzzxl1993 approved these changes Dec 18, 2024

View reviewed changes

airborne12 added the dev/2.1.8-merged label Dec 18, 2024

airborne12 added the dev/3.0.4-merged label Dec 18, 2024

airborne12 approved these changes Dec 18, 2024

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Dec 18, 2024

airborne12 merged commit f167700 into apache:master Dec 18, 2024

xy720 mentioned this pull request Feb 28, 2025

branch-3.0: [fix](beut) fix be ut pipeline #48465

Closed

Conversation

csun5285 commented Nov 25, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

doris-robot commented Nov 25, 2024

Uh oh!

csun5285 commented Nov 25, 2024

Uh oh!

github-actions bot commented Nov 25, 2024

Uh oh!

doris-robot commented Nov 25, 2024

Uh oh!

doris-robot commented Nov 25, 2024

Uh oh!

doris-robot commented Nov 25, 2024

Uh oh!

xiaokang left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Dec 9, 2024

Uh oh!

github-actions bot commented Dec 9, 2024

Uh oh!

csun5285 commented Dec 18, 2024

Uh oh!

csun5285 commented Dec 18, 2024

Uh oh!

github-actions bot commented Dec 18, 2024

Uh oh!

doris-robot commented Dec 18, 2024

Uh oh!

doris-robot commented Dec 18, 2024

Uh oh!

csun5285 commented Dec 18, 2024

Uh oh!

github-actions bot commented Dec 18, 2024

sh-checker report

Uh oh!

github-actions bot commented Dec 18, 2024

Uh oh!

doris-robot commented Dec 18, 2024

Uh oh!

doris-robot commented Dec 18, 2024

Uh oh!

doris-robot commented Dec 18, 2024

Uh oh!

doris-robot commented Dec 18, 2024

Uh oh!

zzzxl1993 left a comment

Choose a reason for hiding this comment

Uh oh!

airborne12 left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Dec 18, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

csun5285 commented Nov 25, 2024 •

edited

Loading

`sh-checker report`