[Fix](auto inc) Fix auto inc db id overflow by Yukang-Lian · Pull Request #34912 · apache/doris

Yukang-Lian · 2024-05-15T09:40:55Z

Proposed changes

Issue Number: close #xxx

db_id, table_id and tablet_id should be int64_t instead of int32_t

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

doris-robot · 2024-05-15T09:41:01Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

Yukang-Lian · 2024-05-15T09:41:04Z

run buildall

github-actions · 2024-05-15T09:46:31Z

clang-tidy review says "All clean, LGTM! 👍"

Yukang-Lian · 2024-05-15T09:54:19Z

run buildall

github-actions · 2024-05-15T09:58:50Z

clang-tidy review says "All clean, LGTM! 👍"

github-actions · 2024-05-15T10:00:22Z

clang-tidy review says "All clean, LGTM! 👍"

zhannngchen

LGTM

github-actions · 2024-05-15T10:07:09Z

PR approved by at least one committer and no changes requested.

github-actions · 2024-05-15T10:07:11Z

PR approved by anyone and no changes requested.

doris-robot · 2024-05-15T11:08:46Z

TPC-H: Total hot run time: 41700 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d2deb8a88c66fea841215e7ae1b10a3773ee3ddb, data reload: false

------ Round 1 ----------------------------------
q1	17597	4425	4254	4254
q2	2029	186	191	186
q3	10459	1264	1267	1264
q4	10207	878	894	878
q5	7486	2740	2712	2712
q6	218	127	130	127
q7	1014	621	630	621
q8	9232	2346	2122	2122
q9	9495	6754	6704	6704
q10	9496	3995	3901	3901
q11	433	245	243	243
q12	432	221	245	221
q13	17237	3169	3232	3169
q14	287	236	231	231
q15	522	486	471	471
q16	504	392	376	376
q17	983	708	819	708
q18	8560	8040	7818	7818
q19	5416	1604	1529	1529
q20	656	330	324	324
q21	5316	3558	4227	3558
q22	342	283	292	283
Total cold run time: 117921 ms
Total hot run time: 41700 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4521	4440	4397	4397
q2	381	260	277	260
q3	3144	2902	2889	2889
q4	1919	1636	1638	1636
q5	5540	5582	5539	5539
q6	216	126	122	122
q7	2361	1955	2028	1955
q8	3291	3430	3478	3430
q9	8705	8827	8686	8686
q10	4008	3897	3814	3814
q11	598	511	497	497
q12	793	639	679	639
q13	16918	3182	3210	3182
q14	283	253	251	251
q15	540	490	480	480
q16	468	418	413	413
q17	1818	1463	1496	1463
q18	7730	7570	7502	7502
q19	1677	1604	1529	1529
q20	1999	1783	1759	1759
q21	5192	5000	5021	5000
q22	599	482	482	482
Total cold run time: 72701 ms
Total hot run time: 55925 ms

doris-robot · 2024-05-15T11:18:29Z

TeamCity be ut coverage result:
Function Coverage: 35.71% (8998/25199)
Line Coverage: 27.37% (74436/271935)
Region Coverage: 26.61% (38485/144605)
Branch Coverage: 23.43% (19633/83800)
Coverage Report: http://coverage.selectdb-in.cc/coverage/d2deb8a88c66fea841215e7ae1b10a3773ee3ddb_d2deb8a88c66fea841215e7ae1b10a3773ee3ddb/report/index.html

doris-robot · 2024-05-15T11:20:04Z

TPC-DS: Total hot run time: 187640 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d2deb8a88c66fea841215e7ae1b10a3773ee3ddb, data reload: false

query1	930	398	372	372
query2	6446	2456	2495	2456
query3	6648	213	217	213
query4	22628	21289	21224	21224
query5	4120	428	429	428
query6	266	186	178	178
query7	4578	298	285	285
query8	237	190	188	188
query9	8794	2400	2388	2388
query10	434	260	261	260
query11	14648	14191	14208	14191
query12	140	93	90	90
query13	1645	386	384	384
query14	10708	7119	7538	7119
query15	222	179	170	170
query16	7727	264	252	252
query17	1729	557	561	557
query18	1906	279	261	261
query19	206	155	147	147
query20	97	87	89	87
query21	201	131	129	129
query22	5024	4923	4888	4888
query23	34166	33769	33726	33726
query24	6651	2968	2960	2960
query25	479	369	367	367
query26	699	155	164	155
query27	1877	324	338	324
query28	3571	2058	2044	2044
query29	834	608	599	599
query30	250	180	189	180
query31	922	785	750	750
query32	60	52	64	52
query33	519	253	243	243
query34	847	490	503	490
query35	761	701	679	679
query36	1081	900	923	900
query37	134	72	70	70
query38	2832	2780	2742	2742
query39	1635	1543	1550	1543
query40	201	123	124	123
query41	42	47	40	40
query42	102	97	97	97
query43	577	574	539	539
query44	1067	734	759	734
query45	268	255	251	251
query46	1078	727	728	727
query47	1975	1875	1913	1875
query48	366	302	297	297
query49	768	394	415	394
query50	772	381	397	381
query51	6887	6758	6789	6758
query52	104	93	94	93
query53	357	290	288	288
query54	519	431	443	431
query55	76	76	73	73
query56	241	222	228	222
query57	1244	1143	1188	1143
query58	219	201	209	201
query59	3367	3236	3144	3144
query60	264	248	252	248
query61	127	112	132	112
query62	577	487	477	477
query63	308	288	285	285
query64	7659	7475	7403	7403
query65	3117	3105	3085	3085
query66	798	389	345	345
query67	15714	15323	14988	14988
query68	4521	532	527	527
query69	468	314	312	312
query70	1195	1093	1155	1093
query71	389	276	278	276
query72	7777	2593	2365	2365
query73	703	339	330	330
query74	6558	6175	6228	6175
query75	3362	2690	2610	2610
query76	2217	975	993	975
query77	401	277	272	272
query78	10722	10149	10140	10140
query79	1506	518	515	515
query80	1006	444	432	432
query81	521	245	246	245
query82	957	98	101	98
query83	260	166	166	166
query84	254	87	84	84
query85	924	305	342	305
query86	401	324	328	324
query87	3296	3114	3131	3114
query88	2974	2368	2376	2368
query89	468	378	386	378
query90	1952	191	191	191
query91	129	96	99	96
query92	57	47	49	47
query93	1285	502	496	496
query94	1034	182	187	182
query95	400	301	304	301
query96	585	274	269	269
query97	3162	3002	2986	2986
query98	233	214	222	214
query99	1237	917	909	909
Total cold run time: 266400 ms
Total hot run time: 187640 ms

doris-robot · 2024-05-15T11:25:36Z

ClickBench: Total hot run time: 30.22 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit d2deb8a88c66fea841215e7ae1b10a3773ee3ddb, data reload: false

query1	0.04	0.03	0.04
query2	0.08	0.04	0.04
query3	0.23	0.05	0.05
query4	1.67	0.07	0.10
query5	0.52	0.50	0.52
query6	1.13	0.73	0.72
query7	0.02	0.02	0.01
query8	0.05	0.04	0.04
query9	0.53	0.49	0.49
query10	0.55	0.55	0.54
query11	0.16	0.11	0.11
query12	0.14	0.12	0.12
query13	0.60	0.60	0.62
query14	0.79	0.78	0.77
query15	0.82	0.80	0.82
query16	0.37	0.35	0.37
query17	0.93	0.93	1.04
query18	0.21	0.26	0.23
query19	1.79	1.66	1.64
query20	0.02	0.01	0.01
query21	15.73	0.67	0.67
query22	4.21	7.60	1.73
query23	18.28	1.42	1.32
query24	2.00	0.22	0.21
query25	0.15	0.08	0.08
query26	0.27	0.18	0.17
query27	0.08	0.08	0.08
query28	13.30	1.00	0.99
query29	13.73	3.30	3.24
query30	0.24	0.06	0.05
query31	2.88	0.39	0.38
query32	3.27	0.46	0.48
query33	2.80	2.84	2.87
query34	17.12	4.42	4.46
query35	4.50	4.48	4.59
query36	0.67	0.46	0.46
query37	0.17	0.15	0.15
query38	0.15	0.14	0.14
query39	0.05	0.04	0.03
query40	0.17	0.13	0.14
query41	0.09	0.05	0.05
query42	0.05	0.05	0.05
query43	0.04	0.04	0.04
Total cold run time: 110.6 s
Total hot run time: 30.22 s

Yukang-Lian · 2024-05-15T11:37:12Z

run performance

Yukang-Lian · 2024-05-15T13:03:10Z

run performance

doris-robot · 2024-05-15T13:33:03Z

TPC-DS: Total hot run time: 187965 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d2deb8a88c66fea841215e7ae1b10a3773ee3ddb, data reload: false

query1	923	397	368	368
query2	6448	2389	2252	2252
query3	6662	220	221	220
query4	23183	21203	21143	21143
query5	4201	416	429	416
query6	259	173	169	169
query7	4583	290	286	286
query8	257	187	193	187
query9	8473	2411	2366	2366
query10	435	251	244	244
query11	14773	14225	14132	14132
query12	140	92	90	90
query13	1645	369	365	365
query14	10185	7882	8318	7882
query15	219	172	175	172
query16	7835	280	254	254
query17	1866	555	545	545
query18	1976	274	275	274
query19	204	148	150	148
query20	94	88	81	81
query21	198	125	127	125
query22	5150	4977	4895	4895
query23	34091	33556	33680	33556
query24	11114	2949	2901	2901
query25	616	367	365	365
query26	1306	159	155	155
query27	3034	325	343	325
query28	7607	2043	2041	2041
query29	893	617	589	589
query30	279	178	173	173
query31	981	765	757	757
query32	92	54	55	54
query33	750	256	249	249
query34	1022	490	484	484
query35	803	684	673	673
query36	1100	939	931	931
query37	125	71	70	70
query38	2872	2793	2735	2735
query39	1610	1568	1669	1568
query40	200	131	127	127
query41	48	41	47	41
query42	102	103	97	97
query43	576	540	543	540
query44	1201	728	748	728
query45	271	264	264	264
query46	1082	721	726	721
query47	1934	1883	1896	1883
query48	381	314	300	300
query49	1096	404	409	404
query50	781	390	394	390
query51	6870	6899	6814	6814
query52	109	93	97	93
query53	354	294	292	292
query54	854	442	432	432
query55	77	74	75	74
query56	254	237	232	232
query57	1244	1182	1177	1177
query58	256	205	203	203
query59	3358	3051	3090	3051
query60	273	239	246	239
query61	109	107	108	107
query62	683	470	464	464
query63	315	283	285	283
query64	9991	7548	7446	7446
query65	3169	3125	3122	3122
query66	1383	361	349	349
query67	15536	15050	15512	15050
query68	4959	560	533	533
query69	540	309	324	309
query70	1219	1097	1155	1097
query71	456	278	271	271
query72	7746	2713	2510	2510
query73	725	327	320	320
query74	6507	6217	6173	6173
query75	3509	2720	2641	2641
query76	3143	1017	976	976
query77	658	271	272	271
query78	10728	10240	10040	10040
query79	7796	532	528	528
query80	1337	590	466	466
query81	534	241	250	241
query82	858	102	102	102
query83	203	169	179	169
query84	264	88	86	86
query85	1389	273	275	273
query86	454	286	310	286
query87	3309	3134	3083	3083
query88	5449	2372	2355	2355
query89	482	377	387	377
query90	1962	190	185	185
query91	124	97	98	97
query92	63	52	53	52
query93	6150	516	500	500
query94	1159	180	185	180
query95	401	312	302	302
query96	618	269	270	269
query97	3168	3027	3005	3005
query98	232	220	223	220
query99	1282	885	903	885
Total cold run time: 297935 ms
Total hot run time: 187965 ms

doris-robot · 2024-05-15T13:38:34Z

ClickBench: Total hot run time: 30.56 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit d2deb8a88c66fea841215e7ae1b10a3773ee3ddb, data reload: false

query1	0.04	0.04	0.04
query2	0.08	0.04	0.04
query3	0.23	0.05	0.05
query4	1.66	0.07	0.08
query5	0.49	0.48	0.50
query6	1.12	0.73	0.73
query7	0.02	0.01	0.01
query8	0.05	0.05	0.04
query9	0.53	0.50	0.49
query10	0.55	0.56	0.54
query11	0.16	0.12	0.11
query12	0.14	0.12	0.12
query13	0.59	0.60	0.59
query14	0.76	0.79	0.77
query15	0.83	0.81	0.80
query16	0.38	0.37	0.38
query17	1.02	1.04	1.03
query18	0.20	0.25	0.24
query19	1.88	1.83	1.68
query20	0.02	0.01	0.01
query21	15.52	0.67	0.66
query22	4.39	7.48	1.69
query23	18.29	1.30	1.34
query24	1.31	0.46	0.22
query25	0.14	0.08	0.08
query26	0.28	0.18	0.17
query27	0.08	0.08	0.08
query28	13.26	1.02	1.03
query29	13.21	3.34	3.35
query30	0.24	0.06	0.06
query31	2.87	0.40	0.39
query32	3.26	0.47	0.48
query33	2.83	2.83	2.80
query34	17.36	4.48	4.46
query35	4.51	4.57	4.57
query36	0.65	0.45	0.47
query37	0.17	0.16	0.15
query38	0.15	0.15	0.14
query39	0.05	0.04	0.04
query40	0.16	0.14	0.14
query41	0.09	0.04	0.05
query42	0.05	0.04	0.05
query43	0.05	0.04	0.04
Total cold run time: 109.67 s
Total hot run time: 30.56 s

hello-stephen · 2024-05-15T14:59:45Z

run performance

dataroaring · 2024-05-15T15:00:05Z

run performance

…_t (apache#34912)

…_t (#34912)

Co-authored-by: abmdocrt <Yukang.Lian2022@gmail.com>

…#34962 (apache#34995) Co-authored-by: abmdocrt <Yukang.Lian2022@gmail.com>

dataroaring added dev/2.1.x p0_c p0_b and removed p0_c labels May 15, 2024

db_id and table_id should be int64_t instead of int32_t

d2deb8a

Yukang-Lian force-pushed the Fix_Auto_Inc_Dbid_Overflow branch from 0e185a1 to d2deb8a Compare May 15, 2024 09:53

zhannngchen approved these changes May 15, 2024

View reviewed changes

github-actions Bot added the approved Indicates a PR has been approved by one committer. label May 15, 2024

github-actions Bot added the reviewed label May 15, 2024

yiguolei approved these changes May 15, 2024

View reviewed changes

dataroaring merged commit 6b5778a into apache:master May 15, 2024

eldenmoon pushed a commit to eldenmoon/incubator-doris that referenced this pull request May 17, 2024

[fix](auto inc) db_id and table_id should be int64_t instead of int32…

32ae475

…_t (apache#34912)

eldenmoon mentioned this pull request May 17, 2024

[Pick](Branch-2.0) pick to fix tablet_id overflow #34995

Merged

yiguolei pushed a commit that referenced this pull request May 18, 2024

[fix](auto inc) db_id and table_id should be int64_t instead of int32…

dff6171

…_t (#34912)

xiaokang pushed a commit that referenced this pull request May 20, 2024

[Pick](Branch-2.0) pick to fix tablet_id overflow #34912 #34962 (#34995)

4fa2bde

Co-authored-by: abmdocrt <Yukang.Lian2022@gmail.com>

yiguolei added dev/2.1.4-merged and removed dev/2.1.x labels Jun 5, 2024

dataroaring added the dev/3.0.0-merged label Aug 5, 2024

mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024

[Pick](Branch-2.0) pick to fix tablet_id overflow apache#34912 apache…

fa766b9

…#34962 (apache#34995) Co-authored-by: abmdocrt <Yukang.Lian2022@gmail.com>

Conversation

Yukang-Lian commented May 15, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Proposed changes

Further comments

Uh oh!

doris-robot commented May 15, 2024

Uh oh!

Yukang-Lian commented May 15, 2024

Uh oh!

github-actions Bot commented May 15, 2024

Uh oh!

Yukang-Lian commented May 15, 2024

Uh oh!

github-actions Bot commented May 15, 2024

Uh oh!

github-actions Bot commented May 15, 2024

Uh oh!

zhannngchen left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented May 15, 2024

Uh oh!

github-actions Bot commented May 15, 2024

Uh oh!

doris-robot commented May 15, 2024

Uh oh!

doris-robot commented May 15, 2024

Uh oh!

doris-robot commented May 15, 2024

Uh oh!

doris-robot commented May 15, 2024

Uh oh!

Yukang-Lian commented May 15, 2024

Uh oh!

Yukang-Lian commented May 15, 2024

Uh oh!

doris-robot commented May 15, 2024

Uh oh!

doris-robot commented May 15, 2024

Uh oh!

hello-stephen commented May 15, 2024

Uh oh!

dataroaring commented May 15, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Yukang-Lian commented May 15, 2024 •

edited

Loading