Skip to content

[enhance](partitionid) check partition id to avoid unexpected behavior#28045

Merged
xiaokang merged 9 commits intoapache:branch-2.0from
dataroaring:partition_id_2.0
Dec 6, 2023
Merged

[enhance](partitionid) check partition id to avoid unexpected behavior#28045
xiaokang merged 9 commits intoapache:branch-2.0from
dataroaring:partition_id_2.0

Conversation

@dataroaring
Copy link
Copy Markdown
Contributor

@dataroaring dataroaring commented Dec 6, 2023

Proposed changes

Issue Number: close #xxx

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Dec 6, 2023

clang-tidy review says "All clean, LGTM! 👍"

@dataroaring
Copy link
Copy Markdown
Contributor Author

dataroaring commented Dec 6, 2023

run buildall

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Dec 6, 2023

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link
Copy Markdown

doris-robot commented Dec 6, 2023

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 47.52 seconds
stream load tsv: 564 seconds loaded 74807831229 Bytes, about 126 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 33 seconds loaded 861443392 Bytes, about 24 MB/s
insert into select: 29.7 seconds inserted 10000000 Rows, about 336K ops/s
storage size: 17162941144 Bytes

xiaokang
xiaokang reviewed Dec 6, 2023
View reviewed changes
Comment thread be/src/olap/tablet.cpp
if (tablet_meta->partition_id() <= 0) {
LOG(WARNING) << "invalid partition id " << tablet_meta->partition_id() << ", tablet "
<< tablet_meta->tablet_id();
return nullptr;
Copy link
Copy Markdown
Contributor

@xiaokang xiaokang Dec 6, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will it cause NPE for caller?

Comment thread be/src/olap/txn_manager.cpp
@dataroaring
Copy link
Copy Markdown
Contributor Author

dataroaring commented Dec 6, 2023

run buildall

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Dec 6, 2023

clang-tidy review says "All clean, LGTM! 👍"

1 similar comment
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Dec 6, 2023

clang-tidy review says "All clean, LGTM! 👍"

@dataroaring
Copy link
Copy Markdown
Contributor Author

dataroaring commented Dec 6, 2023

run buildall

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Dec 6, 2023

clang-tidy review says "All clean, LGTM! 👍"

@dataroaring
Copy link
Copy Markdown
Contributor Author

dataroaring commented Dec 6, 2023

run buildall

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Dec 6, 2023

clang-tidy review says "All clean, LGTM! 👍"

@dataroaring
Copy link
Copy Markdown
Contributor Author

dataroaring commented Dec 6, 2023

run buildall

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Dec 6, 2023

clang-tidy review says "All clean, LGTM! 👍"

@dataroaring
Copy link
Copy Markdown
Contributor Author

dataroaring commented Dec 6, 2023

run buildall

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Dec 6, 2023

clang-tidy review says "All clean, LGTM! 👍"

@dataroaring
Copy link
Copy Markdown
Contributor Author

dataroaring commented Dec 6, 2023

run buildall

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Dec 6, 2023

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link
Copy Markdown

doris-robot commented Dec 6, 2023

TeamCity be ut coverage result:
Function Coverage: 37.95% (7982/21035)
Line Coverage: 29.65% (64746/218399)
Region Coverage: 29.07% (33336/114660)
Branch Coverage: 24.93% (17119/68668)
Coverage Report: http://coverage.selectdb-in.cc/coverage/da882e4108c07d82eb0ce98771eaed4778132914_da882e4108c07d82eb0ce98771eaed4778132914/report/index.html

@doris-robot
Copy link
Copy Markdown

doris-robot commented Dec 6, 2023

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 51.43 seconds
stream load tsv: 573 seconds loaded 74807831229 Bytes, about 124 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 29.7 seconds inserted 10000000 Rows, about 336K ops/s
storage size: 17162137104 Bytes

xiaokang
xiaokang approved these changes Dec 6, 2023
View reviewed changes
Copy link
Copy Markdown
Contributor

@xiaokang xiaokang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@xiaokang xiaokang merged commit 26dd70a into apache:branch-2.0 Dec 6, 2023
eldenmoon pushed a commit to eldenmoon/incubator-doris that referenced this pull request Dec 8, 2023
@dataroaring @eldenmoon
[enhance](partitionid) check partition id to avoid unexpected behavior (
c1d0959 
apache#28045)
eldenmoon added a commit that referenced this pull request Dec 8, 2023
@eldenmoon @BePPPower
@jackwener @HappenLee @xinyiZzz @zclllyybb @Jibing-Li @Kikyou1997 @jacktengg @morningman @dataroaring @shuke987 @keanji-x
[pick](branch-2.0) pick prs from branch-2.0 (#28176)
9d5f314 
* [fix](hudi-catalog) fix hudi catalog code (#27966)

backport: #27963

* [fix](Nereids): fix datetime fold-constant-be in Branch-2.0 (#27938)

fix fold-constant-be datetimev2 will lose the scale of datetime.

* [bug](function) fix compound expr coredump problem #27988 (#27989)

* [branch-2.0](cache) Fix partition cache support DATEV2  #27978 (#27990)

* [branch-2.0] delete auto partition docs (#28001)

* [fix](stats) Drop stats or update updated rows after truncate table (#27956)

1. Also clear follower's stats cache when doing drop stats.
2. Drop stats when truncate a table.

backport #27931

* [minor](stats) truncate min/max if too long #27955 (#27997)

* [agg](profile) fix incorrect profile (#28004) (#28035)

* [fix](remote-scanner-pool) missing _remote_thread_pool_max_size value #28057 (#28053)

bp #28057

* [enhance](partitionid) check partition id to avoid unexpected behavior (#28045)

* [opt](resource-tag) root and admin user can use any resource tag by default #28088 (#28080)

bp #28088

* [compile](compile option) add deflate to X86 only (#28089)

* change version to 2.0.3-rc05-var05

* [fix](Nereids): Preserve `""` in single quote strings and `''` in double quote strings. (#27959)

---------

Co-authored-by: Tiewei Fang <43782773+BePPPower@users.noreply.github.com>
Co-authored-by: jakevin <jakevingoo@gmail.com>
Co-authored-by: HappenLee <happenlee@hotmail.com>
Co-authored-by: Xinyi Zou <zouxinyi02@gmail.com>
Co-authored-by: zclllyybb <zhaochangle@selectdb.com>
Co-authored-by: Jibing-Li <64681310+Jibing-Li@users.noreply.github.com>
Co-authored-by: AKIRA <33112463+Kikyou1997@users.noreply.github.com>
Co-authored-by: TengJianPing <18241664+jacktengg@users.noreply.github.com>
Co-authored-by: Mingyu Chen <morningman@163.com>
Co-authored-by: Yongqiang YANG <98214048+dataroaring@users.noreply.github.com>
Co-authored-by: shuke <37901441+shuke987@users.noreply.github.com>
Co-authored-by: 谢健 <jianxie0@gmail.com>
@xiaokang
Copy link
Copy Markdown
Contributor

xiaokang commented Dec 22, 2023

#28055

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@xiaokang xiaokang xiaokang approved these changes

Assignees

No one assigned

Labels

dev/2.0.3-merged

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@dataroaring @doris-robot @xiaokang