[fix](cloud-mow) make cloud_txn_delete_bitmap_cache's expired time more reasonable#40333
Merged
zhannngchen merged 3 commits intoapache:masterfrom Sep 14, 2024
Merged
[fix](cloud-mow) make cloud_txn_delete_bitmap_cache's expired time more reasonable#40333zhannngchen merged 3 commits intoapache:masterfrom
zhannngchen merged 3 commits intoapache:masterfrom
Conversation
|
Thank you for your contribution to Apache Doris. Since 2024-03-18, the Document has been moved to doris-website. |
Contributor
Author
|
run buildall |
Contributor
|
clang-tidy review says "All clean, LGTM! 👍" |
TPC-H: Total hot run time: 38387 ms |
|
TeamCity be ut coverage result: |
TPC-DS: Total hot run time: 188201 ms |
ClickBench: Total hot run time: 31.99 s |
zhannngchen
reviewed
Sep 4, 2024
hust-hhb
changed the title
Contributor
|
clang-tidy review says "All clean, LGTM! 👍" |
Contributor
Author
|
run buildall |
TPC-H: Total hot run time: 38297 ms |
TPC-DS: Total hot run time: 192083 ms |
ClickBench: Total hot run time: 31.27 s |
|
TeamCity be ut coverage result: |
…removing expired one
hust-hhb
force-pushed
the
check-status
branch
from
80a1682 to
2312f3f
Compare
Contributor
Author
|
run buildall |
|
TeamCity be ut coverage result: |
TPC-H: Total hot run time: 42866 ms |
TPC-DS: Total hot run time: 199980 ms |
ClickBench: Total hot run time: 30.94 s |
zhannngchen
approved these changes
Sep 14, 2024
Contributor
|
PR approved by at least one committer and no changes requested. |
github-actions
bot
added
approved
Contributor
|
PR approved by anyone and no changes requested. |
Yukang-Lian
approved these changes
Sep 14, 2024
dataroaring
pushed a commit
that referenced
this pull request
Sep 19, 2024
…re reasonable (#40333) Now cloud_txn_delete_bitmap_cache's expired time is relay rely on txn_timeout_s, however if the cost time of calculating delete bitmap is bigger than txn_timeout_s, updating the publish status of cloud_txn_delete_bitmap_cache will be failed, because this cache may be remove by the cleaning thread. In rountine load, this txn_timeout_s is rely on max_batch_interval session variable, so if someone use small interval to submit rountine load task, cloud_txn_delete_bitmap_cache may be removed when calculating delete bitmap task is going, so we need to set a min expired time on cloud_txn_delete_bitmap_cache to avoid this scenario.
Closed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Now cloud_txn_delete_bitmap_cache's expired time is relay rely on txn_timeout_s, however if the cost time of calculating delete bitmap is bigger than txn_timeout_s, updating the publish status of cloud_txn_delete_bitmap_cache will be failed, because this cache may be remove by the cleaning thread. In rountine load, this txn_timeout_s is rely on max_batch_interval session variable, so if someone use small interval to submit rountine load task, cloud_txn_delete_bitmap_cache may be removed when calculating delete bitmap task is going, so we need to set a min expired time on cloud_txn_delete_bitmap_cache to avoid this scenario.