Pick some pr to branch 3.0 #41674 #43095 #44164#44290
Merged
morrySnow merged 4 commits intoapache:branch-3.0from Nov 22, 2024
Merged
Pick some pr to branch 3.0 #41674 #43095 #44164#44290morrySnow merged 4 commits intoapache:branch-3.0from
morrySnow merged 4 commits intoapache:branch-3.0from
Conversation
…s other join conjuncts (apache#41674) Support rewrite by materialized view when join has other join conjuncts Such as mv def is select l_orderkey, o_orderdate from lineitem inner join orders on l_orderkey = o_orderkey and l_shipdate <= o_orderdate inner join partsupp on ps_partkey = l_partkey and l_orderkey + o_orderkey != ps_availqty; The query can be rewtritten by mv sucessfully when has not equal conjuncts in join l_shipdate <= o_orderdate` and `ps_partkey = l_partkey and l_orderkey + o_orderkey != ps_availqty; select l_orderkey, o_orderdate from lineitem inner join orders on l_orderkey = o_orderkey and l_shipdate <= o_orderdate inner join partsupp on ps_partkey = l_partkey and l_orderkey + o_orderkey != ps_availqty;
When shrinking the test data set, the corresponding out file data was not refreshed. As a result, the data in the out file was refreshed again. There are some tests that hit both materialized views as expected, modifying such corresponding cases.
…zed view (apache#44164) Problem Summary: When materialized view is rewritten, it would use the mv metadata. Should try to get read lock before use these metadata. or it would cause error. Such as mv def is as following CREATE MATERIALIZED VIEW mv1 BUILD IMMEDIATE REFRESH COMPLETE ON MANUAL DISTRIBUTED BY RANDOM BUCKETS 2 PROPERTIES ('replication_num' = '1') AS select o_orderdate, o_shippriority, o_comment, o.o_code as o_o_code, l_orderkey, l_partkey, l.o_code as l_o_code from orders_same_col o left join lineitem_same_col l on l_orderkey = o_orderkey left join partsupp on ps_partkey = l_partkey and l_suppkey = ps_suppkey; When handling transparent rewriting, a MV scan plan is used for the transparent rewrite. During the initialization of the scan plan, the partitions of the table are retrieved, so it is necessary to attempt to acquire a read lock on the table during initialization. If the read lock is not acquired, subsequent operations may add or delete partitions, and in the later processing of table partitions, calling get Partition may not retrieve the corresponding partition, leading to data errors.
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
Member
Author
|
run buildall |
Member
Author
|
run buildall |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What problem does this PR solve?
6fdc8a5
#41674
7c45912
#43095
7aec6ff
#44164
Related PR: #xxx
Problem Summary:
Release note
None
Check List (For Author)
Test
Behavior changed:
Does this need documentation?
Check List (For Reviewer who merge this PR)