Skip to content

Pick some pr to 30 #44779 #44786 #44857 #45130

Merged
morrySnow merged 4 commits intoapache:branch-3.0from
seawinde:pick_some_pr_to_3_0
Dec 9, 2024
Merged

Pick some pr to 30 #44779 #44786 #44857 #45130
morrySnow merged 4 commits intoapache:branch-3.0from
seawinde:pick_some_pr_to_3_0

Conversation

@seawinde
Copy link
Copy Markdown
Member

@seawinde seawinde commented Dec 6, 2024

What problem does this PR solve?

5ce7604
#44779

d350e3c
#44786

8ea48af
#44857

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@seawinde
[fix](mtmv) Fix rewrite fail by materialized view when filter or join…
df43cdd 
… condition has alias (apache#44779)

### What problem does this PR solve?

Related PR: apache#27922

Problem Summary:
query and mv def are as following,` partsupp.public_col as public_col `
is alias, this would cause rewritting fail by materialized view with
msg, the graph logic between query and view is different.

      select 
      o_custkey, 
      o_orderdate, 
      o_shippriority, 
      o_comment, 
      o_orderkey, 
      orders.public_col as col1, 
      l_orderkey, 
      l_partkey, 
      l_suppkey, 
      lineitem.public_col as col2, 
      ps_partkey, 
      ps_suppkey, 
      partsupp.public_col as col3, 
      partsupp.public_col * 2 as col4, 
      o_orderkey + l_orderkey + ps_partkey * 2, 
      sum(
        o_orderkey + l_orderkey + ps_partkey * 2
      ), 
      count() as count_all 
    from 
      (
        select 
          o_custkey, 
          o_orderdate, 
          o_shippriority, 
          o_comment, 
          o_orderkey, 
          orders.public_col as public_col 
        from 
          orders
      ) orders 
      left join (
        select 
          l_orderkey, 
          l_partkey, 
          l_suppkey, 
          lineitem.public_col as public_col 
        from 
          lineitem 
        where 
          lineitem.public_col is null 
          or lineitem.public_col <> 1
      ) lineitem on l_orderkey = o_orderkey 
      inner join (
        select 
          ps_partkey, 
          ps_suppkey, 
          partsupp.public_col as public_col 
        from 
          partsupp
      ) partsupp on ps_partkey = o_orderkey
    where 
      lineitem.public_col is null 
      or lineitem.public_col <> 1 
      and o_orderkey = 2
    group by 
      1, 
      2, 
      3, 
      4, 
      5, 
      6, 
      7, 
      8, 
      9, 
      10, 
      11, 
      12, 
      13, 
      14;

### Release note

Fix rewrite fail by materialized view when filter or join condition has
alias
@seawinde
[opt](mtmv) Optimize plan generate when create mtmv and use mtmv cach…
fe58833 
…e when collect table of mtmv (apache#44786)

Optimize plan generate when create mtmv and use mtmv cache when collect
table of mtmv
1. Reuse plans when creating materialized views to minimize plan
generation overhead.
2. During recursive base table resolution for MTMVs, prioritize MTMV
cache lookup. Fall back to real-time generation only when cache miss
occurs.
@seawinde
[fix](mtmv) Fix refresh materialized view fail when mv def contains c…
d40c427 
…te (apache#44857)

### What problem does this PR solve?

Related PR: apache#33988

Problem Summary:

if mv def contain cte and is partition mv, such as following:

    CREATE MATERIALIZED VIEW mv_name
        BUILD IMMEDIATE REFRESH COMPLETE ON MANUAL
        PARTITION BY (l_shipdate)
        DISTRIBUTED BY RANDOM BUCKETS 2
        PROPERTIES ('replication_num' = '1') 
        AS
    WITH scan_data_cte as (
        select t1.l_shipdate, t1.L_LINENUMBER, orders.O_CUSTKEY, l_suppkey
        from (select * from lineitem where L_LINENUMBER > 1) t1
        left join orders on t1.L_ORDERKEY = orders.O_ORDERKEY
    )
    SELECT *  FROM scan_data_cte; 

if run refresh cmd as following, this would fail, throw exception `no
partition for this tuple`, this pr fix this

refresh materialized view mv_name partitions(p_20231210_20231211);

### Release note

Fix refresh materialized view fail when mv def contains cte
@doris-robot
Copy link
Copy Markdown

doris-robot commented Dec 6, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@seawinde
Copy link
Copy Markdown
Member Author

seawinde commented Dec 6, 2024

run buildall

@seawinde
Copy link
Copy Markdown
Member Author

seawinde commented Dec 8, 2024

run buildall

@morrySnow morrySnow merged commit e70f7f8 into apache:branch-3.0 Dec 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@seawinde @doris-robot @morrySnow