[feat](skew & kurt) New aggregate function skew & kurt #40945 by zhiqiang-hhhh · Pull Request #41277 · apache/doris

zhiqiang-hhhh · 2024-09-25T07:34:50Z

cherry pick from #40945

zhiqiang-hhhh · 2024-09-25T07:34:52Z

run buildall

doris-robot · 2024-09-25T07:34:56Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

github-actions

clang-tidy made some suggestions

github-actions · 2024-09-25T07:41:46Z

+
+#pragma once
+
+#include <stddef.h>


warning: inclusion of deprecated C++ header 'stddef.h'; consider using 'cstddef' instead [modernize-deprecated-headers]

Suggested change

#include <stddef.h>

#include <cstddef>

github-actions · 2024-09-25T07:41:46Z

+        ++m[0];
+        m[1] += x;
+        m[2] += x * x;
+        if constexpr (_level >= 3) m[3] += x * x * x;


warning: statement should be inside braces [readability-braces-around-statements]

Suggested change

if constexpr (_level >= 3) m[3] += x * x * x;

if constexpr (_level >= 3) { m[3] += x * x * x;

}

github-actions · 2024-09-25T07:41:46Z

+        m[1] += x;
+        m[2] += x * x;
+        if constexpr (_level >= 3) m[3] += x * x * x;
+        if constexpr (_level >= 4) m[4] += x * x * x * x;


warning: statement should be inside braces [readability-braces-around-statements]

Suggested change

if constexpr (_level >= 4) m[4] += x * x * x * x;

if constexpr (_level >= 4) { m[4] += x * x * x * x;

}

github-actions · 2024-09-25T07:41:46Z

+        m[0] += rhs.m[0];
+        m[1] += rhs.m[1];
+        m[2] += rhs.m[2];
+        if constexpr (_level >= 3) m[3] += rhs.m[3];


warning: statement should be inside braces [readability-braces-around-statements]

Suggested change

if constexpr (_level >= 3) m[3] += rhs.m[3];

if constexpr (_level >= 3) { m[3] += rhs.m[3];

}

github-actions · 2024-09-25T07:41:46Z

+        m[1] += rhs.m[1];
+        m[2] += rhs.m[2];
+        if constexpr (_level >= 3) m[3] += rhs.m[3];
+        if constexpr (_level >= 4) m[4] += rhs.m[4];


warning: statement should be inside braces [readability-braces-around-statements]

Suggested change

if constexpr (_level >= 4) m[4] += rhs.m[4];

if constexpr (_level >= 4) { m[4] += rhs.m[4];

}

github-actions · 2024-09-25T07:41:47Z

+                    ErrorCode::INTERNAL_ERROR,
+                    "Variation moments should be obtained by 'get_population' method");
+        } else {
+            if (m[0] == 0) return std::numeric_limits<T>::quiet_NaN();


warning: statement should be inside braces [readability-braces-around-statements]

Suggested change

if (m[0] == 0) return std::numeric_limits<T>::quiet_NaN();

if (m[0] == 0) { return std::numeric_limits<T>::quiet_NaN();

}

github-actions · 2024-09-25T07:41:47Z

+        } else {
+            if (m[0] == 0) return std::numeric_limits<T>::quiet_NaN();
+            // to avoid accuracy problem
+            if (m[0] == 1) return 0;


warning: statement should be inside braces [readability-braces-around-statements]

Suggested change

if (m[0] == 1) return 0;

if (m[0] == 1) { return 0;

}

github-actions · 2024-09-25T07:41:47Z

+                    ErrorCode::INTERNAL_ERROR,
+                    "Variation moments should be obtained by 'get_population' method");
+        } else {
+            if (m[0] == 0) return std::numeric_limits<T>::quiet_NaN();


warning: statement should be inside braces [readability-braces-around-statements]

Suggested change

if (m[0] == 0) return std::numeric_limits<T>::quiet_NaN();

if (m[0] == 0) { return std::numeric_limits<T>::quiet_NaN();

}

github-actions · 2024-09-25T07:41:47Z

+        } else {
+            if (m[0] == 0) return std::numeric_limits<T>::quiet_NaN();
+            // to avoid accuracy problem
+            if (m[0] == 1) return 0;


warning: statement should be inside braces [readability-braces-around-statements]

Suggested change

if (m[0] == 1) return 0;

if (m[0] == 1) { return 0;

}

github-actions · 2024-09-25T07:41:47Z

+        return;
+    }


warning: redundant return statement at the end of a function with a void return type [readability-redundant-control-flow]

Suggested change

return;

}

}

zhiqiang-hhhh · 2024-09-25T11:58:28Z

run buildall

zhiqiang-hhhh · 2024-09-26T15:41:07Z

run buildall

zhiqiang-hhhh · 2024-09-27T02:09:42Z

run buildall

`skew`,`skew_pop` and `skewness` is used to calculate [skewness](https://en.wikipedia.org/wiki/Skewness#Pearson.27s_moment_coefficient_of_skewness) of a data distribution. `kurt`,`kurt_pop` and `kurtosis` is used to calculate [kurtosis](https://en.wikipedia.org/wiki/Kurtosis) of a data distribution. The implementation references ClickHouse/ClickHouse#5200, and modified result type to AlwaysNullable since doris do not support NaN. The formula used to calculate skew is `3-th moments / (variance^{1.5})` The formula used to calculate kurt is `4-th moments / (variance^{2}) - 3` when value of any result is NaN, doris will return NULL. doc: apache/doris-website#1127

zhiqiang-hhhh · 2024-09-27T12:03:45Z

run buildall

github-actions bot reviewed Sep 25, 2024

View reviewed changes

zhiqiang-hhhh force-pushed the pick_40945_to_upstream_branch-3.0 branch from 8c81afd to ab3e14b Compare September 25, 2024 11:58

zhiqiang-hhhh force-pushed the pick_40945_to_upstream_branch-3.0 branch from ab3e14b to 89ca88b Compare September 26, 2024 15:40

zhiqiang-hhhh force-pushed the pick_40945_to_upstream_branch-3.0 branch from 4df774f to 696ad91 Compare September 27, 2024 12:03

dataroaring merged commit da3ab02 into apache:branch-3.0 Sep 28, 2024

gavinchou mentioned this pull request Oct 13, 2024

Release Note 3.0.2 #41558

Closed

	if constexpr (_level >= 3) m[3] += x * x * x;
	if constexpr (_level >= 3) { m[3] += x * x * x;
	}

	if constexpr (_level >= 4) m[4] += x * x * x * x;
	if constexpr (_level >= 4) { m[4] += x * x * x * x;
	}

	if constexpr (_level >= 3) m[3] += rhs.m[3];
	if constexpr (_level >= 3) { m[3] += rhs.m[3];
	}

	if constexpr (_level >= 4) m[4] += rhs.m[4];
	if constexpr (_level >= 4) { m[4] += rhs.m[4];
	}

	if (m[0] == 0) return std::numeric_limits<T>::quiet_NaN();
	if (m[0] == 0) { return std::numeric_limits<T>::quiet_NaN();
	}

Conversation

zhiqiang-hhhh commented Sep 25, 2024

Uh oh!

zhiqiang-hhhh commented Sep 25, 2024

Uh oh!

doris-robot commented Sep 25, 2024

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot Sep 25, 2024

Choose a reason for hiding this comment

Uh oh!

github-actions bot Sep 25, 2024

Choose a reason for hiding this comment

Uh oh!

github-actions bot Sep 25, 2024

Choose a reason for hiding this comment

Uh oh!

github-actions bot Sep 25, 2024

Choose a reason for hiding this comment

Uh oh!

github-actions bot Sep 25, 2024

Choose a reason for hiding this comment

Uh oh!

github-actions bot Sep 25, 2024

Choose a reason for hiding this comment

Uh oh!

github-actions bot Sep 25, 2024

Choose a reason for hiding this comment

Uh oh!

github-actions bot Sep 25, 2024

Choose a reason for hiding this comment

Uh oh!

github-actions bot Sep 25, 2024

Choose a reason for hiding this comment

Uh oh!

github-actions bot Sep 25, 2024

Choose a reason for hiding this comment

Uh oh!

zhiqiang-hhhh commented Sep 25, 2024

Uh oh!

zhiqiang-hhhh commented Sep 26, 2024

Uh oh!

zhiqiang-hhhh commented Sep 27, 2024

Uh oh!

zhiqiang-hhhh commented Sep 27, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants