Implements ``dpctl.tensor.logsumexp`` and ``dpctl.tensor.reduce_hypot`` by ndgrigorian · Pull Request #1446 · IntelPython/dpctl

ndgrigorian · 2023-10-19T00:46:18Z

This pull request proposes the additions of logsumexp and reduce_hypot into dpctl.

reduce_hypot can be used to find the Euclidean distance in N-dimensional space

In [1]: import dpctl.tensor as dpt, numpy as np

In [2]: x1 = dpt.full(10, 3, dtype="f4")

In [3]: x2 = dpt.ones(10, dtype="f4")

In [4]: d = dpt.reduce_hypot(x1 - x2)

In [5]: d
Out[5]: usm_ndarray(6.324554, dtype=float32)

logsumexp can be used to sum probabilities stored as the logarithm of the probability.

This pull request also slips in a change to disable the use of sycl::atomic_ref for arithmetic reductions (sum and prod) of inexact data types due to concerns with numerical stability. To better support this change, dedicated kernels for tree reductions over contiguous inputs have been implemented.

Have you provided a meaningful PR description?
Have you added a test, reproducer or referred to an issue with a reproducer?
Have you tested your changes locally for CPU and GPU devices?
Have you made sure that new changes do not introduce compiler warnings?
Have you checked performance impact of proposed changes?
If this PR is a work in progress, are you opening the PR as a draft?

coveralls · 2023-10-19T01:33:55Z

coverage: 85.748% (+0.01%) from 85.735% when pulling d88e78f on logsumexp-reduce-hypot-impl into 2eba93e on master.

github-actions · 2023-10-19T01:36:42Z

View rendered docs @ https://intelpython.github.io/dpctl/pulls/1446/index.html

github-actions · 2023-10-19T02:33:29Z

Array API standard conformance tests for dpctl=0.15.1dev0=py310ha25a700_48 ran successfully.
Passed: 934
Failed: 66
Skipped: 119

github-actions · 2023-10-20T03:34:14Z

Array API standard conformance tests for dpctl=0.15.1dev0=py310ha25a700_50 ran successfully.
Passed: 934
Failed: 66
Skipped: 119

github-actions · 2023-10-24T05:07:47Z

Array API standard conformance tests for dpctl=0.15.1dev0=py310ha25a700_69 ran successfully.
Passed: 935
Failed: 65
Skipped: 119

github-actions · 2023-10-24T06:58:13Z

Array API standard conformance tests for dpctl=0.15.1dev0=py310ha25a700_69 ran successfully.
Passed: 933
Failed: 67
Skipped: 119

github-actions · 2023-10-24T18:51:52Z

Array API standard conformance tests for dpctl=0.15.1dev0=py310ha25a700_70 ran successfully.
Passed: 934
Failed: 66
Skipped: 119

github-actions · 2023-10-25T02:38:24Z

Array API standard conformance tests for dpctl=0.15.1dev0=py310ha25a700_71 ran successfully.
Passed: 934
Failed: 66
Skipped: 119

github-actions · 2023-10-25T02:43:40Z

Array API standard conformance tests for dpctl=0.15.1dev0=py310ha25a700_71 ran successfully.
Passed: 934
Failed: 66
Skipped: 119

ndgrigorian · 2023-10-25T04:35:31Z

@oleksandr-pavlyk
I've added the per-type per-function atomic support checks we've discussed.

github-actions · 2023-10-25T06:43:32Z

Array API standard conformance tests for dpctl=0.15.1dev0=py310ha25a700_72 ran successfully.
Passed: 935
Failed: 65
Skipped: 119

… contiguous arrays

This change was made to improve the accuracy of these functions

This change is intended to improve the numerical stability of sum and prod

…nction

These variants were using ``sycl::maximum`` rather than ``sycl::minimum``

Use lambdas to ignore atomic-specific arguments to hypot and logsumexp dtype_supported functions

For add/multiplies reductions, use tree reduction for FP types, real and complex, to get better round-off accumulation properties.

Reduces code repetition between logsumexp and logaddexp

github-actions · 2023-10-25T23:22:40Z

Array API standard conformance tests for dpctl=0.15.1dev0=py310ha25a700_77 ran successfully.
Passed: 935
Failed: 65
Skipped: 119

oleksandr-pavlyk

LGTM! Thank you @ndgrigorian

ndgrigorian force-pushed the logsumexp-reduce-hypot-impl branch from c8683e3 to e95ac44 Compare October 24, 2023 05:16

ndgrigorian requested a review from oleksandr-pavlyk October 25, 2023 01:22

ndgrigorian force-pushed the logsumexp-reduce-hypot-impl branch from 5e3a671 to fff36a1 Compare October 25, 2023 01:23

ndgrigorian and others added 14 commits October 25, 2023 14:15

Implements logsumexp and reduce_hypot

c2d04a1

Implements dedicated kernels for temp reductions over axes 1 and 0 in…

7b4a835

… contiguous arrays

logsumexp and reduce_hypot no longer use atomics

aa749b4

This change was made to improve the accuracy of these functions

Adds tests for reduce_hypot and logsumexp

6b07501

Arithmetic reductions no longer use atomics for inexact types

1247e9b

This change is intended to improve the numerical stability of sum and prod

Removed support of atomic reduction for max and min

9d99b71

Adds new tests for reductions

10ede9a

Split reductions into multiple source files

7d243d8

Remove unneccessary imports of reduction init functions

4638025

Added functions for querying reduction atomic support per type per fu…

f2babe8

…nction

Corrected min contig variant typo

4e2789d

These variants were using ``sycl::maximum`` rather than ``sycl::minimum``

Removes _tree_reduction_over_axis

4dd054f

Use lambdas to ignore atomic-specific arguments to hypot and logsumexp dtype_supported functions

Always use atomic implementation for min/max if available

448a7f1

For add/multiplies reductions, use tree reduction for FP types, real and complex, to get better round-off accumulation properties.

logaddexp implementation moved to math_utils

d88e78f

Reduces code repetition between logsumexp and logaddexp

ndgrigorian force-pushed the logsumexp-reduce-hypot-impl branch from 599fadb to d88e78f Compare October 25, 2023 21:25

oleksandr-pavlyk approved these changes Oct 26, 2023

View reviewed changes

ndgrigorian merged commit 03fd737 into master Oct 26, 2023

oleksandr-pavlyk deleted the logsumexp-reduce-hypot-impl branch October 27, 2023 02:24

antonwolfy mentioned this pull request Oct 30, 2023

Sensible performance degradation in dpt.tensor.sum #1461

Closed

ndgrigorian mentioned this pull request Nov 4, 2023

Implements statistical functions mean, std, var #1465

Merged

6 tasks

ndgrigorian added a commit that referenced this pull request Dec 1, 2023

Removes reduction files made redundant by gh-1446

4060e1a

ndgrigorian added a commit that referenced this pull request Dec 4, 2023

Removes reduction files made redundant by gh-1446 (#1480)

07dea79

oleksandr-pavlyk mentioned this pull request Jan 26, 2024

Populated changelog of new features since 0.15.0 release #1510

Merged

6 tasks

Conversation

ndgrigorian commented Oct 19, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

coveralls commented Oct 19, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Oct 19, 2023

Uh oh!

github-actions bot commented Oct 19, 2023

Uh oh!

github-actions bot commented Oct 20, 2023

Uh oh!

github-actions bot commented Oct 24, 2023

Uh oh!

github-actions bot commented Oct 24, 2023

Uh oh!

github-actions bot commented Oct 24, 2023

Uh oh!

github-actions bot commented Oct 25, 2023

Uh oh!

github-actions bot commented Oct 25, 2023

Uh oh!

ndgrigorian commented Oct 25, 2023

Uh oh!

github-actions bot commented Oct 25, 2023

Uh oh!

github-actions bot commented Oct 25, 2023

Uh oh!

oleksandr-pavlyk left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

ndgrigorian commented Oct 19, 2023 •

edited

Loading

coveralls commented Oct 19, 2023 •

edited

Loading