[PaddlePaddle Hackathon 4][Frontend][Paddle]Add tile/mish/stack/unstack/silu/softshrink/where op for paddle frontend by XG-zheng · Pull Request #14160 · apache/tvm

XG-zheng · 2023-03-01T10:13:24Z

Add tile/mish/stack/unstack/silu/softshrink/where op for paddle frontend

tvm-bot · 2023-03-01T10:13:27Z

Thanks for contributing to TVM! Please refer to the contributing guidelines https://tvm.apache.org/docs/contribute/ for useful information and tips. Please request code reviews from Reviewers by @-ing them in a comment.

cc @heliqi, @jiangjiajun _{See #10317 for details}

_{Generated by tvm-bot}

…p for paddle frontend

jiangjiajun · 2023-03-01T10:44:56Z

+def convert_tile(g, op, block):
+    """Operator converter for tile."""
+
+    x = g.get_node(op.input("X")[0])


There're some cases are not covered, please refer this https://github.com/PaddlePaddle/Paddle2ONNX/blob/develop/paddle2onnx/mapper/tensor/tile.cc#L33 for its implemenation

Also please update the test case

ok, I fixed it.

XG-zheng · 2023-03-01T17:25:52Z

Due to the old version(2.1.3) of paddle in ci-gpu docker, mish and tile may have compatibility problems.

Mish is a new api of paddle 2.3
Tensor type conversion error in layer.

class Tile5(nn.Layer):
    @paddle.jit.to_static
    def forward(self, inputs):
        reps = paddle.to_tensor([3, 2])
        return paddle.tile(inputs, repeat_times=reps)

verify_model(Tile6(), input_data=input_data)
input_shapes = [[10], [2, 3], [5, 10, 11], [3, 4, 5, 6]]
for input_shape in input_shapes:
        input_data = paddle.randn(shape=input_shape, dtype="float32")
        verify_model(Tile5(), input_data=input_data)

The above code will be error in version 2.1: Tensor reps holds the wrong type, it holds int64_t, but desires to be int.

But in version 2.4, the code works well and passes the test cases

reps = paddle.to_tensor([3, 2], dtype='int32')
reps = paddle.cast(reps, "int32")

In paddle 2.1.3, must explicitly cast again to avoid this error.

masahi · 2023-03-01T21:32:41Z

Let me know if you need CI update for paddle cc @jiangjiajun

jiangjiajun · 2023-03-06T03:57:35Z

Let me know if you need CI update for paddle cc @jiangjiajun

@masahi It's better that we upgrade paddlepaddle, could you help to update ci image? Let's talk in this pull request #14206

jiangjiajun · 2023-03-06T03:07:11Z

+    add_threshold = _op.add(x, threshold_right)
+    condition_2 = tvm.relay.less(calc_right, threshold_left)
+    calc_left = tvm.relay.where(condition_2, add_threshold, calc_right)
+    g.add_node(op.output("Out")[0], calc_left)


Can you refer the following code to optimize this implementation

tvm/python/tvm/relay/frontend/onnx.py

Lines 2280 to 2291 in befdc4e

class Shrink(OnnxOpConverter):

"""Operator converter for Shrink."""

@classmethod

def _impl_v9(cls, inputs, attr, params):

x = inputs[0]

dtype = infer_type(x).checked_type.dtype

lambd = _op.const(attr.get("lambd", 0.5), dtype=dtype)

bias = _op.const(attr.get("bias", 0.0), dtype=dtype)

zeros = _op.zeros_like(x)

return _op.where(x < -lambd, x + bias, zeros) + _op.where(x > lambd, x - bias, zeros)

OK，I‘ve done it

jiangjiajun · 2023-03-07T02:37:32Z

+    softplus = _op.where(x > threshold, x, log)
+    tanh = _op.tanh(softplus)
+    out = _op.multiply(x, tanh)
+    g.add_node(op.output("Out")[0], out)


I think we only need to implement this formula, there's no need to process situation that beta * x > threshold, it's just a strategy to make calculation stable.

I refer to the api doc. The softplus in the mish function does not contain the beta parameter. And the attribute of mish(op.attr) only contain threshold.

I modified the implementation and only process the case of x <= threshold (beta = 1.0).

jiangjiajun · 2023-03-08T02:12:48Z

Since the ci image is not updated yet, you can modify the test case to fix mish error

@tvm.testing.uses_gpu
def test_forward_mish():
    class Mish(nn.Layer):
        @paddle.jit.to_static
        def forward(self, inputs):
            return nn.functional.mish(inputs)

    input_shapes = [[10], [2, 3], [5, 10, 11], [3, 4, 5, 6]]
    if paddle.version.full_version >= '2.4.2':
      for input_shape in input_shapes:
          input_data = paddle.randn(shape=input_shape, dtype="float32")
          verify_model(Mish(), input_data=input_data)
          input_data += 20.0
          verify_model(Mish(), input_data=input_data)

      input_data = paddle.to_tensor([-5.0, 0.0, 5.0, 23.1, 20.0])
      verify_model(Mish(), input_data=input_data)

junrushao · 2023-03-08T02:19:53Z

Please feel free to ping me as soon as it’s ready to be merged

jiangjiajun · 2023-03-08T05:22:45Z

Please feel free to ping me as soon as it’s ready to be merged

HI, @junrushao thanks, this PR is approved, I think we can merge now

junrushao

LGTM!

junrushao · 2023-03-08T08:09:35Z

The pipeline “cross-isa-minimal” timeouts too often…re-triggered

[Frontend][Paddle]Add tile/mish/stack/unstack/silu/softshrink/where o…

523e1c5

…p for paddle frontend

jiangjiajun suggested changes Mar 1, 2023

View reviewed changes

XG-zheng added 3 commits March 1, 2023 11:52

fix convert tile and update test case

e3b94e1

fix test case and tile dtype

07cbe38

remove mish and tile

bd6dbf5

XG-zheng mentioned this pull request Mar 2, 2023

【PaddlePaddle Hackathon 第四期】任务总览 PaddlePaddle/Paddle#50629

Closed

fix tensor type error in test case

1293950

XG-zheng mentioned this pull request Mar 3, 2023

【PaddlePaddle Hackathon 4 No.230】为TVM 添加tile、mish、stack、unstack、silu、softshrink、where算子支持 PaddlePaddle/community#402

Merged

jiangjiajun suggested changes Mar 6, 2023

View reviewed changes

XG-zheng added 2 commits March 6, 2023 16:42

optimize convert softshrink

ec00555

add convert mish and test case

278548a

jiangjiajun suggested changes Mar 7, 2023

View reviewed changes

optimize mish implementation

1808117

jiangjiajun approved these changes Mar 7, 2023

View reviewed changes

modify mish test case

613dfa2

jiangjiajun approved these changes Mar 8, 2023

View reviewed changes

junrushao approved these changes Mar 8, 2023

View reviewed changes

masahi merged commit 1453fe0 into apache:main Mar 8, 2023

This was referenced Mar 10, 2023

【PaddlePaddle Hackathon 第四期】任务总览 PaddlePaddle/Paddle#51281

Closed

[PaddlePaddle Hackathon 4][Frontend][Paddle]add thresholded_relu/index_select/eye/linspace/take_alone_axis/dist for paddle frontend #14172

Merged

ysh329 mentioned this pull request Apr 17, 2023

[Release] v0.12.0 Release Candidate Notes #14645

Closed

	class Shrink(OnnxOpConverter):
	"""Operator converter for Shrink."""

	@classmethod
	def _impl_v9(cls, inputs, attr, params):
	x = inputs[0]
	dtype = infer_type(x).checked_type.dtype
	lambd = _op.const(attr.get("lambd", 0.5), dtype=dtype)
	bias = _op.const(attr.get("bias", 0.0), dtype=dtype)

	zeros = _op.zeros_like(x)
	return _op.where(x < -lambd, x + bias, zeros) + _op.where(x > lambd, x - bias, zeros)

Conversation

XG-zheng commented Mar 1, 2023

Uh oh!

tvm-bot commented Mar 1, 2023

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

XG-zheng commented Mar 1, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

masahi commented Mar 1, 2023

Uh oh!

jiangjiajun commented Mar 6, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jiangjiajun commented Mar 8, 2023

Uh oh!

junrushao commented Mar 8, 2023

Uh oh!

jiangjiajun commented Mar 8, 2023

Uh oh!

junrushao left a comment

Choose a reason for hiding this comment

Uh oh!

junrushao commented Mar 8, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

XG-zheng commented Mar 1, 2023 •

edited

Loading

jiangjiajun commented Mar 6, 2023 •

edited

Loading

junrushao commented Mar 8, 2023 •

edited

Loading