[RELAY][FRONTEND]Onnx to relay frontend by zhreshold · Pull Request #2302 · apache/tvm

zhreshold · 2018-12-17T23:05:35Z

ONNX -> relay frontend migration.
This is co-authered by @siju-samuel with initial attempt #2245

Based on the fact that relay will not depend on NNVM, all nnvm onnx modules are stripped out and put into relay/frontend
Put possible reusable code into common module.

siju-samuel · 2018-12-18T16:55:33Z

@zhreshold I had some patches on top of this change. you can apply this.
0001-bug-fixes-and-more-testcases-added.patch.zip

tqchen · 2018-12-29T18:08:58Z

@srkreddy1238 @nishi-t @Huyuwei @hlu1 please help review this PR

srkreddy1238

Initial review. Will revisit after the CI passed.

srkreddy1238 · 2018-12-30T13:45:56Z

+                          'dilations': ('dilation', (0, 0)),
+                          'pads': ('padding', (0, 0), revert_caffe2_pad),
+                          'group': ('groups', 1)},
+                      custom_check=dimension_constraint())(inputs, attr, params)


I think we shouldn't pass 3 inputs (if bias available).

will strip to first 2 inputs

srkreddy1238 · 2018-12-30T13:46:21Z

+            },
+            disables=['output_shape'],
+            extras={'use_bias': len(inputs) == 3},
+            custom_check=dimension_constraint())(inputs, attr, params)


Same as above.

srkreddy1238 · 2018-12-30T13:50:53Z

+        return AttrCvt('upsampling')(inputs, attr)
+
+
+class Shape(OnnxOpConverter):


The shape op is a workaround in NNVM. We may need to revisit in relay.
Can be disabled here and handled outside this PR.

will disable for now

nishi-t · 2019-01-02T07:04:22Z

+        'LRN': LRN.get_converter(opset),
+
+        # defs/reduction
+        #'ReduceMax': AttrCvt('max', transforms={'axes': 'axis'}),


Did you forget to delete above comment line?

yes, will clean the comments

zhreshold · 2019-01-04T19:11:35Z

@srkreddy1238 @nishi-t Can you guys help review again?

jroesch

Added a bunch of comments on the PR. Mostly issues with documentation, otherwise if tests pass looks good.

jroesch · 2019-01-06T23:55:29Z

        return default
+
+def get_relay_op(op_name):
+    """Get the callable function from relay based on opname:


Suggested change

"""Get the callable function from relay based on opname:

"""Get the callable function from Relay based on operator name.

jroesch · 2019-01-06T23:55:36Z

+    Parameters
+    ----------
+    op_name : str
+        The relay operator name.


Suggested change

The relay operator name.

The Relay operator name.

jroesch · 2019-01-06T23:56:16Z

+    return out_shapes
+
+def infer_channels(inputs, transpose=False):
+    """A hack for getting 'channels' or 'units' since caffe2 don't provide


Suggested change

"""A hack for getting 'channels' or 'units' since caffe2 don't provide

"""A hack for getting 'channels' or 'units' since caffe2 does not provide

jroesch · 2019-01-06T23:56:36Z

@@ -0,0 +1,1077 @@
+# pylint: disable=invalid-name, import-self, len-as-condition, unused-argument, too-many-lines
+"""ONNX: Open Neural Network Exchange frontend for relay."""


Suggested change

"""ONNX: Open Neural Network Exchange frontend for relay."""

"""ONNX: Open Neural Network Exchange frontend for Relay."""

jroesch · 2019-01-06T23:56:56Z

+    return _impl
+
+def revert_caffe2_pad(pads):
+    """Caffe2 require two times the normal padding."""


Suggested change

"""Caffe2 require two times the normal padding."""

"""Caffe2 requires two times the normal padding."""

jroesch · 2019-01-06T23:59:56Z

+                          inputs,
+                          attrs,
+                          opset):
+        """Convert from onnx operator to relay operator.


Suggested change

"""Convert from onnx operator to relay operator.

"""Convert ONNX operator into a Relay operator.

jroesch · 2019-01-07T00:01:23Z

+def from_onnx(model,
+              shape=None,
+              dtype="float32"):
+    """Convert from ONNX"s model into compatible relay Function.


Suggested change

"""Convert from ONNX"s model into compatible relay Function.

"""Convert a ONNX model into an equivalent Relay function.

jroesch · 2019-01-07T00:01:50Z

+              shape=None,
+              dtype="float32"):
+    """Convert from ONNX"s model into compatible relay Function.
+    Onnx graph is a python protobuf object. The companion parameters will be handled automatically.


Suggested change

Onnx graph is a python protobuf object. The companion parameters will be handled automatically.

ONNX graphs are represented as a Python Protobuf object. The companion parameters will be handled automatically.

jroesch · 2019-01-07T00:03:47Z

+
+This article is an introductory tutorial to deploy ONNX models with Relay.
+
+For us to begin with, onnx module is required to be installed.


Suggested change

For us to begin with, onnx module is required to be installed.

For us to begin with, ONNX package must be installed.

jroesch · 2019-01-07T00:05:25Z

+with relay.build_config(opt_level=1):
+    graph, lib, params = relay.build(sym, target, params=params)
+
+######################################################################


Can we maybe just show how to use the create_executor interface in the tutorial as well? its much simpler then building a graph runtime by hand

@jroesch Can you please advice how to elegantly evaluate executor created with an unordered dict of weights?

exec = relay.build_module.create_executor('graph', mod, tvm.cpu(0), 'llvm') # params is a dict of {'name', tvm.ndarray} tvm_output = exec.evaluate(mode)(x, *params.values()).asnumpy() # wrong order

How do I feed the executor with correct weights without changing the params dict to odict?

@zhreshold I realize the current API does not actually support keyword argument style for parameters.

I'm going to update it with a PR right now, it should work like this:

exec = relay.build_module.create_executor('graph', mod, tvm.cpu(0), 'llvm') tvm_output = exec.evaluate(mode)(x, **params).asnumpy()

Does that seem elegant enough?

Yes, that would perfectly solve my current problem

https://github.com/dmlc/tvm/pull/2402/files

Will update once the PR get merged.

PR is merged.

Co-authored-by: Siju Samuel <siju.samuel@huawei.com>

icemelon · 2019-01-10T01:56:36Z

+    def _impl_v1(cls, inputs, attr, params):
+        # Result of this operator is prominently used by reshape operator.
+        # Just pass the input as it is so that reshape_like can be used there.
+        print("Shape: Differently implemented in relay as a bypass (dummy operator)")


Use logging.warning instead of print.

icemelon · 2019-01-10T02:01:53Z

@srkreddy1238 @jroesch @nishi-t please help review this PR again

Huyuwei · 2019-01-10T18:37:40Z

@zhreshold Should we put the tests under https://github.com/dmlc/tvm/tree/master/tests/python/frontend, and add it to CI https://github.com/dmlc/tvm/blob/master/tests/scripts/task_python_frontend.sh/#L30-L34?

zhreshold · 2019-01-10T19:00:50Z

@Huyuwei Seems like there is tests/python/frontend and tests/python/relay/frontend both including mxnet and keras frontend already, can you clarify the relationship between them?

Huyuwei · 2019-01-10T19:48:23Z

@zhreshold tests/python/frontend is the right place to hold frontend tests, tests/python/relay/frontend is duplicated and should be removed. In fact, tests/python/relay/frontend is not running in the CI
See #2316 (comment)

icemelon · 2019-01-10T20:03:03Z

@Huyuwei I'll remove the duplicate tests in a separate PR.
@zhreshold Let's use tests/python/frontend for frontend tests.

zhreshold · 2019-01-10T20:11:41Z

ok, working on that

icemelon · 2019-01-11T01:29:20Z

@Huyuwei @jroesch If you are both okay with this PR, I'll then merge it.

jroesch

LGTM 👍

zhreshold mentioned this pull request Dec 17, 2018

[RELAY][FRONEND] Frontend scalfolding #2246

Closed

8 tasks

zhreshold force-pushed the onnx-relay branch from fea4c06 to c045be0 Compare December 21, 2018 21:11

icemelon added the status: WIP label Dec 22, 2018

srkreddy1238 requested changes Dec 30, 2018

View reviewed changes

nishi-t reviewed Jan 2, 2019

View reviewed changes

tqchen added status: need review and removed status: WIP labels Jan 3, 2019

jroesch requested changes Jan 7, 2019

View reviewed changes

icemelon added status: need update need update based on feedbacks and removed status: need review labels Jan 7, 2019

tqchen assigned icemelon Jan 8, 2019

zhreshold and others added 16 commits January 9, 2019 15:34

better install deps

7a3b0af

add onnx to relay frontend

8fd0bfc

fix import

6bda072

fix test

5119268

fix common import

8d1afd6

fix common import

7eda7af

fix op getter

0098953

fix _get_relay_op

44a7892

fix _get_relay_op

1dee915

add tutorial onnx frontend

6b6fc84

Co-authored-by: Siju Samuel <siju.samuel@huawei.com>

apply patch

188e9e3

Co-authored-by: Siju Samuel <siju.samuel@huawei.com>

fix pylint

a39f01d

fix syntax

ec3ae74

fix ci

4401e47

wip

71a5fcc

fix unittests

0790c03

zhreshold added 4 commits January 9, 2019 15:37

fix common function get op by name

ae779d1

address comments

0f30d0d

address comments

0003886

update tutorial according to new executor changes

f93b181

zhreshold force-pushed the onnx-relay branch from e6dd207 to f93b181 Compare January 9, 2019 23:54

zhreshold added 2 commits January 9, 2019 16:00

fix lint

0eb78b8

update submodule

2c63c7d

icemelon reviewed Jan 10, 2019

View reviewed changes

use logging.warning

018ae4f

srkreddy1238 approved these changes Jan 10, 2019

View reviewed changes

zhreshold added 2 commits January 10, 2019 13:09

move test to tests/python/frontend/onnx

b1e07c5

fix const dtype

89d279e

Huyuwei approved these changes Jan 11, 2019

View reviewed changes

nishi-t approved these changes Jan 11, 2019

View reviewed changes

icemelon approved these changes Jan 11, 2019

View reviewed changes

icemelon added status: need review and removed status: need update need update based on feedbacks labels Jan 11, 2019

jroesch approved these changes Jan 11, 2019

View reviewed changes

icemelon merged commit 30a5a60 into apache:master Jan 11, 2019

tqchen added status: accepted and removed status: need review labels Jan 18, 2019

wweic pushed a commit to neo-ai/tvm that referenced this pull request Feb 20, 2019

[RELAY][FRONTEND]Onnx to relay frontend (apache#2302)

870872c

wweic pushed a commit to neo-ai/tvm that referenced this pull request Feb 20, 2019

[RELAY][FRONTEND]Onnx to relay frontend (apache#2302)

57a27a2

		return AttrCvt('upsampling')(inputs, attr)


		class Shape(OnnxOpConverter):

	"""Get the callable function from relay based on opname:
	"""Get the callable function from Relay based on operator name.

	"""A hack for getting 'channels' or 'units' since caffe2 don't provide
	"""A hack for getting 'channels' or 'units' since caffe2 does not provide

		@@ -0,0 +1,1077 @@
		# pylint: disable=invalid-name, import-self, len-as-condition, unused-argument, too-many-lines
		"""ONNX: Open Neural Network Exchange frontend for relay."""

	"""Caffe2 require two times the normal padding."""
	"""Caffe2 requires two times the normal padding."""

	"""Convert from onnx operator to relay operator.
	"""Convert ONNX operator into a Relay operator.

	"""Convert from ONNX"s model into compatible relay Function.
	"""Convert a ONNX model into an equivalent Relay function.

	Onnx graph is a python protobuf object. The companion parameters will be handled automatically.
	ONNX graphs are represented as a Python Protobuf object. The companion parameters will be handled automatically.


		This article is an introductory tutorial to deploy ONNX models with Relay.

		For us to begin with, onnx module is required to be installed.

	For us to begin with, onnx module is required to be installed.
	For us to begin with, ONNX package must be installed.

Conversation

zhreshold commented Dec 17, 2018

Uh oh!

siju-samuel commented Dec 18, 2018

Uh oh!

tqchen commented Dec 29, 2018

Uh oh!

srkreddy1238 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

nishi-t Jan 2, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

zhreshold commented Jan 4, 2019

Uh oh!

jroesch left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

icemelon commented Jan 10, 2019

Uh oh!

Huyuwei commented Jan 10, 2019

Uh oh!

zhreshold commented Jan 10, 2019

Uh oh!

Huyuwei commented Jan 10, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

icemelon commented Jan 10, 2019

Uh oh!

zhreshold commented Jan 10, 2019

Uh oh!

icemelon commented Jan 11, 2019

Uh oh!

nishi-t Jan 2, 2019 •

edited

Loading

Huyuwei commented Jan 10, 2019 •

edited

Loading