[microTVM] Add support for AutoTVM by mehrdadh · Pull Request #8715 · apache/tvm

mehrdadh · 2021-08-11T01:40:28Z

This PR fixes broken auto tune for microTVM targets using project api.

* use template crt_config.h for host test runtime; delete src/runtime/crt/host/crt_config.h so that it doesn't diverge from the template * bring template crt_config.h inline with the one actually in use * rename to MAX_STRLEN_DLTYPE * Create a dedicated TVM-side host crt_config.h in src/runtime/micro

* move all zephyr projects to apps/microtvm/zephyr/template_project

* Allows tracker clients to open non-traditional RPC sessions

* Needed for AutoTVM.

mehrdadh · 2021-08-12T23:31:50Z

@tkonolige thanks for you review! I think I answered most comments, the only thing left is the global RPC_SESSION that I have to figure out. PTAL.
Thanks!

leandron

It looks generally good. A few nits below.

mehrdadh · 2021-08-13T17:07:30Z

@leandron thanks for the review. I think I addressed your concerns. PTAL.

tkonolige · 2021-08-13T18:01:12Z

+@register_func
+def destroy_micro_session():
+    global RPC_SESSION
+    if RPC_SESSION is not None:
+        exc_type, exc_value, traceback = RPC_SESSION.__exit__(None, None, None)
+        RPC_SESSION = None
+        if (exc_type, exc_value, traceback) != (None, None, None):
+            exc = exc_type(exc_value)  # See PEP 3109
+            exc.__traceback__ = traceback


Where is this called? I can't seem to find another reference to it in this PR or the codebase.

good catch @tkonolige . so at present, the return value from session_constructor is not captured into RPCEndpoint. This means we cannot unfortunately rely on del hook from RPCEndpoint getting destructed in order to destroy any Python-side resources. I suggest then to avoid blocking this PR, we add a call (and only log exceptions, not throw them--in case the link breaks) to destroy_micro_session after the yield:

try: remote.get_function("tvm.micro.destroy_micro_session")() except TVMError: _LOG.warning("Error destroying remote session", exc_info=1)

another option is to capture the return value as ObjectRef or TVMValue in RPCEndpoint and then either have the return value be a PackedFunc which can be invoked at RPCEndpoint disconnect time (but that sucks due to destructor-may-throw-exception) or an ObjectRef which we del. I prefer the former; but the former kind of is asking for us to ensure Shutdown() gets called before ~RPCEndpoint.

We can do the call for now. A separate PR for handling session lifetimes would be good to have.

@areusch thanks for the suggestion. I added that.

tkonolige

Looks good now. Thanks for all the hard work (and many commits :) )!

areusch

thanks @mehrdadh !

leandron

LGTM. I'm happy for this to be merged once CI is green.

mehrdadh · 2021-08-17T07:41:14Z

@leandron Thanks for the review. Does this need a rebase?

leandron · 2021-08-17T12:40:41Z

@leandron Thanks for the review. Does this need a rebase?

I think it needs at least a force-push to re-trigger CI.

* Initial commit of API server impl. * initial commit of api client * Add TVM-side glue code to use Project API * Change tvm.micro.Session to use Project API * Rework how crt_config.h is used on the host. * use template crt_config.h for host test runtime; delete src/runtime/crt/host/crt_config.h so that it doesn't diverge from the template * bring template crt_config.h inline with the one actually in use * rename to MAX_STRLEN_DLTYPE * Create a dedicated TVM-side host crt_config.h in src/runtime/micro * Modify Transport infrastructure to work with Project API * Add host microTVM API server * Zephyr implementation of microTVM API server * move all zephyr projects to apps/microtvm/zephyr/template_project * consolidate CcompilerAnnotator * Allow model library format with c backend, add test. * Update unit tests * fix incorrect doc * Delete old Zephyr build infrastructure * Delete old build abstractions * Delete old Transport implementations and simplify module * lint * ASF header * address gromero comments * final fixes? * fix is_shutdown * fix user-facing API * fix TempDirectory / operator * Update micro_tflite tutorial * lint * fix test_crt and test_link_params * undo global micro import, hopefully fix fixture * lint * fix more tests * Add session_constructor_args to tracker request() function. * Allows tracker clients to open non-traditional RPC sessions * Generate entry_func symbol in C host codegen. * Needed for AutoTVM. * print MeasureErrorNo enum value in MeasureResult repr * Add microTVM session constructor. * This constructor is to be called from the RPC driver to flash and connect to the RPC server on the microcontroller. * add build_kwargs as a Builder constructor arg. * build_kwargs is derived from pre-configured args, the runner, and now from the script. * user-supplied build kwargs override the other two, and a warning is printed if any key is overridden. * Add do_fork option to Builder, to support stateful builders * When AutoTVM builder forks, any global state modified by the build_func is lost between builds * Checkin module_loader used to build and flash microTVM for autotuning. * Import micro into top-level when enabled. * AutoTVM RPC server needs to load the micro session constructor. * Add tvm.contrib.random.random_fill to microTVM. * Allows autotuning with random data. * Move compilation to runner :O * Add a tutorial for AutoTVM with microcontrollers. * Fix si_prefix in autotuner callback * black format and git-clang-format * Switch tutorial back to qemu version * improve error reporting so CI will show test error * black format * autotvm is working * fix tutorial * fix dependencies * fix auto tune issue * lint * address comments * fix lint * test crt and zephyr added * fix func registery size * moved autotune test and fixed * fix crt test * address comments * change relay text * change relay in text_zephyr * class added * changed relay module in tutorial and cleanup * address comments * address TK comments * change fork * final comments * retrigger due to flahy test * fix tutorial * retrigger * fix changes due to merge Co-authored-by: Andrew Reusch <areusch@octoml.ai>

Andrew Reusch added 30 commits August 3, 2021 17:08

Initial commit of API server impl.

a9a67ef

initial commit of api client

14678ab

Add TVM-side glue code to use Project API

98f3468

Change tvm.micro.Session to use Project API

687d4d5

Modify Transport infrastructure to work with Project API

f3fb2c0

Add host microTVM API server

0ffb0c3

Zephyr implementation of microTVM API server

cffa8f9

* move all zephyr projects to apps/microtvm/zephyr/template_project

consolidate CcompilerAnnotator

26e78ca

Allow model library format with c backend, add test.

3d97f6a

Update unit tests

142bc0e

fix incorrect doc

8678346

Delete old Zephyr build infrastructure

ef0d331

Delete old build abstractions

b739908

Delete old Transport implementations and simplify module

cab83a9

lint

1d09e96

ASF header

9d79ef7

address gromero comments

367525c

final fixes?

2da7b1b

fix is_shutdown

c4efe81

fix user-facing API

b340703

fix TempDirectory / operator

adcfe4b

Update micro_tflite tutorial

226082d

lint

1780f91

fix test_crt and test_link_params

8e39635

undo global micro import, hopefully fix fixture

0ea7bd7

lint

ac61a56

fix more tests

9e32c8e

Add session_constructor_args to tracker request() function.

8de0606

* Allows tracker clients to open non-traditional RPC sessions

Generate entry_func symbol in C host codegen.

7853a4f

* Needed for AutoTVM.

leandron reviewed Aug 13, 2021

View reviewed changes

Comment thread python/tvm/micro/build.py Outdated

Comment thread tutorials/micro/micro_autotune.py Outdated

Comment thread tutorials/micro/micro_autotune.py Outdated

Comment thread python/tvm/autotvm/measure/measure_methods.py Outdated

address comments

69bf13f

tkonolige requested changes Aug 13, 2021

View reviewed changes

mehrdadh added 3 commits August 13, 2021 13:59

address TK comments

72ae64f

change fork

539e1bc

final comments

818f822

tkonolige approved these changes Aug 13, 2021

View reviewed changes

Merge branch 'main' into microtvm-project-autotune

9ccbbb2

areusch approved these changes Aug 13, 2021

View reviewed changes

retrigger due to flahy test

b22dd00

leandron approved these changes Aug 16, 2021

View reviewed changes

mehrdadh added 6 commits August 24, 2021 01:03

Merge branch 'main' into microtvm-project-autotune

38ba16b

Merge branch 'main' into microtvm-project-autotune

86768ff

Merge branch 'main' into microtvm-project-autotune

274e482

Merge branch 'main' into microtvm-project-autotune

0205908

fix tutorial

6889ea1

retrigger

c14d7b4

mehrdadh requested a review from manupak as a code owner September 8, 2021 13:37

mehrdadh added 3 commits September 8, 2021 08:47

merge fix

b0b44f6

fix changes due to merge

ff49394

merge conflict fix

38509fa

masahi merged commit aa2b37d into apache:main Sep 9, 2021

mehrdadh deleted the microtvm-project-autotune branch September 9, 2021 14:46

junrushao mentioned this pull request Nov 1, 2021

Apache TVM v0.8 Release Note Candidate #9416

Closed

Conversation

mehrdadh commented Aug 11, 2021

Uh oh!

mehrdadh commented Aug 12, 2021

Uh oh!

leandron left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mehrdadh commented Aug 13, 2021

Uh oh!

Uh oh!

Uh oh!

Uh oh!

tkonolige Aug 13, 2021

Choose a reason for hiding this comment

Uh oh!

areusch Aug 13, 2021

Choose a reason for hiding this comment

Uh oh!

areusch Aug 13, 2021

Choose a reason for hiding this comment

Uh oh!

tkonolige Aug 13, 2021

Choose a reason for hiding this comment

Uh oh!

mehrdadh Aug 13, 2021

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

tkonolige left a comment

Choose a reason for hiding this comment

Uh oh!

areusch left a comment

Choose a reason for hiding this comment

Uh oh!

leandron left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mehrdadh commented Aug 17, 2021

Uh oh!

leandron commented Aug 17, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

leandron left a comment •

edited

Loading