Skip to content

[1/3][AOT][DeviceAPI] Connecting devices structure to relevant operators#9395

Merged
areusch merged 2 commits intoapache:mainfrom
Mousius:device-api-wiring
Nov 12, 2021
Merged

[1/3][AOT][DeviceAPI] Connecting devices structure to relevant operators#9395
areusch merged 2 commits intoapache:mainfrom
Mousius:device-api-wiring

Conversation

@Mousius
Copy link
Copy Markdown
Member

@Mousius Mousius commented Oct 29, 2021

This patch adds support for passing the device context via the unpacked API in AOT, generating an additional struct if necessary:

/*!
 * \brief Device context pointers for TVM module "default"
 */
struct tvmgen_default_devices {
  void* npu;
};

Which is then added as an argument to the entry function:

/*!
 * \brief entrypoint function for TVM module "default"
 * \param inputs Input tensors for the module
 * \param outputs Output tensors for the module
 * \param devices Device context pointers for the module
 */
int32_t tvmgen_default_run(
  struct tvmgen_default_inputs* inputs,
  struct tvmgen_default_outputs* outputs,
  struct tvmgen_default_devices* devices
);

I've temporarily added the collection of external code generators to the TE compiler pending proper annotation of the eventual functions.

Co-authored-by: Grant Watson grant.watson@arm.com

@Mousius
Copy link
Copy Markdown
Member Author

Mousius commented Oct 29, 2021

@areusch I've done the bare minimum to wire this through here, and want to follow up later with the cpacked variant that was actually working pretty well from just variables rather than adding the resource_handle attribute.

@Mousius Mousius force-pushed the device-api-wiring branch 3 times, most recently from 4198fb6 to 9959a13 Compare October 29, 2021 11:09
@Mousius Mousius requested a review from icemelon as a code owner November 8, 2021 10:59
areusch
areusch reviewed Nov 9, 2021
View reviewed changes
Copy link
Copy Markdown
Contributor

@areusch areusch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Mousius thanks for this. just a couple comments here

Comment thread apps/microtvm/ethosu/src/tvm_ethosu_runtime.c
Comment thread src/target/target_kind.cc Outdated
Comment thread tests/cpp/target/source/interface_c_test.cc
@Mousius Mousius force-pushed the device-api-wiring branch 2 times, most recently from 45f05d6 to 442ab30 Compare November 9, 2021 18:30
mbs-octoml
mbs-octoml suggested changes Nov 10, 2021
View reviewed changes
Copy link
Copy Markdown
Contributor

@mbs-octoml mbs-octoml left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A personal appeal from Mark not to add more to te_compiler.cc

:-)

Comment thread src/relay/backend/aot_executor_codegen.cc
@mbs-octoml
Copy link
Copy Markdown
Contributor

mbs-octoml commented Nov 11, 2021

LGTM from me.

@Mousius Mousius force-pushed the device-api-wiring branch 2 times, most recently from 78bffcc to 21c2832 Compare November 11, 2021 23:26
@Mousius Mousius mentioned this pull request Nov 12, 2021
Merged
Mousius and others added 2 commits November 12, 2021 11:12
@Mousius @grant-arm
[AOT][DeviceAPI] Connecting devices structure to relevant operators
0a52385 
This patch adds support for passing the device context via the unpacked API in AOT, generating an additional struct if necessary:

```c
/*!
 * \brief Device context pointers for TVM module "default"
 */
struct tvmgen_default_devices {
  void* npu;
};
```

Which is then added as an argument to the entry function:
```c
/*!
 * \brief entrypoint function for TVM module "default"
 * \param inputs Input tensors for the module
 * \param outputs Output tensors for the module
 * \param devices Device context pointers for the module
 */
int32_t tvmgen_default_run(
  struct tvmgen_default_inputs* inputs,
  struct tvmgen_default_outputs* outputs,
  struct tvmgen_default_devices* devices
);
```

I've temporarily added the collection of external code generators to the TE compiler pending proper annotation of the eventual functions.

Co-authored-by: Grant Watson <grant.watson@arm.com>
@Mousius
Correct "use_device_api" attribute name on Target
4cea8a4
@Mousius Mousius mentioned this pull request Nov 12, 2021
Merged
@Mousius Mousius changed the title [AOT][DeviceAPI] Connecting devices structure to relevant operators [1/3][AOT][DeviceAPI] Connecting devices structure to relevant operators Nov 12, 2021
@areusch areusch merged commit 67714c6 into apache:main Nov 12, 2021
@areusch
Copy link
Copy Markdown
Contributor

areusch commented Nov 12, 2021

thanks @Mousius @mbs-octoml the PR is now merged!

@Mousius Mousius deleted the device-api-wiring branch November 12, 2021 20:43
@Mousius Mousius mentioned this pull request Nov 16, 2021
Closed
mehrdadh pushed a commit to mehrdadh/tvm that referenced this pull request Dec 1, 2021
@Mousius @grant-arm @mehrdadh
[1/3][AOT][DeviceAPI] Connecting devices structure to relevant operat…
2f0c742 
…ors (apache#9395)

* [AOT][DeviceAPI] Connecting devices structure to relevant operators

This patch adds support for passing the device context via the unpacked API in AOT, generating an additional struct if necessary:

```c
/*!
 * \brief Device context pointers for TVM module "default"
 */
struct tvmgen_default_devices {
  void* npu;
};
```

Which is then added as an argument to the entry function:
```c
/*!
 * \brief entrypoint function for TVM module "default"
 * \param inputs Input tensors for the module
 * \param outputs Output tensors for the module
 * \param devices Device context pointers for the module
 */
int32_t tvmgen_default_run(
  struct tvmgen_default_inputs* inputs,
  struct tvmgen_default_outputs* outputs,
  struct tvmgen_default_devices* devices
);
```

I've temporarily added the collection of external code generators to the TE compiler pending proper annotation of the eventual functions.

Co-authored-by: Grant Watson <grant.watson@arm.com>

* Correct "use_device_api" attribute name on Target

Co-authored-by: Grant Watson <grant.watson@arm.com>
mehrdadh pushed a commit to mehrdadh/tvm that referenced this pull request Dec 1, 2021
@Mousius @grant-arm @mehrdadh
[1/3][AOT][DeviceAPI] Connecting devices structure to relevant operat…
93e8a13 
…ors (apache#9395)

* [AOT][DeviceAPI] Connecting devices structure to relevant operators

This patch adds support for passing the device context via the unpacked API in AOT, generating an additional struct if necessary:

```c
/*!
 * \brief Device context pointers for TVM module "default"
 */
struct tvmgen_default_devices {
  void* npu;
};
```

Which is then added as an argument to the entry function:
```c
/*!
 * \brief entrypoint function for TVM module "default"
 * \param inputs Input tensors for the module
 * \param outputs Output tensors for the module
 * \param devices Device context pointers for the module
 */
int32_t tvmgen_default_run(
  struct tvmgen_default_inputs* inputs,
  struct tvmgen_default_outputs* outputs,
  struct tvmgen_default_devices* devices
);
```

I've temporarily added the collection of external code generators to the TE compiler pending proper annotation of the eventual functions.

Co-authored-by: Grant Watson <grant.watson@arm.com>

* Correct "use_device_api" attribute name on Target

Co-authored-by: Grant Watson <grant.watson@arm.com>
ylc pushed a commit to ylc/tvm that referenced this pull request Jan 7, 2022
@Mousius @grant-arm @ylc
[1/3][AOT][DeviceAPI] Connecting devices structure to relevant operat…
39fdac3 
…ors (apache#9395)

* [AOT][DeviceAPI] Connecting devices structure to relevant operators

This patch adds support for passing the device context via the unpacked API in AOT, generating an additional struct if necessary:

```c
/*!
 * \brief Device context pointers for TVM module "default"
 */
struct tvmgen_default_devices {
  void* npu;
};
```

Which is then added as an argument to the entry function:
```c
/*!
 * \brief entrypoint function for TVM module "default"
 * \param inputs Input tensors for the module
 * \param outputs Output tensors for the module
 * \param devices Device context pointers for the module
 */
int32_t tvmgen_default_run(
  struct tvmgen_default_inputs* inputs,
  struct tvmgen_default_outputs* outputs,
  struct tvmgen_default_devices* devices
);
```

I've temporarily added the collection of external code generators to the TE compiler pending proper annotation of the eventual functions.

Co-authored-by: Grant Watson <grant.watson@arm.com>

* Correct "use_device_api" attribute name on Target

Co-authored-by: Grant Watson <grant.watson@arm.com>
yangulei pushed a commit to yangulei/tvm that referenced this pull request Jan 11, 2022
@Mousius @grant-arm @yangulei
[1/3][AOT][DeviceAPI] Connecting devices structure to relevant operat…
d417cb2 
…ors (apache#9395)

* [AOT][DeviceAPI] Connecting devices structure to relevant operators

This patch adds support for passing the device context via the unpacked API in AOT, generating an additional struct if necessary:

```c
/*!
 * \brief Device context pointers for TVM module "default"
 */
struct tvmgen_default_devices {
  void* npu;
};
```

Which is then added as an argument to the entry function:
```c
/*!
 * \brief entrypoint function for TVM module "default"
 * \param inputs Input tensors for the module
 * \param outputs Output tensors for the module
 * \param devices Device context pointers for the module
 */
int32_t tvmgen_default_run(
  struct tvmgen_default_inputs* inputs,
  struct tvmgen_default_outputs* outputs,
  struct tvmgen_default_devices* devices
);
```

I've temporarily added the collection of external code generators to the TE compiler pending proper annotation of the eventual functions.

Co-authored-by: Grant Watson <grant.watson@arm.com>

* Correct "use_device_api" attribute name on Target

Co-authored-by: Grant Watson <grant.watson@arm.com>
ylc pushed a commit to ylc/tvm that referenced this pull request Jan 13, 2022
@Mousius @grant-arm @ylc
[1/3][AOT][DeviceAPI] Connecting devices structure to relevant operat…
41f9e75 
…ors (apache#9395)

* [AOT][DeviceAPI] Connecting devices structure to relevant operators

This patch adds support for passing the device context via the unpacked API in AOT, generating an additional struct if necessary:

```c
/*!
 * \brief Device context pointers for TVM module "default"
 */
struct tvmgen_default_devices {
  void* npu;
};
```

Which is then added as an argument to the entry function:
```c
/*!
 * \brief entrypoint function for TVM module "default"
 * \param inputs Input tensors for the module
 * \param outputs Output tensors for the module
 * \param devices Device context pointers for the module
 */
int32_t tvmgen_default_run(
  struct tvmgen_default_inputs* inputs,
  struct tvmgen_default_outputs* outputs,
  struct tvmgen_default_devices* devices
);
```

I've temporarily added the collection of external code generators to the TE compiler pending proper annotation of the eventual functions.

Co-authored-by: Grant Watson <grant.watson@arm.com>

* Correct "use_device_api" attribute name on Target

Co-authored-by: Grant Watson <grant.watson@arm.com>
@driazati driazati mentioned this pull request Jul 14, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@areusch areusch areusch approved these changes

@anijain2305 anijain2305 Awaiting requested review from anijain2305

@comaniac comaniac Awaiting requested review from comaniac

@jroesch jroesch Awaiting requested review from jroesch

@junrushao junrushao Awaiting requested review from junrushao

@jwfromm jwfromm Awaiting requested review from jwfromm

@kazum kazum Awaiting requested review from kazum

@kparzysz-quic kparzysz-quic Awaiting requested review from kparzysz-quic

@leandron leandron Awaiting requested review from leandron

@liangfu liangfu Awaiting requested review from liangfu

@manupak manupak Awaiting requested review from manupak

@MarisaKirisame MarisaKirisame Awaiting requested review from MarisaKirisame

@masahi masahi Awaiting requested review from masahi

@mbaret mbaret Awaiting requested review from mbaret

@mbrookhart mbrookhart Awaiting requested review from mbrookhart

@merrymercy merrymercy Awaiting requested review from merrymercy

@slyubomirsky slyubomirsky Awaiting requested review from slyubomirsky

@tmoreau89 tmoreau89 Awaiting requested review from tmoreau89

@tqchen tqchen Awaiting requested review from tqchen

@trevor-m trevor-m Awaiting requested review from trevor-m

@vinx13 vinx13 Awaiting requested review from vinx13

@wweic wweic Awaiting requested review from wweic

@yzhliu yzhliu Awaiting requested review from yzhliu

@zhiics zhiics Awaiting requested review from zhiics

@ZihengJiang ZihengJiang Awaiting requested review from ZihengJiang

@icemelon icemelon Awaiting requested review from icemelon

+1 more reviewer

@mbs-octoml mbs-octoml mbs-octoml requested changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@Mousius @mbs-octoml @areusch