Skip to content

[Texture support][Part 4] Add CollectStorageInfo and CollectBufferBinds relay passes for Adreno GPU #7689

Closed
csullivan wants to merge 19 commits intoapache:mainfrom
csullivan:texture2d/graph_runtime_adreno_storage
Closed

[Texture support][Part 4] Add CollectStorageInfo and CollectBufferBinds relay passes for Adreno GPU #7689
csullivan wants to merge 19 commits intoapache:mainfrom
csullivan:texture2d/graph_runtime_adreno_storage

Conversation

@csullivan
Copy link
Copy Markdown
Contributor

@csullivan csullivan commented Mar 18, 2021 •
edited
Loading

This PR adds a target specific implementation of CollectStorageInfo and CollectBufferBinds for the opencl --device=adreno target.

CollectStorageInfo

  • Returns a storage mapping for the outputs of each Expr in a Relay function. For Adreno, the output storage is set to "texture" for primitive functions containing a Conv2d with data and kernel layouts of NCHW4c and OIHW4o, respectively, are.
  • The output storage scope for Constants and Vars is inferred from the output storage scope of their consumers.
  • A legalization step occurs that ensures that the output of any primitive function is set to global scope if one or more of its consumers requires global scope for its output. This legalization step occurs after the storage scopes have been collected and therefore is not transitive so that the global scope does not propagate toward the inputs.
  • An additional legalization step occurs to ensure that the global outputs of the relay Function are marked as global scope. This can be removed if CopyFromTo will support passing shape information for the source and destination tensors, or if texture memory reuse is disabled
    • The key issue is that the allocated texture space being read from for a output tensor may be a pool, in which case pitch region and row pitch will be required to correctly read the texture from the pool to the host. The region and pitch information can be inferred from the shape of the src and dst tensors if provided to CopyToFrom.

CollectBufferBinds

  • Translates the storage scopes information provided by the memory planner (and ultimately the above CollectStorageInfo pass) to tir::Buffers that can be bound to te::Tensors when lowering Relay primitive functions. These buffers are provided to CompileEngine and used in tvm::lower/build via the binds field. If the provided buffers have texture scope, the TIR TextureFlattening pass will handle lowering the multidimensional accesses to two dimensions.

See RFC here: https://discuss.tvm.apache.org/t/rfc-texture-memory-support/9467

csullivan added 19 commits March 17, 2021 14:07
@csullivan
Add Adreno specific impl. of CollectStorageInfo.
aea4d95
@csullivan
Populate consumer_scopes with any assigned storage scope.
3b9df08
@csullivan
Add legalization to ensure producer expr
7c14af4 
only produces texture if all consumers support
reading from texture.
@csullivan
Tighten texture assignment to only apply to
2a1ce0e 
primitives containing conv2d in NCHW4c/OIHW4o layout.
@csullivan
Refactor consumer storage homogeneity check.
2630bea
@csullivan
Apply distinct storage scope annotation for weights
345bafb 
in adreno storage scope annotation (CollectStorageInfo).
@csullivan
Only set output storage scope if non-global
517326c
@csullivan
Add CollectBufferBinds packed func for opencl adreno target
d702191
@csullivan
Bug fix in storage scope legalization:
fb25dfa 
Ensure that output is marked as global if single
consumer requires global input. Previous legalization
only ensured this if multiple consumers had differing
storage requirements.
@csullivan
Slight fix to storage scope legalization.
fc9343d
@csullivan
[Part 3/3] Support texture:weight lowering convention for externally …
a9de458 
…provided

texture buffers. Need to propagate this to allocated textures when
cache_read(texture) is used for weights.
@csullivan
Update adreno collect storage to support multiple outputs.
9c039df 
Note for now the Adreno impl. assumes the output storage
types are uniform for multi-output nodes.
@csullivan
Force the output storage of top level relay function
dfd3fbe 
outputs to be global storage to avoid need for image
row pitch calc.
@csullivan
Register opencl adreno target specific CollectStorageInfo impl.
ec0880f
@csullivan
More renaming
20c5de5
@csullivan
Use anonymous namespace
95b18cd
@csullivan
Add annotate_texture_storage.cc docstring.
5c1ad88
@csullivan
Increase texture operator coverage:
ef5ff26 
 - Max/Avg/Global Pooling
 - Concatenate
 - LayoutTransform (NCHW -> NCHW4c)
@csullivan
Refactor out texture storage support for each call into helper method.
817b186
@tmoreau89 tmoreau89 mentioned this pull request Jan 10, 2022
Closed
@masahi masahi mentioned this pull request Jan 10, 2022
Closed
@jroesch jroesch added needs-triage PRs or issues that need to be investigated by maintainers to find the right assignees to address it status: need review labels Jan 19, 2022
@csullivan
Copy link
Copy Markdown
Contributor Author

csullivan commented Oct 14, 2022

Subsumed by #11878.

@csullivan csullivan closed this Oct 14, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

needs-triage PRs or issues that need to be investigated by maintainers to find the right assignees to address it status: need review

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@csullivan @jroesch