Skip to content

hexagon: tiling, tracing and optimizations for unary ops#25474

Merged
max-krasnyansky merged 10 commits into
ggml-org:masterfrom
qualcomm:aparmp/hex-tile-unary
Jul 9, 2026
Merged

hexagon: tiling, tracing and optimizations for unary ops#25474
max-krasnyansky merged 10 commits into
ggml-org:masterfrom
qualcomm:aparmp/hex-tile-unary

Conversation

@aparmp-quic

@aparmp-quic aparmp-quic commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Overview

This PR extends the "VTCM layout" concept already used for the big Ops (MUL_MAT, FA) (#25425) to pointwise unary/elementwise ops.

  • VTCM layout: host CPU selects the layout used to partition VTCM; the NPU uses the same functions to do the actual partitioning, keeping the two in sync and fixing a VTCM overflow on wide rows.
  • Kernel params computed on the host: row tiling, thread assignment, and other kernel params are now precomputed on the host and passed to the kernel instead of being derived on-device.
  • Specialized loops/functions: per-thread worker functions are specialized to help the compiler generate tighter code for the hot path.
  • Removed code duplication: shared HVX kernel logic factored out into hvx-norm.h.
  • Fastdivs: runtime integer division replaced with fastdiv in the unary kernels.
  • Tracing instrumentation: added for profiling unary ops.

Requirements

  • I have read and agree with the contributing guidelines
  • AI usage disclosure: YES, Antigravity assist for refactoring, profile analysis and drafting

@aparmp-quic aparmp-quic requested a review from a team as a code owner July 9, 2026 05:39
@github-actions github-actions Bot added ggml changes relating to the ggml tensor library for machine learning Hexagon labels Jul 9, 2026
@max-krasnyansky max-krasnyansky force-pushed the aparmp/hex-tile-unary branch from c98c854 to 107103c Compare July 9, 2026 16:44
@max-krasnyansky

Copy link
Copy Markdown
Member

@lhez this should be good to go. second review/ACK please when you get a minute.

@max-krasnyansky max-krasnyansky merged commit fb30ba9 into ggml-org:master Jul 9, 2026
26 of 27 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ggml changes relating to the ggml tensor library for machine learning Hexagon

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants