feat: build llama.cpp with ggml-hexagon backend for Snapdragon NPU (Android + Windows ARM + Linux ARM)

[Ericson246]

Summary

Build llama.cpp with -DGGML_HEXAGON=ON for Snapdragon devices. Upstream merged the ggml-hexagon backend (PR ggml-org#16547) but publishes no pre-built binaries with NPU support.

Target runtimes

ID	Platform	Arch	Backend	CMake flags
android-arm64-hexagon	android	arm64	hexagon	-DGGML_HEXAGON=ON -DGGML_OPENCL=ON -DGGML_OPENMP=OFF -DGGML_LLAMAFILE=OFF
windows-arm64-hexagon	windows	arm64	hexagon	same + MSVC arm64 toolchain
linux-arm64-hexagon	linux	arm64	hexagon	-DGGML_HEXAGON=ON -DGGML_OPENCL=OFF -DGGML_OPENMP=OFF -DGGML_LLAMAFILE=OFF

Build dependencies

Android (Docker)

• Image: ghcr.io/snapdragon-toolchain/arm64-android:v0.7
• Includes: Android NDK r28b, Hexagon SDK 6.6.0.0, OpenCL SDK, CMake, Ninja
• Host: any (Docker Desktop), --platform linux/amd64
• Build time: ~15-30 min first, ~3-8 min incremental
• Output: pkg-snapdragon/ with lib/ (HTP skels) + bin/ (llama-server, llama-cli, llama-bench)

Windows ARM64 (native)

• Host: ARM64 Windows (self-hosted runner or cross-compile from x64)
• MSVC 2026 arm64 + LLVM/Clang (via winget)
• Hexagon SDK 6.6+ (free, Qualcomm account required)
  • URL: https://softwarecenter.qualcomm.com/catalog/item/Hexagon_SDK?version=6.6.0.0
  • CI-optimized: https://github.com/snapdragon-toolchain/hexagon-sdk
• OpenCL SDK 2.3+ (Adreno GPU)
  • CI-optimized: https://github.com/snapdragon-toolchain/opencl-sdk
• Qualcomm NPU driver: https://softwarecenter.qualcomm.com/catalog/item/Qualcomm_HND
• Extra: Windows test-signing mode (bcdedit /set TESTSIGNING ON) + self-signed cert for NPU driver

Linux ARM64 (cross-compile)

• Image: ghcr.io/snapdragon-toolchain/arm64-linux:v0.1
• Host: Linux x86_64 only (cross-compilation)
• Currently no OpenCL support on Linux Snapdragon (GGML_OPENCL=OFF)

Asset naming

Follow existing pattern:

llama-{tag}-bin-{platform}-{backend}-{arch}.{ext}

Examples:

llama-daily-2026-06-20-bin-android-hexagon-arm64.tar.gz
llama-daily-2026-06-20-bin-win-hexagon-arm64.zip
llama-daily-2026-06-20-bin-linux-hexagon-arm64.tar.gz

Proposed CI workflow

name: Build Hexagon NPU runtimes

on:
  schedule:
    - cron: '0 2 * * *'
  workflow_dispatch:

jobs:
  android:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Build in Docker toolchain
        run: |
          docker run --rm -u $(id -u):$(id -g) \
            -v $(pwd):/workspace \
            --platform linux/amd64 \
            ghcr.io/snapdragon-toolchain/arm64-android:v0.7 \
            bash -c "
              cp docs/backend/snapdragon/CMakeUserPresets.json .
              cmake --preset arm64-android-snapdragon-release -B build-snapdragon
              cmake --build build-snapdragon -j \$(nproc)
              cmake --install build-snapdragon --prefix pkg-snapdragon
            "
      - name: Package
        run: |
          TAG=$(date +daily-%Y-%m-%d)
          tar czf "llama-$TAG-bin-android-hexagon-arm64.tar.gz" -C pkg-snapdragon .
      - uses: softprops/action-gh-release@v2
        with:
          tag_name: daily-${{ github.event.repository.updated_at }}
          files: llama-*-hexagon-*.tar.gz
          prerelease: true

  windows-arm64:
    runs-on: [self-hosted, windows, arm64]
    # ... requires ARM64 Windows runner

  linux-arm64:
    runs-on: ubuntu-latest
    # ... uses arm64-linux Docker toolchain

Runtime usage

# Android
adb push pkg-snapdragon /data/local/tmp/
adb shell "cd /data/local/tmp/pkg-snapdragon && \
  LD_LIBRARY_PATH=./lib ADSP_LIBRARY_PATH=./lib \
  ./bin/llama-server --no-mmap --device HTP0 -m /model.gguf"

# Windows
$env:ADSP_LIBRARY_PATH=".\lib"
.\bin\llama-server.exe --device HTP0 -m model.gguf

# Linux
LD_LIBRARY_PATH=./lib ADSP_LIBRARY_PATH=./lib \
  ./bin/llama-server --device HTP0 -m model.gguf

Runtime environment variables

Variable	Default	Description
ADSP_LIBRARY_PATH	./lib	Path for Hexagon DSP to find HTP skel files
GGML_HEXAGON_NDEV	1	Number of HTP sessions (2 for 8B, 4 for 20B)
GGML_HEXAGON_VERBOSE	off	Debug logging
GGML_HEXAGON_PROFILE	off	Performance profiling
GGML_HEXAGON_OPFILTER	""	Regex to disable ops on Hexagon (e.g. "FLASH_ATTN_EXT")

Hardware reference

SoC	Hexagon Arch	Devices
Snapdragon 8 Gen 2	v73	Galaxy S23, OnePlus 11
Snapdragon 8 Gen 3	v73	Galaxy S24, OnePlus 12
Snapdragon 8s Gen 3	v75	Mid-tier 2025+
Snapdragon 8 Elite	v79	Galaxy S25, OnePlus 13
Snapdragon 8 Elite Gen 5	v81	Galaxy S26+
Snapdragon X Plus / X Elite	v81	Surface Laptop 7, ThinkPad T14s Gen 6 ARM, XPS 13 9345
Snapdragon X2 Elite	v81+	2026+ laptops

Known limitations

Limitation	Impact
Experimental backend	May have stability issues
Only Q4_0, Q8_0, MXFP4, FP32	Most HF GGUF models use Q4_K_M — need re-quant
No Flash Attention in prefill	Falls back to CPU for prefill
Windows driver signing	Test mode required (self-signed cert)
Models >8B need multi-session	GGML_HEXAGON_NDEV=2+ may exhaust NPU memory
OpenCL disabled on Linux	No Adreno GPU fallback on Linux
LLAMAFILE disabled	Incompatible with target arch
No on-disk REPACK format	~600ms extra load time per model

Related

• Tracking in npu-optimize: feat: Qualcomm Hexagon NPU support (Android + Windows ARM + Linux ARM) npu-optimize#5
• Upstream docs: docs/backend/snapdragon/README.md
• Upstream PRs: Add experimental ggml-hexagon backend for the Hexagon NPU ggml-org/llama.cpp#16547 (ggml-hexagon), hexagon: improved Op queuing, buffer and cache management ggml-org/llama.cpp#21705 (op queuing), hexagon: enable offloading to Hexagon on Windows on Snapdragon ggml-org/llama.cpp#19150 (Windows)
• Snapdragon toolchain: https://github.com/snapdragon-toolchain
• Qualcomm Hexagon SDK (free): https://softwarecenter.qualcomm.com/catalog/item/Hexagon_SDK
• Perf benchmarks: https://specpicks.com/reviews/llama-cpp-snapdragon-hexagon-npu-benchmarks-2026