[API Proposal]: Add managed surface for AVX-512 BF16 hardware intrinsics

[jamesburton]

Background and motivation

The classic AVX512_BF16 instruction set adds bfloat16 support to AVX-512: a bf16 dot-product accumulating into FP32 (VDPBF16PS) and FP32→bf16 round-to-nearest-even conversions (VCVTNE2PS2BF16, VCVTNEPS2BF16). It is supported on Intel Cooper Lake / Ice Lake-SP / Sapphire Rapids+ and on AMD Zen 4 and Zen 5 (including Strix Halo), but there is currently no managed surface for it in System.Runtime.Intrinsics.X86 on the classic-AVX512 path.

This is exactly the situation #86849 / PR #128365 solved for AVX512_VNNI: an R2R name and the underlying JIT plumbing existed under AVX10v*, but shipping Sapphire Rapids / Zen 4 / Zen 5 hardware does not enable AVX10, so the surface was unreachable on the parts where it matters. The fix shape that landed there — separate managed surface tied to the classic gate, AVX10 path stays parallel — is the precedent this proposal follows.

What already exists in dotnet/runtime (verified by grep)

• System.BFloat16 primitive (src/libraries/System.Private.CoreLib/src/System/BitConverter.cs:312-1016 — full GetBytes(BFloat16), ToBFloat16, Int16BitsToBFloat16, BFloat16ToInt16Bits etc.). So Vector*<BFloat16> is the natural element type — no ushort interim.
• R2R IDs already allocated: READYTORUN_INSTRUCTION_Avx512Bf16 = 72, READYTORUN_INSTRUCTION_Avx512Bf16_VL = 73 (src/coreclr/inc/readytoruninstructionset.h:82-83).
• InstructionSetDesc.txt:76-77 has rows for Avx512Bf16 and Avx512Bf16_VL — managed-name column (column 3) is empty, parent (column 6) is AVX10v1:

  instructionset     ,X86   ,                     ,Avx512Bf16         ,72 ,AVX10v1               ,avx10v1
  instructionset     ,X86   ,                     ,Avx512Bf16_VL      ,73 ,AVX10v1               ,avx10v1
  

• JIT instruction encodings already in src/coreclr/jit/instrsxarch.h:
  • vdpbf16ps (line 1035) — EVEX
  • vcvtne2ps2bf16 (line 994) — EVEX
  • vcvtneps2bf16 (line 995) — EVEX

So the scope of the BF16 PR is small relative to a green-field intrinsic: fill in the managed-name column, route to the classic gate, add the public C# class, wire the JIT dispatch, add tests.

Motivating use case — quantized / mixed-precision ML inference

VDPBF16PS accumulates bf16×bf16 products directly into an FP32 accumulator. For block-scaled integer quantization (e.g. GGUF Q8_0: 32 int8 + one fp16 scale per block), an integer-VNNI dot must convert+rescale after every block, which prevents long accumulation. A BF16 path can dequantize int8→bf16 folding the per-block scale into the value, then accumulate the already-scaled products across all blocks in one FP32 accumulator via VDPBF16PS — removing the per-block reduction entirely. Concrete data point: an A/B on Zen 5 / Strix Halo had a 512-bit integer-VNNI Q8_0 GEMM kernel hitting only parity with the existing maddubs path precisely because of those per-block reductions; BF16 long-accumulation is the path past that ceiling. BF16 GEMM and attention more generally also benefit.

API Proposal

Mirrors the established Avx512* / AvxVnni pattern.

namespace System.Runtime.Intrinsics.X86
{
    /// <summary>Provides access to X86 AVX512-BF16 hardware instructions via intrinsics.</summary>
    [Intrinsic]
    public abstract class Avx512Bf16 : Avx512F
    {
        public static new bool IsSupported { get; }

        /// <summary>__m512 _mm512_dpbf16_ps (__m512 src, __m512bh a, __m512bh b) — VDPBF16PS zmm, zmm, zmm/m512</summary>
        public static Vector512<float> MultiplyWideningAndAdd(Vector512<float> addend, Vector512<BFloat16> left, Vector512<BFloat16> right);

        /// <summary>__m512bh _mm512_cvtne2ps_pbh (__m512 a, __m512 b) — VCVTNE2PS2BF16 zmm, zmm, zmm/m512 (packs two fp32 vectors → one bf16 vector)</summary>
        public static Vector512<BFloat16> ConvertToBFloat16(Vector512<float> lower, Vector512<float> upper);

        /// <summary>__m256bh _mm512_cvtneps_pbh (__m512 a) — VCVTNEPS2BF16 ymm, zmm/m512</summary>
        public static Vector256<BFloat16> ConvertToBFloat16(Vector512<float> value);

        [Intrinsic] public new abstract class X64 : Avx512F.X64 { public static new bool IsSupported { get; } }

        [Intrinsic]
        public new abstract class VL : Avx512F.VL
        {
            public static new bool IsSupported { get; }
            public static Vector128<float> MultiplyWideningAndAdd(Vector128<float> addend, Vector128<BFloat16> left, Vector128<BFloat16> right);
            public static Vector256<float> MultiplyWideningAndAdd(Vector256<float> addend, Vector256<BFloat16> left, Vector256<BFloat16> right);
            public static Vector128<BFloat16> ConvertToBFloat16(Vector128<float> lower, Vector128<float> upper);
            public static Vector256<BFloat16> ConvertToBFloat16(Vector256<float> lower, Vector256<float> upper);
            public static Vector128<BFloat16> ConvertToBFloat16(Vector128<float> value);
            public static Vector128<BFloat16> ConvertToBFloat16(Vector256<float> value);
        }
    }
}

Proposed ISA grouping — alignment with the #128365 precedent

This is the architectural question that ate the most review time on the VNNI PR, so it is being surfaced up-front for confirmation rather than left for review-time discovery.

@tannergooding's stated principle on #128365 (resolved in commit ebddfff3):

"While there is a very small subset of hardware that contains just AVX512v2 + AVX-512-VNNI, we don't want to support that being enabled/disabled independently without significant justification, as it adds unnecessary complexity to the overall support and testing matrix. Instead, we just want to separate this based on what captures the broadest overall support."

Applied to BF16, the symmetric question is: does v3-as-currently-defined imply BF16 on shipping hardware?

	full AVX512v3 group	AVX512_BF16
Intel Cooper Lake / Ice Lake-SP / Sapphire Rapids+	✓	✓
AMD Zen 4 (Genoa, Bergamo)	✓	✓
AMD Zen 5 (Granite Ridge, Turin, Strix Halo)	✓	✓
Intel Ice Lake-U/Y, Tiger Lake, Rocket Lake	✓	✗

This is asymmetric with the VNNI case: there is real shipping hardware (Intel client AVX-512 parts pre-disable) that ships full v3 but lacks BF16. Widening v3 to require BF16 would silently drop v3 enablement on those parts — a regression for those users.

Recommended shape (option a): add AVX512_BF16 as its own ISA group with implication AVX512_BF16 → AVX512v3. That preserves Tanner's "what captures the broadest overall support" principle (Tiger Lake keeps its v3 surface; SPR / Zen 4 / Zen 5 / Strix Halo gain BF16) and satisfies the "significant justification" he asked for on VNNI — namely, that v3-without-BF16 is a real shipping configuration, which v2-and-VNNI-without-v3 was not.

Concretely this means editing InstructionSetDesc.txt:76-77 to:

1. Fill in column 3 with the managed names (Avx512Bf16, Avx512Bf16.VL).
2. Re-point column 6 from AVX10v1 to the new AVX512_BF16 group.
3. Add an instructionset row for the AVX512_BF16 group plus implication ,X86 ,AVX512_BF16 ,AVX512v3.

Alternative shapes considered:

• (b) Same as (a) framed as "edit the existing rows" rather than introduce a new group — likely produces a smaller diff but functionally equivalent.
• (c) Widen v3 to require BF16. Mentioned for completeness only — regresses Tiger Lake et al. as noted, do not pursue unless explicitly directed.

Alignment with #128365's specific guidance

The PR that follows this issue will mirror these patterns from the VNNI review verbatim:

• No separate AVXVNNI_V512-style ISA "just because it's a V512 thing" — the v512 width and the classic-gate group are one ISA, not two.
• simdSize = -1 on the HARDWARE_INTRINSIC rows so that V128/V256/V512 calls all dispatch through one set of entries — matching the AVXVNNIINT / AVXVNNIINT_V512 precedent. Mixing fixed simdSize = 64 with VL forms in one ISA was the trap on Add AvxVnni.V512 hardware intrinsics #128365.
• Dispatch in Compiler::lookupInstructionSet, not a codeman.cpp-side fallback. (Tanner's review on codeman.cpp:1430 of Add AvxVnni.V512 hardware intrinsics #128365 was explicit: "the right place is the JIT's class-name dispatch.") BF16 has no VEX-encoded sibling to fall back to, so the dispatch is just "return the group" — no compSupportsHWIntrinsic branch.
• Don't drop tokens from the required-ISAs comments next to existing config flags. Nit pass on Add AvxVnni.V512 hardware intrinsics #128365 (commit 91cdc6c2) twice flagged accidentally-dropped VNNI tokens; same care needed here.
• JIT-EE GUID regenerated every InstructionSetDesc.txt change. Expect 4-6 merge cycles, each requiring a fresh GUID — never pick one side of the conflict.

Open questions for confirmation by reviewers

I'd like @tannergooding (and the API-review team where relevant) to confirm or vary:

1. Group shape. Approve option (a) — AVX512_BF16 as own group with implication → AVX512v3 — vs option (b) (same semantics, edit the existing rows in place) vs option (c) (widen v3, regressing Tiger Lake). My read is (a) best satisfies the "broadest overall support" principle while honouring the "significant justification" requirement. Open to direction.
2. Element type. Vector*<BFloat16> using the existing System.BFloat16 primitive — confirm this is the intended shape rather than a Vector*<ushort> raw-bits stand-in.
3. Parent class. Avx512Bf16 : Avx512F (sibling of Avx512Vbmi, AvxVnni, etc.) — confirm rather than nesting under a tier.
4. Naming. MultiplyWideningAndAdd for the dot-product (parallel to AvxVnni), ConvertToBFloat16 for the two/one-source conversions. Open to convention.
5. Preview gating. [RequiresPreviewFeatures] initially, per AvxVnni precedent.

If any of (1)–(5) need to vary from what's proposed, please flag here so the draft PR (linked below once opened) can be aligned before review cycles start.

Alternatives considered

• AVX10 BF16 (Avx10v*) — gated on AVX10, unavailable on Sapphire Rapids / Zen 4 / Zen 5 / Strix Halo. Does not serve the parts where the ML use case lives.
• Software bf16 dot — no hardware acceleration; defeats the motivating use case.

References

• [API Proposal]: Add support for AVX-512 VNNI hardware instructions #86849 / PR Add AvxVnni.V512 hardware intrinsics #128365 — AvxVnni.V512 — the direct precedent this proposal follows.
• The classic Intel Software Developer's Manual entries for VDPBF16PS, VCVTNE2PS2BF16, VCVTNEPS2BF16.

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/area-system-runtime-intrinsics
See info in area-owners.md if you want to be subscribed.

[jamesburton]

Draft implementation opened at #129326 — JIT-side codegen wiring intentionally left as TODO pending your call on the ISA grouping question (option (a) is what the draft implements). Happy to pivot to (b) or (c) on a follow-up commit before that codegen work.

[tannergooding]

This is exactly the situation #86849 / PR #128365 solved for AVX512_VNNI

This is notably a very different situation. For the VNNI case there are two separate and fully independent ISAs involved AVX512-VNNI and AVX-VNNI. For the AVX512-BF16 case, there is only the AVX512-BF16 ISA. AVX10v1 is simply a superset of it and one where we decided at the time it was not worth creating an AVX512v4 ISA to encompass AVX512v3 + FP16 + BF16

So the scope of the BF16 PR is small relative to a green-field intrinsic: fill in the managed-name column, route to the classic gate, add the public C# class, wire the JIT dispatch, add tests.

This is actually not correct. The reason that F16C, FP16, and BF16 have all not happened yet is because they require very extensive changes to the JIT and the entire SIMD support the T in Vector128<T> to be new non-runtime primitive types. It also ends up impacting ABI and a few other things in various scenarios, so may cross over into the VM, Debugger, etc. There are a few somewhat cheaper ways to do some of this, but it is still a fundamentally expensive and expansive change due to Vector128<BFloat16>.IsSupported requiring to report true

---

Group shape. Approve option (a) — AVX512_BF16 as own group with implication → AVX512v3 — vs option (b) (same semantics, edit the existing rows in place) vs option (c) (widen v3, regressing Tiger Lake). My read is (a) best satisfies the "broadest overall support" principle while honouring the "significant justification" requirement. Open to direction.

This is an implementation detail of the JIT. We can do whatever we feel is best given the hardware and other considerations and even change it over time with enough user feedback.

I don't believe your table is quite correct, however. Cooper Lake is generally an interesting piece, it came after Ice Lake, yet has overall less AVX512 support and is generally still classified as "first gen AVX512". Its Vector512 support is even setup to report IsHardwareAccelerated=false (by default, there is a config switch opt-in) because of the significant downclocking that may occur. It is missing the VBMI2, BITALG, and VPOPCNTDQ support that AVX512v3 requires, as well as the IFMA and VBMI support that AVX512v2 requires. Then Ice Lake, Tiger Lake, and Rocket Lake are all missing AVX512-BF16 support.

So what we actually have is this:

• Skylake-X (June 2017): AVX512v1
  • V512.IsHardwareAccelerated is false by default
• Cannon Lake (May 2018): AVX512v2
• Cascade Lake (April 2019): AVX512v1 + VNNI
  • V512.IsHardwareAccelerated is false by default
• Ice Lake (September 2019): AVX512v3
• Cooper Lake (June 2020): AVX512v1 + VNNI + BF16
  • V512.IsHardwareAccelerated is false by default
• Tiger Lake (September 2020): AVX512v3 + VP2INTERSECT
• Rocket Lake (March 2021): AVX512v3
• Zen4 (September 2022): AVX512v3 + VP2INTERSECT + BF16
• Sapphire Rapids (January 2023): AVX512v3 + BF16 + FP16
• Zen5 (July 2024): AVX512v3 + BF16
• Zen6 (TBD): AVX512v3 + VP2INTERSECT + BF16 + FP16

and what this really gives us is just Zen4, Sapphire Rapids, Zen5, and Zen6 as things which don't have AVX10v1 but which do have BF16+FP16. With Sapphire Rapids and Zen6 having all the features required for AVX10v1 (afair) just simply not having the CPUID bits reported.

The question is therefore whether introducing AVX512v4 for Zen4/5 is sufficient, as we'd be much more likely to update the JIT to just have Sapphire Rapids and Zen6 "lie" and report they do support AVX10v1 here, given its purely a unifying ISA.

I'm not convinced that's worthwhile, particularly for BF16 where there isn't even that significant of a perf advantage. The conversion ops are actually relatively simple here with bfloat16 -> float being even more trivial and essentially just "widen ushort -> uint". float -> bfloat16 is a little more interesting, as you need to account for rounding, but its still like a 4-5 instruction sequence and you need to implement such a fallback anyways.

---

Element type. Vector* using the existing System.BFloat16 primitive — confirm this is the intended shape rather than a Vector* raw-bits stand-in.

This is the long term intended shape. However, we could get bfloat16 support in for significantly cheaper if we exposed the initial signatures as Vector128<ushort> and then later also exposed overloads taking Vector128<BFloat16> when that support is complete. We've just not viewed that as impactful enough to do it yet, particularly not for BF16 where the fallbacks are so trivial and still cheap.

Parent class. Avx512Bf16 : Avx512F (sibling of Avx512Vbmi, AvxVnni, etc.) — confirm rather than nesting under a tier.

Yes, the managed ISA tries to most correctly model the actual hierarchy and is independent of how we internally model the JIT support, as the managed ISA needs to be future proof and support runtimes that may want to support things at a fine grained level; while the JIT support is just simplifying our own implementation and maintenance.

Naming. MultiplyWideningAndAdd for the dot-product (parallel to AvxVnni), ConvertToBFloat16 for the two/one-source conversions. Open to convention.

The names sound correct at a glance.

Preview gating. [RequiresPreviewFeatures] initially, per AvxVnni precedent.

This is unnecessary. AvxVnni was preview gated because we didn't have hardware to test and validate against when support was first added.

[jamesburton]

Thanks @tannergooding — every one of your corrections is taken.

Hardware table: I had Cooper Lake mis-grouped (it's first-gen AVX512 + VNNI + BF16, not v3+BF16) and Tiger/Rocket/Ice Lake without enumerating their VP2INTERSECT story. Updated mental model is your "Zen4 / Sapphire Rapids / Zen5 / Zen6 are the v3-without-AVX10v1 set with BF16" reading.

Scope: also taken. I was framing "the surface is small" because the R2R ID, the instruction encodings, and the BFloat16 BCL primitive all exist already — but you're right that Vector128<BFloat16>.IsSupported = true and the inlining/codegen story require non-trivial JIT/VM/Debugger/ABI work that I had not surfaced. That's the same reason F16C and FP16 are still outstanding. I won't claim small-scope on that path.

Acting on the cheaper path you described: I'll pivot the draft #129326 to expose the signatures as Vector*<ushort> and leave the Vector*<BFloat16> overloads for the follow-up when the primitive-type story is ready. Also dropping the [RequiresPreviewFeatures] framing — you're right that was a VNNI-era hardware-availability hedge.

On the grouping: I'll defer to your "I'm not convinced AVX512v4 is worthwhile" read for now and leave the draft's AVX512_BF16-as-own-group shape as the minimum-impact wiring (R2R IDs 72/73 keep their current numeric value, parent re-pointed). If the right answer is actually "just leave it under AVX10v1 and have Sapphire Rapids / Zen6 lie about AVX10v1", happy to do that on the next push — but I'd want your nod before redirecting that part.

I'll push the ushort pivot to the draft in the next hour. Thanks again for the detailed corrections.

[jamesburton]

Heads-up while the grouping question is still open: a downstream consumer (a BF16 quant kernel) tested PR #129326 on Zen5 Strix Halo and confirmed Avx512Bf16.IsSupported == true but all three V512 entry points throw NotImplementedException (expected — the JIT-side special codegen was a flagged TODO in the draft PR body).

Going to start wiring the special codegen for VDPBF16PS / VCVTNE2PS2BF16 / VCVTNEPS2BF16 (importer dispatch in hwintrinsicxarch.cpp, emit in hwintrinsiccodegenxarch.cpp, LSRA in lsraxarch.cpp) on top of the ushort-shape pivot. That codegen path is identical regardless of whether the final grouping is option-a / AVX10v1-lie / AVX512v4, so it shouldn't conflict with whichever direction you call here.

Will keep the PR in draft until you sign off on the grouping — but the codegen wire-up unblocks the consumer for testing and proves the ushort signature path end-to-end. Shout if you'd rather I hold off.

[jamesburton]

Quick follow-up — codegen is wired and verified on Zen5 (Strix Halo).

Two commits pushed to the PR branch (still draft):

• 1daa6f25c00: Wire JIT codegen for the ushort-shape intrinsics — MultiplyWideningAndAdd goes table-driven (VDPBF16PS in the USHORT slot, LSRA RMW alongside AVXVNNI), ConvertToBFloat16 keeps SpecialCodeGen with a small handler that picks VCVTNE2PS2BF16 (2-arg) vs VCVTNEPS2BF16 (1-arg) by node operand count.
• b8bf2951b65: Add InstructionSet_AVX512_BF16 / _X64 to the per-ISA dispatch in genHWIntrinsic — without this the SpecialCodeGen-flagged ConvertToBFloat16 fell through to default:unreached() → fatal(CORJIT_RECOVERABLEERROR) → InvalidProgramException at JIT time.

Probe results on Zen5 (Ryzen AI Max / Strix Halo) with the rebuilt 11.0.0-dev runtime:

Avx512Bf16.IsSupported = True
P1 ConvertToBFloat16(lo,hi) OK  lane0=0x3F80 (expect 0x3F80)
P2 ConvertToBFloat16(v) OK    lane0=0x3F80 (expect 0x3F80)
P3 MultiplyWideningAndAdd OK  lane0=2 (expect 2)

All three V512 entry points emit and execute correctly. Codegen is robust to whichever ISA-grouping direction you call here — happy to fold whatever you decide on (option-a / AVX10v1-lie / AVX512v4) when you're ready.

[jamesburton]

Second iteration — found and fixed a FullOpts codegen bug specific to vdpbf16ps.

Downstream consumer ran their kernel under [MethodImpl(AggressiveOptimization)] (which we hadn't been testing — bf16probe runs at tier-0/MinOpts). FullOpts reproduces with ExecutionEngineException: Illegal instruction (#UD) on the 4-acc control and AccessViolation on the 24-acc spill-pressure case; MinOpts is clean.

Root cause: INS_vdpbf16ps was missing from emitter::Is3OpRmwInstruction (emitxarch.cpp:107). The 3-operand-RMW codegen path is:

emitIns_SIMD_R_R_R_R(ins, attr, targetReg, op1Reg, op2Reg, op3Reg, instOptions)

which dispatches on Is3OpRmwInstruction:

• true → movaps target, op1 + emitIns_R_R_R(target, op2, op3) → 6-byte 3-operand encoding
• false → fall through to a 4-operand blendv encoding emitting 7 bytes with the 4th operand garbage-encoded in the IS4 byte

Missing-from-table → fell through to the blendv path. Bytes for vdpbf16ps zmm0, zmm0, zmm4, zmm23 were 62 F2 7E 48 52 C4 70 (7 bytes — note trailing 0x70 IS4). MinOpts happened to land on operand layouts where the garbage byte didn't trip the CPU; FullOpts varied register triples enough that the CPU rejected sequences as #UD.

Fix in 32572f8e82e: added case INS_vdpbf16ps: to Is3OpRmwInstruction. After the fix, bytes are 62 B2 5E 48 52 C7 (proper 6-byte 3-operand RMW with accumulator in EVEX.vvvv). All three downstream stress modes pass:

CONTROL  (4 acc, FullOpts):  STRESS_OK
FAITHFUL (24 acc, FullOpts): STRESS_OK
MINOPTS  (24 acc):           STRESS_OK

Note: VCVTNE2PS2BF16 / VCVTNEPS2BF16 were emitting correct bytes the whole time (they go through genHWIntrinsic_R_R_RM / genHWIntrinsic_R_RM, not the R_R_R_R path). The consumer's hypothesis pointed at vcvtne2ps2bf16 because it was the disasm line just before the actual fault.