JIT: `ArgumentOutOfRangeException` in debug but not release with `DotProductBySelectedQuadruplet`

[jakobbotsch]

// Generated by Fuzzlyn v2.2 on 2024-08-17 15:13:27
// Run on Arm64 MacOS
// Seed: 17566447992392941035-vectort,vector64,vector128,armadvsimd,armadvsimdarm64,armaes,armarmbase,armarmbasearm64,armcrc32,armcrc32arm64,armdp,armrdm,armrdmarm64,armsha1,armsha256
// Reduced from 52.5 KiB to 0.7 KiB in 00:00:21
// Debug: Throws 'System.ArgumentOutOfRangeException'
// Release: Runs successfully
using System;
using System.Runtime.CompilerServices;
using System.Numerics;
using System.Runtime.Intrinsics;
using System.Runtime.Intrinsics.Arm;

public class Program
{
    public static void Main()
    {
        var vr7 = M1();
    }

    public static ushort M1()
    {
        var vr2 = Vector64.CreateScalar(0);
        var vr1 = AdvSimd.Extract(vr2, 0);
        var vr4 = Vector64.Create<int>(0);
        var vr5 = Vector64.Create<sbyte>(0);
        var vr6 = Vector64.Create<sbyte>(0);
        var vr3 = Dp.DotProductBySelectedQuadruplet(vr4, vr5, vr6, 7);
        return (ushort)((byte)ArmBase.Arm64.MultiplyHigh(vr1, 0) >> (-32768 & AdvSimd.Extract(vr3, 0)));
    }
}

cc @dotnet/jit-contrib

[dotnet-policy-service]

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.

[jakobbotsch]

I'm not sure that we need this for .NET 9. Would not expect users to be relying on these boundary conditions. Going to put it in 10.

[AndyAyersMS]

Possibly the same thing we're seeing in #106546.

[amanasifkhalid]

I've confirmed the solution proposed in #106546 fixes this.