SOTA 2-bit quants#4773

Merged

ikawrakow merged 17 commits into

Jan 8, 2024

Commits on Jan 8, 2024

iq2_xxs: basics
Iwan Kawrakow
committed
iq2_xxs: scalar and AVX2 dot products
Iwan Kawrakow
committed
iq2_xxs: ARM_NEON dot product
Iwan Kawrakow
committed
iq2_xxs: WIP Metal
Iwan Kawrakow
committed
iq2_xxs: Metal dot product now works
Iwan Kawrakow
committed
iq2_xxs: slighty faster dot product
Iwan Kawrakow
committed
iq2_xxs: slighty faster dot product
Iwan Kawrakow
committed
iq2_xxs: even faster Metal dot product
Iwan Kawrakow
committed
iq2_xxs: dequantize CUDA kernel - fix conflict with master
Iwan Kawrakow
committed
iq2_xxs: quantized CUDA dot product (MMVQ)
Iwan Kawrakow
committed
iq2_xxs: slightly faster CUDA dot product
Iwan Kawrakow
committed
iq2_xxs: add to llama ftype enum
Iwan Kawrakow
committed
iq2_xxs: fix MoE on Metal
Iwan Kawrakow
committed
Fix missing MMQ ops when on hipBLAS
Iwan Kawrakow
committed
Fix bug in qequantize_row_iq2_xxs
Iwan Kawrakow
committed
Fixing tests
Iwan Kawrakow
committed
PR suggestion
Iwan Kawrakow
committed