ggml : fix MUL_MAT_ID repack with Q8_K

[ggerganov]

fix #12528

• The mul_mat_id assumed Q8_0 type for the src1
• Fix IQ4_NL param type to by Q8_0 instead of IQ4_NL
• Code indentations

[ggerganov]

@Djip007 Could you take a look at these fixes?

[Djip007]

a very quick review...

[Djip007]

For C++ I'll use in this case

        if constexpr (PARAM_TYPE == GGML_TYPE_Q8_0) { ... }
        if constexpr (PARAM_TYPE == GGML_TYPE_Q8_K) { ... }

and if this is the only to possible may may be some

  static_assert( (PARAM_TYPE == GGML_TYPE_Q8_K) || (PARAM_TYPE == GGML_TYPE_Q8_0), "comment");

but it may be trap by adding gem[v/m] PARAM_TYPE as template param

[Djip007]

may be remove the if and change (transforme) the quantize_mat_q8_K/quantize_mat_q8_0 to template quantize_mat<PARAM_TYPE>

template <ggml_type PARAM_TYPE>
void quantize_mat(...);

tempate<> quantize_mat<GGML_TYPE_Q8_0>(...) {
    quantize_mat_q8_0(...);  // or "inline" it.
}
tempate<> quantize_mat<GGML_TYPE_Q8_0>(...) {
    quantize_mat_q8_K(...);
}

[Djip007]

with the static_assert it will be catch a build time.

[Djip007]

may be add the PARAM_TYPE on this template to

template <typename BLOC_TYPE, int64_t INTER_SIZE, int64_t NB_COLS, ggml_type PARAM_TYPE>
void gem[m,v]....

[ggerganov]

@Djip007 Thanks. Let me know if you have any other suggestions.

[ggerganov]

Merging this for now and happy to hear if you have any additional suggestions.