Support for bnb nf4 quant?

[col-in-coding]

It seems that there's no decreasing of lattency on llama-13b model using nf4 quant

[Viol2000]

I'm not sure about its compatibility with bnb.
But it is already integrated into llama.cpp. You can check it if you want to run quant models.