**Problem** This is new implementation of Self-Extend to make LLM extend to 8k or 16k without further training **Success Criteria** We can also use in Nitro ``` --grp-attn-n 2 --grp-attn-w 128 ``` **Additional context** The llama.cpp supports it: https://github.com/ggerganov/llama.cpp/pull/4815
Problem
This is new implementation of Self-Extend to make LLM extend to 8k or 16k without further training
Success Criteria
We can also use in Nitro
Additional context
The llama.cpp supports it: ggml-org/llama.cpp#4815