lookahead-prompt : add example

[ggerganov]

Add an example implementing the "Prompt Lookup Decoding" technique:

https://github.com/apoorvumang/prompt-lookup-decoding

This should be a great exercise for people looking to become familiar with llama.cpp's KV cache management and batched decoding API. Looking for contributions.

The following examples can be used as starting points:

• speculative
• lookahead
• batched

[0xdevalias]

@apoorvumang FYI

[wsxiaoys]

I just implemented this for tabby in https://github.com/TabbyML/tabby/pull/916/files - it's a slightly more complicated implementation (since tabby runs on continuous batching), but should be something can be used as reference.

[LeonEricsson]

I'd love to give this a try, first time contributing.

[bullno1]

Somewhat related: https://arxiv.org/abs/2312.11462

It seems someone looked at lookup decoding (ngram) and speculative decoding and asked themselves: "Why not both?".
And thus this is the result.

I'm still reading through the paper.

[ggerganov]

Implemented here: https://github.com/ggerganov/llama.cpp/tree/master/examples/lookup