Add integration/CMake for SYCL backend

[jackie1218]

At this point, given the >4GB allocation bug was fixed, if possible, I'd like to see the SYCL backend integrated into koboldcpp. While Intel GPUs are supported by Vulkan, SYCL is much faster at prompt processing (4-7x in some cases, though Vulkan is getting faster.) Even if it's not included by default with koboldcpp (much like hipBLAS) it should be supported as a backend.

[LostRuins]

Unfortunately I have no way to test it as I have not gotten SYCL to run on my device.

[jackie1218]

once ggml-org#5357 is done upstream it should at least build to test on non-Intel hardware, but understandable.

[tpimh]

I have an A770 16Gb, is there anything I can help with?

[ChristophHaag]

amd sycl support has been merged upstream btw. ggml-org#9658

Seems to only work with the sycl implementation in intel's oneapi sdk and not with any other sycl implementation but at least it's something.

[DarwinAnim8or]

I can confirm that the oneapi sycl implementation works, I've been using it myself on a Arc A380; I might see if I can adapt it to koboldcpp

[VeleSila]

I can confirm that the oneapi sycl implementation works, I've been using it myself on a Arc A380; I might see if I can adapt it to koboldcpp

Do you have any updates?

[gustrd]

I was also interested at this, but my last tests shows almost equal performance between Vulkan and SYCL for my ARC GPU.

@VeleSila, did you have different results? Vulkan improved a lot lately, you just must not use Flash Attention or it becomes very slow!

[VeleSila]

I was also interested at this, but my last tests shows almost equal performance between Vulkan and SYCL for my ARC GPU.

@VeleSila, did you have different results? Vulkan improved a lot lately, you just must not use Flash Attention or it becomes very slow!

I'm waiting for the availability of the Intel B60.

Regarding Flash Attention, LostRuins replied to a Q&A saying Flash Attention does not work well on Vulkan. The cause is Vulkan does not have native support for Flash Attention.

[LostRuins]

Actually this is no longer the case. Ever since ggml-org#13324 was merged, Vulkan now supports flash attention, although its implementation is slower than the CUDA one.

[VeleSila]

Unfortunately I have no way to test it as I have not gotten SYCL to run on my device.

In fact, do you mean you don't have a Intel ARC graphic card, or you already have the necessary hardware?

[gustrd]

I was able to vibecode (as an exercise) an adaptation of koboldcpp to use SYCL, ported from upstream.

I tested only at Windows with Lunar Lake. The performance is way inferior to the current Vulkan one.

The only advantage was the superior stability, as Intel drivers are prone to crashing with Vulkan.

https://github.com/gustrd/koboldcpp/tree/feat-sycl