diff --git a/README.md b/README.md index 4ceb19d..3b9c272 100644 --- a/README.md +++ b/README.md @@ -1,20 +1,19 @@ # vit.cpp -The objective of the project is to : - -* Create a simple C/C++ inference engine for Vision Transformer(ViT) models with no dependencies +The objective of the project is to create a C++ inference engine for Vision Transformer(ViT) models +using [ggml](https://github.com/ggerganov/ggml) which focuses on performance edge devices. The implementation is destined to be lightweight and self-contained to be able to run it on different platforms. -This is inspired by the following projects: -* [whisper.cpp](https://github.com/ggerganov/whisper.cpp) -* [llama.cpp](https://github.com/ggerganov/llama.cpp) +Per device optimizations are possible and quantization techniques will be added. ## [This is a work in progress] + ## Vision Transformer architecture ![Vision Transfomer overview](assets/image.png) # Convert PyTorch to ggml format + pip install torch timm python save_pth.py python convert-pth-to-ggml.py vit_base.pth . 1 @@ -85,6 +84,10 @@ This is inspired by the following projects: - [✔] Compare with PyTorch output - [ ] Benchmark inference speed vs. PyTorch models +This project was highly inspired by the following projects: +* [whisper.cpp](https://github.com/ggerganov/whisper.cpp) +* [llama.cpp](https://github.com/ggerganov/llama.cpp) +
ViT Base PyTorch weights @@ -240,4 +243,4 @@ This is inspired by the following projects: norm.bias : [768] head.weight : [1000, 768] head.bias : [1000] -
\ No newline at end of file +