llama.cpp

Inference of LLaMA and other large language models in pure C/C++

2.2× faster than LLM Infrastructure this week