by ggml-org
Free MIT-licensed C/C++ engine that runs LLMs locally. 130,000+ GitHub stars, GGUF models, OpenAI-compatible server. A developer tool.
Pick llama.cpp if you want the engine itself rather than an app around it. It suits developers and tinkerers who need fine control or have limited memory. Everyone else is better served by a wrapper app. The new Llama menu bar app, a 1 MB download for Mac and Windows, is the gentler way in.
Backends for Nvidia, AMD, Intel, Apple and Snapdragon hardware in one engine, with new model support that wrapper apps inherit later.
The MIT code, a very large contributor base and its role as the engine inside other local runners create strong gravity, but the licence lets anyone fork it and a competing engine could displace it.
Main trade-off
It has no friendly interface in the core project, so most newcomers meet it through another app.
No paid plans and no accounts. The only cost is your own hardware and power. Checked on llama.app and the GitHub repository 2026-10-07.
llama.cpp is an MIT-licensed C/C++ engine that runs language and vision models on local hardware, with more than 130,000 GitHub stars as of 7 October 2026. It supports 1.5-bit to 8-bit quantization and backends for CUDA, Metal, Vulkan, HIP and SYCL, and its llama serve command exposes an OpenAI-compatible API.
Runs on almost any hardware
Plain C/C++ with no required dependencies and backends for CUDA, HIP, Metal, Vulkan, SYCL and CPU instruction sets from AVX512 to ARM NEON.
Quantization from 8-bit to 1.5-bit
Integer quantization in GGUF files cuts memory use so larger models fit on ordinary machines, at some cost in output quality.
OpenAI-compatible local server
llama serve exposes OpenAI-style endpoints with streaming, tool calling and a built-in web chat page, so existing apps only need a new base URL.
Hybrid CPU and GPU inference
Splits a model across video memory and system memory so models larger than your VRAM still run, more slowly.
One-command model download
llama cli -hf pulls a GGUF model straight from a Hugging Face repository and starts a chat, with no separate model manager.
Llama menu bar and tray app
A 1 MB Mac and Windows app from the llama.cpp team and Hugging Face that picks settings per machine and unloads models after 5 minutes idle.
Most users expect these and they are not currently supported.
| Tool | Price from | Free plan | Runs on | Rating |
|---|---|---|---|---|
| Free | Yes | Mac · Windows | -- | |
| $20/mo | Yes | Web · Mac · Windows | 4.0 | |
| $20/mo | Yes | Mac · Windows | -- | |
| Free | Yes | Mac · Windows | -- |
MIT
Nothing. llama.cpp and the new Llama desktop app are free and open source under the MIT licence, with no paid tiers, accounts or usage fees. You pay for the computer and electricity that run the model, and for any API you choose to connect it to.
Yes. The engine runs on CPU alone, using AVX, AVX2, AVX512 or ARM NEON instructions, and can split a large model between CPU and GPU when video memory runs short. Expect modest speed on a laptop without a graphics card, and plan on smaller or more heavily quantized models.
Ollama adds a model library and background service on top of the same engine, LM Studio offers a desktop model browser, and Jan and GPT4All give a simple chat window. Open WebUI and AnythingLLM add team workspaces and document chat. Choose llama.cpp when you want direct control or the newest model support.
Both use the same core engine, so raw speed on one model is usually close. The difference is control: llama.cpp exposes every setting and tends to receive new model support first, while Ollama hides most settings behind defaults and a model library. Measure on your own hardware before assuming a gap.
On Windows run the install script from llama.app in PowerShell, or use Winget or the pre-built binaries on the releases page. Then run llama cli with a Hugging Face model repository to download and chat, or llama serve to start the local API. Mac and Windows users can instead install the Llama app.
llama.cpp is an open-source inference engine written in plain C and C++ that runs language and vision models on your own hardware. It is a developer tool: you install a command line program, point it at a model file and either chat in the terminal or start a local server. The code is MIT licensed, the GitHub repository holds more than 130,000 stars as of 7 October 2026, and it sits underneath many of the friendlier apps in this category, including Ollama and LM Studio.
The design goal is to run a model with minimal setup on a wide range of machines. The engine has no required dependencies, treats Apple silicon as a first-class target through Metal and Accelerate, and ships backends for CUDA on Nvidia, HIP on AMD, Vulkan, SYCL for Intel GPUs and several others. Models are stored in the GGUF format and can be quantized from 8-bit down to 1.5-bit to fit into less memory, and a model larger than your video memory can be split between GPU and CPU. A single command such as llama cli -hf followed by a Hugging Face repository name downloads and runs a model from Hugging Face.
For applications, llama serve starts an OpenAI-compatible HTTP server with a built-in web chat page, so tools such as Aider, LangChain and LlamaIndex can talk to a local model by changing one base URL. In 2026 the llama.cpp team and Hugging Face also released Llama, a menu bar and system tray app for Mac and Windows that wraps the engine, picks settings for your machine and exposes the same local API. The vendor lists it as a 1 MB download.
Who it suits: developers building on local models, people who want the newest model support and the most control over settings, and anyone who needs to squeeze a model onto limited memory. If you want a polished model browser or document chat without a terminal, Jan, GPT4All, AnythingLLM and Open WebUI are friendlier starting points, and our guide to local coding models covers which models are worth running.
There is no price. Everything is free and open source, and the real cost is the computer and electricity you run it on. Pre-built binaries, Docker images, Winget and a Linux install script are available, or you can build from source.
Product facts on this page (pricing, plans, integrations, compliance) are read from ggml-org's own pages and documentation. The verdict is Hokai's reading of them, not the vendor's claim.