Last updated: 2026-10-14
Cactus is an open-source inference engine from YC-backed Cactus Compute that runs language, speech and vision models on phones and wearables. Its README reports Gemma-4-E2B at 4-bit decoding 37 tokens per second on an iPhone 17 Pro using 644 MB of RAM. App developers can call it from Python, Rust, Flutter and mobile native code.
About Cactus
Cactus comes from Cactus Compute, a Y Combinator Summer 2025 company based in San Francisco. The project is an open-source inference stack that keeps a model on the user's hardware, whether that is a handset, a smartwatch or a laptop, and exposes OpenAI-compatible APIs so an app can swap a cloud call for a local one. The code is a stack of four layers: an engine, a zero-copy computation graph, ARM kernels and a rotation-based quantizer that compresses weights from 4 bits down to 1 bit.
The mechanism that sets it apart from desktop runners such as Ollama, llama.cpp and LM Studio is hybrid routing. A probe of about 65,000 parameters reads the local model's hidden states and returns a confidence score with every answer. Above your threshold the on-device answer is used. Below it, the same prompt goes to a larger cloud model. Cactus publishes checkpoints for this on Hugging Face, including post-trained versions of Google's open Gemma 4 models that hand off to Gemini 3.1 Flash-Lite.
It suits mobile and embedded developers who need tool calling, transcription or small vision tasks to work offline. Its own models target that gap: Needle is a small function-calling model, and Whistle is a 16.9 MB speech recognition model for seven languages that runs on CPU only. App bindings cover Apple and Android platforms plus Flutter, React Native, Python and Rust, and Pebble says it runs Needle locally in its ring app.
The engine is free to use under a permissive license published in the repository. Cactus does not publish prices for cloud handoff or Needle fine-tuning on its site, so budget for those only after talking to the team. If you want a desktop chat app instead of an SDK, Jan, GPT4All, AnythingLLM or Msty are closer matches. Liquid models are among the families the project tests, and Liquid AI is on HokAI as a company.
Pricing
The engine and CLI are free and open source. Cactus lists no prices on its site: the /pricing URL returns a 404 and the cloud and fine-tuning rates are blank.
Key Features
- OpenAI-compatible engine: The C engine offers chat completion, streaming, tool calling, transcription, embeddings, RAG and vision behind OpenAI-style APIs.
- Rotation-based quantization: Weights compress from 4-bit down to 1-bit, and the README shows Gemma-4-E2B at 4-bit scoring 71.20 on GSM8K against 73.67 at FP16.
- Hybrid cloud handoff: A probe of about 65,000 parameters scores confidence per answer, and prompts below your threshold route to a cloud model.
- Six language bindings: Swift, Kotlin, Flutter, React Native, Python and Rust bindings ship in the repository for app developers.
- Tiny task models: Needle covers on-device function calling, and Whistle transcribes seven languages in 16.9 MB on CPU.
- Built-in benchmark command: The cactus benchmark command measures speed on a connected iPhone or Android device with an optional flag.
Pros
- Runs the hard-query fallback for you: confidence comes back as structured data with every generation, so routing is one if-statement.
- Mobile first design means it targets phones and wearables, where desktop runners such as Ollama do not ship app bindings.
- Source and model checkpoints are public, so you can inspect the kernels and quantizer before shipping them in an app.
Cons
- No public price list: cloud handoff and Needle fine-tuning rates are unpublished, so cost planning needs a conversation with the team.
- The homepage cites 20k GitHub stars while the repository page shows about 6.1k, so treat vendor popularity claims with care.
- Any Hugging Face model can be converted, but the README labels conversion experimental and only certain families are tested.
Frequently Asked Questions
How much do you pay for Cactus?
Cactus publishes no price list. The engine and command line tool are free and open source, the site has no pricing page at all, and charges for cloud handoff or Needle fine-tuning are unstated. Ask the team at founders@cactuscompute.com before planning a budget around the hosted parts.
Is Cactus free to use?
The engine, the CLI and the published model checkpoints can be used without paying, under the permissive license in the repository. Creating an account is only needed to fine-tune and deploy Needle. The cloud model you hand off to is billed by its own provider.
What should you use instead of Cactus?
Use Ollama or LM Studio for a local model server on a laptop. Use llama.cpp when you want the widest model compatibility and are comfortable building from source. Hugging Face hosts the model files all of these runners load.
Is Cactus better than Ollama?
For an app that ships to phones or wearables, yes, because Cactus has Swift, Kotlin, Flutter and React Native bindings and a confidence-based cloud handoff. For a desktop model server with a large public library, Ollama is simpler. The two solve different deployment problems.
How long does it take to get going with Cactus?
On a Mac, install with brew install cactus-compute/cactus/cactus and run cactus run, which downloads a model on first use. Mobile work means adding a binding such as Swift or Kotlin to your project. Running cactus benchmark with the iOS or Android flag then measures speed on a connected phone.
Top Alternatives
- Ollama: Pick Cactus if you are building a phone or wearable app, and Ollama if you want a local model server on a desktop.
- llama.cpp: Choose Cactus when you want built-in confidence routing to a cloud model, and llama.cpp when you want the broadest model support.
- LM Studio: Choose Cactus to embed inference in your own app, and LM Studio to chat with local models in a desktop interface.