Cactus Needle 2 fits an offline AI model into just 28MB of RAM

Cactus Needle 2 is an ultra-tiny, 45-million-parameter model built specifically for micro-devices where full-sized LLMs could never dream of running. Instead of chasing broad general knowledge, it focuses entirely on offline tool calling and hardware control without touching a GPU.

Even the smallest local models these days tend to start at one to two billion parameters and demand a few gigabytes of VRAM. If you are building on a cheap board or an ancient smartphone, you are usually out of luck. That is exactly what Needle 2 is meant to fix.

What it does

Needle 2 is designed to act as a local command router. It parses instructions, extracts parameters, and triggers OpenAI-style function calls directly on the edge.

Because of its 45M footprint, you cannot expect deep reasoning or long-horizon agentic workflows from it. If you ask it to solve complex logic puzzles, it will fail. But for toggling smart home states, triggering API tools, or parsing short queries, it gets the job done cleanly.

In short, think of it less like an intellectual partner and more like a lightning-fast neural switchboard for your hardware.

Tiny specs and wild speeds

Hardware requirements here are almost non-existent. The entire model is packaged into a compact 14MB binary and sips roughly 28MB of RAM during inference.

That eliminates the need for discrete graphics or dedicated VRAM entirely. Even better, that microscopic footprint turns into blistering generation speed across everyday low-power hardware.

On a Raspberry Pi 5, Needle 2 decodes at around 500 tokens per second. On sub-$200 budget phones, it hits roughly 700 tokens per second. And on standalone VR headsets, decode speed can reach up to 1,500 tokens per second.

This is powered by the accompanying open-source Cactus runtime. Built in C and C++, the engine features zero-copy computation graphs, ARM NEON SIMD kernels, and custom rotation-based quantization. It also includes an automatic cloud-handoff feature that forwards requests to cloud models whenever local confidence falls below a set threshold.

How to get started

Getting started takes just a couple of CLI commands. On macOS, you can install the engine via brew install cactus-compute/cactus/cactus and launch a model with cactus run.

On Linux, clone the cactus-compute/cactus repository on GitHub, run the setup script, and fire off cactus run Cactus-Compute/needle to test tool calling with your own JSON tool definitions.

You can also launch an OpenAI-compatible local server using cactus serve, letting your other local apps talk to it over standard REST endpoints.

Why it matters

For embedded tinkerers, robotics hobbyists, and IoT builders, Needle 2 is definitely worth checking out today. It brings responsive, deterministic function calling to devices that previously could not run an AI model at all, removing both internet dependency and ongoing cloud API fees.

We think this ultra-specialized approach is remarkably practical. Instead of trying to fake human intelligence at a tiny scale, Cactus trimmed everything down to a dependable command layer. You will still need the cloud for heavy lifting, but as a local trigger sitting on cheap silicon, Needle 2 sets an impressive efficiency benchmark.