Needle 3 fits an AI agent into 8 MB

When developers talk about running AI locally, the conversation usually turns to heavyweight setups: dozens of gigabytes of VRAM, massive memory bandwidth, or at least a few gigabytes of storage just to run a heavily quantized small model. But if you just want a smartwatch to log an expense or a robot vacuum to understand "clean the kitchen but leave the bedroom," your gadget does not need a poetic genius stuffed inside it.

Cactus rolled out Needle 3, a tiny foundation model built specifically for microcontrollers, wearables, and edge hardware. The standout figure here is its physical footprint: the entire model ships as a single 8 to 29 MB binary. Packing between 29 million and 121 million parameters, Needle 3 bypasses the traditional Transformer architecture in favor of Cactus's proprietary Simple Attention Network, delivering offline intelligence in milliseconds.

Trading chat for real-world actions

Needle 3 pulls off this tiny footprint by making a ruthless tradeoff: it dumps open-ended general chat entirely. Instead, every bit of its parameter budget is channeled into tool calling, structured data extraction, and local text embeddings.

Don't expect it to write essays or answer trivia. What it does do is manipulate devices with remarkable precision. According to Cactus, Needle 3 beats models ten times its size on mobile tool calling benchmarks and matches models two to three times larger on extraction tasks. When a user speaks, it maps intent to registered functions and extracts arguments reliably. If no tool fits the request, it returns an empty list rather than hallucinating an action.

For structured extraction, developers simply pass a schema, and Needle 3 converts unstructured messy text into typed JSON. It can parse an invoice, pull booking details from an alert, or classify customer sentiment directly on a wristband, smart glasses, or an in-cabin automotive system without touching the internet.

Intelligence laddering: 2 to 20 layers

Normally, deploying AI across a fragmented hardware ecosystem means training and maintaining separate models for watches, phones, and embedded boards. Cactus tackled this with what it calls intelligence laddering.

Every layer in Needle 3 acts as a subnetwork with progressively increasing capacity. Developers can selectively activate anywhere from 2 layers up to the full 20 layers depending on the compute budget of the target chip. A low-power microcontroller can spin up just 2 layers to save battery, while a Raspberry Pi or smartphone can engage more layers for higher accuracy, all running off the same underlying weight archive.

Trained on 360 billion tokens of structured data and compressed via 2-bit CQ2 quantization, the model clocks between 1,000 and 10,000 prefill tokens per second and 400 to 4,000 decode tokens per second on a Raspberry Pi 5. The inference runtime itself sits under 1 MB, supporting macOS, Windows, Linux, Android, iOS, watchOS, and WebAssembly out of the box. Fine-tuning is equally nimble: training a 4-layer subnetwork on downstream data for just one epoch allows it to match DeepSeek V4 Flash, gaining 18 to 36 points on the DroidCall benchmark.

Why it matters

While the rest of the industry remains locked in an arms race over trillion-parameter frontier models, Needle 3 takes a refreshing, pragmatic detour. We think this approach is brilliant.

For years, IoT and smart device makers have been trapped between two bad options. They could route voice commands to cloud APIs, introducing latency, recurring server bills, and obvious privacy headaches. Alternatively, they were stuck with brittle, hardcoded keyword matching that broke the moment someone phrased a request differently. Squeezing reliable intent mapping into an 8 MB package means cheap microcontrollers can finally run private, offline, natural language control.

For hardware engineers and app developers, the friction to adopt is practically zero. You can bundle a full semantic parser directly into device firmware without managing cloud infrastructure or asking users for data permissions.

There are clear caveats, of course. Needle 3 possesses zero world knowledge and cannot handle open-ended reasoning. If your tool definitions are ambiguous or poorly structured, the model has far less wiggle room to recover than a full-size LLM. But as a dedicated software component, it is exceptionally compelling: it does not try to teach your thermostat how to write poetry, it just makes sure it never fails to turn down the heat.