PrismML fits a 27B multimodal model onto an iPhone

Bonsai 27B is an open-source low-bit multimodal model from PrismML that shrinks a serious 27B-class foundation model down to run locally on consumer phones and laptops. Built on top of Alibaba's Qwen3.6 27B, it does away with the usual trade-off where running a capable reasoning model requires enterprise-grade cloud GPUs.

Normally, a standard FP16 27.8B model occupies roughly 54GB of memory just to load its weights. Even a traditional 4-bit quantized version hovers around 18GB, which is still a no-go for everyday mobile devices. PrismML bypassed that wall by releasing two extreme low-bit variants: a 1-bit binary model weighing just 3.9GB, and a 1.58-bit ternary model at 5.9GB.

What it does and how it gets so small

Bonsai 27B is built as a local workhorse rather than an offline conversational toy. It ships with thinking enabled out of the box, handling complex multi-step reasoning, coding, refactoring, and agentic workflows. Because it is natively multimodal, it accepts both text and images, meaning it can process device screenshots, documents, and live camera feeds entirely on-device.

To compress 27.8 billion parameters into 3.9GB, PrismML deployed an end-to-end low-bit architecture across the entire network—from embeddings and attention layers to MLP blocks and the LM head. Instead of storing weights as long floating-point decimals like 0.7342, the 1.58-bit ternary representation limits every weight strictly to {-1, 0, +1}. Basically, it only tracks whether a connection pushes a signal down, leaves it alone, or pushes it up.

Surprisingly, this brutal compression does not ruin performance. The 3.9GB 1-bit binary version is roughly 14x smaller than FP16 while retaining over 90% of the full-precision benchmark performance. The 5.9GB ternary version is about 9x smaller while retaining over 95%. Across standard benchmarks where full-precision Qwen3.6 scores around 85%, ternary Bonsai still pulls an 80.5%.

Hardware specs and where to get it

In terms of real-world speed, PrismML claims the 1-bit binary Bonsai 27B hits 11 tokens per second on an iPhone 17 Pro. On an Apple M5 Max laptop, throughput reaches 87 tokens per second for the 1-bit version and 58 tokens per second for the ternary build.

The models were trained using Google v5 TPUs and tuned for low-latency execution across consumer-grade CPUs and edge GPUs. Both the 1-bit and 1.58-bit variants are now freely available under the Apache 2.0 license, with full download links and deployment instructions posted on GitHub.

Why it matters

If you build offline tools, prioritize complete data privacy, or want an autonomous coding agent that never pings a remote server, Bonsai 27B is worth an immediate download. It challenges the assumption that running serious 27B-class models will forever belong behind cloud API paywalls.

Keep in mind that 11 tokens per second on mobile is usable but will inevitably tax phone batteries, and extreme quantization still sheds some edge-case accuracy. Still, compared to tiny 2B or 3B local models that struggle with complex logic, Bonsai 27B proves that consumer hardware can now host genuine, multimodal engineering intelligence without sending a single byte over the wire.