Bonsai-2 crunches 27B models down to 6GB with ternary weights

Bonsai-2 is an open-source collection of radically compressed models released by prism-ml. It shrinks a 27B-parameter base model down into less than 6GB, allowing you to run a solid mid-sized model locally on mid-tier consumer GPUs or a standard Mac.

How ternary compression works

Normally, neural networks crunch through very long decimal weights, which rapidly eats up all available VRAM.

Bonsai-2 takes an aggressive approach using ternary weights. That means every single numerical weight in the network is restricted to just three possible values: -1, 0, or +1.

This dramatically lowers the memory footprint. For comparison, the original base model sits at a hefty 55.6GB.

Using this new ternary method, the team compressed the entire model footprint down to just 5.9GB, making it over nine times smaller than the uncompressed original.

Crucially, across various benchmark tests, its performance remains remarkably close to the full model despite the massive diet.

Specs and getting it running

prism-ml released several tiers depending on how much memory you have to spare.

If you want the smallest possible footprint, the Q1 version comes in at just 5.95GB.

If you want slightly higher generation quality, you can grab the Q2 version, which still sits at a compact 7.21GB.

They also released an MLX version specifically tailored for Apple silicon devices, making local execution seamless on Mac unified memory.

The entire collection is open and available on Hugging Face under prism-ml/bonsai-2.

Why it matters

For local AI enthusiasts and developers, Bonsai-2 removes a massive hardware barrier. Getting 27B-class reasoning on an everyday consumer GPU without paying for expensive cloud instances is an undeniable win.

It also shows that extreme ternary quantization does not have to wreck a model's capabilities. If you have been squeezed by tight VRAM limits, this collection is definitely worth pulling down to test.