Google's encoder-free Gemma 4 12B fits on a laptop

Google just released Gemma 4 12B, an open-source multimodal model designed to run agentic workflows directly on everyday laptops.

Positioned between the edge-friendly E4B and the larger 26B MoE, it fills the mid-sized gap while serving as Google's first model in this tier to feature native audio input. The launch comes as Gemma 4 models surpass 150 million downloads across the community.

What it can do

In plain terms, Gemma 4 12B handles vision, audio, and multi-step reasoning on consumer hardware without breaking a sweat.

On standard benchmarks, it delivers performance nearing Google's 26B MoE model. The real draw is efficiency: it achieves this at less than half the total memory footprint.

You only need 16GB of VRAM or unified memory, meaning a standard modern laptop can run multimodal agents locally. It also comes equipped with Multi-Token Prediction (MTP) drafters to reduce latency during local inference.

Ditching the encoders

Traditional multimodal models rely on separate vision and audio encoders to translate incoming data before passing representations to the LLM backbone. Those split encoders add latency and eat up memory.

Gemma 4 12B drops external encoders entirely, routing vision and audio inputs directly into the language model core.

For vision, Google replaced the conventional encoder with a lightweight embedding module consisting of a single matrix multiplication, positional embedding, and normalizations. The LLM backbone takes over the actual visual processing.

Audio processing is even simpler. There is no audio encoder at all. Raw audio signals are projected directly into the same dimensional space as text tokens, treating voice and text as native peers.

How to run it

Google released both pretrained and instruction-tuned weights under an Apache 2.0 license, available directly on Hugging Face and Kaggle.

If you just want to test it out, you can launch it with a few clicks in LM Studio, Ollama, the Google AI Edge Gallery app, or the LiteRT-LM CLI.

Developers can plug it right into llama.cpp, MLX, vLLM, SGLang, and Hugging Face Transformers, or fine-tune it efficiently using Unsloth. Google also rolled out an official Skills Repository to give local agents plug-and-play capabilities.

Why it matters

For local AI developers and power users, Gemma 4 12B hits an elusive sweet spot. Running multimodal agents locally used to require either stitching together multiple models that max out hardware, or settling for weaker intelligence. Fitting capable multimodal reasoning into a 16GB laptop genuinely expands what individuals can build on personal devices.

We think Google's encoder-free architecture is a sharp engineering move. Proving that an LLM backbone can directly process vision and audio without dragging along heavy encoder baggage will likely push the wider open-source ecosystem to reconsider split architectures.