Google shrinks Gemma 4 to under 1GB for phones

Google just released Gemma 4 QAT, a collection of Quantization-Aware Training checkpoints tailored to run smoothly on smartphones, laptops, and everyday consumer GPUs. The biggest headline here: the Gemma 4 E2B text-only checkpoint fits into under 1GB of memory without falling apart.

Running models locally usually requires aggressive compression. But traditional post-training quantization simply squashes a finished model, which often trashes its reasoning skills. QAT simulates that low-precision environment right inside the training loop, teaching the weights to adapt so you get tiny memory footprints with way less quality degradation.

How it squeezes into 1GB

Alongside standard Q4_0 checkpoints, Google built a mobile-specific quantization schema specifically for the edge models, E2B and E4B.

Instead of crunching the whole network evenly, it gets surgical. Google aggressively compressed the specific layers responsible for token generation down to 2-bit, while keeping core reasoning layers at higher precision. It also pre-calculates scaling settings during training rather than forcing mobile chips to compute activations on the fly, speeding up decode times.

Data layouts are structured around native channel-wise operations supported by mobile neural accelerators, eliminating slow runtime workarounds. On top of that, Google targeted the vocabulary embeddings and KV cache to stop long conversations from chewing through active RAM. Strip out the vision and audio encoders, and the E2B text model easily sits under 1GB of total memory.

How to run it

The weights are already live on Hugging Face across multiple formats depending on your stack.

For local desktop testing, you can grab ready-to-run GGUF files in llama.cpp, Ollama, or LM Studio. Apple Silicon users can tap MLX optimizations, while server workloads can deploy compressed tensors straight into vLLM or SGLang.

For on-device mobile rollouts, Google supports its lightweight LiteRT-LM runtime, alongside browser-based Transformers.js. If you plan to fine-tune, both Unsloth and Hugging Face Transformers work out of the box, with dedicated checkpoints that also preserve Multi-Token Prediction speedups.

Why it matters

If you build on-device apps or tinker with offline assistants on budget hardware, this is an immediate download. Community post-training quantizations below 4-bit are notoriously brittle, often reducing sub-3B models to incoherent babble; getting official, training-integrated QAT from Google delivers a vastly more dependable baseline on underpowered chips.

It also sets a smarter precedent for edge AI. Instead of merely chasing smaller parameter counts that lack basic reasoning, Google proved that co-designing quantization with mobile silicon constraints, vocabulary footprint, and KV caching makes 2B-scale models practical on ordinary phones.