Google launches PaLM 2, powering Bard and mobile devices

For the past year, the AI race felt like a crude contest of who could brag about the biggest parameter count. With PaLM 2, Google is finally changing the narrative.

At its I/O developer conference, Google officially launched PaLM 2, its next-generation foundation model. It will immediately power the updated Bard chatbot and serve as the backbone for virtually all the new AI features Google announced. Developers can already access it via the PaLM API, Firebase, and Colab.

Moving past the parameter race

Much like OpenAI with GPT-4, Google refused to disclose the exact parameter size of PaLM 2, breaking from the precedent set by the 540-billion-parameter original PaLM.

DeepMind VP Zoubin Ghahramani put it plainly, noting that larger is not always better and that raw parameter counts are no longer a useful measure of capability. Instead, Google is treating PaLM 2 as a versatile family of models across various sizes, built entirely on Google's latest JAX framework and TPU v4 infrastructure.

Google focused heavily on logic, math, and multilinguality. By training on massive corpora of scientific papers and mathematical expressions across more than 100 languages, Google claims PaLM 2 can reason through complex math puzzles and even generate diagrams without relying on external plugins.

Specialized models and on-device ambitions

Alongside the core model, Google unveiled several specialized variants tailored for specific industry workloads.

There is Codey, a model trained on 20 programming languages ranging from Python to legacy systems like Fortran and Prolog, designed to power code completion and generation services. Google also highlighted Med-PaLM 2 for healthcare and Sec-PaLM for cybersecurity operations.

Perhaps the most intriguing reveal was a compact version of PaLM 2 designed to run natively on mobile phones. Google claims it processes around 20 tokens per second.

While Google did not specify the test hardware or provide a shipping date, running a functional LLM on-device opens serious possibilities for privacy-first, offline mobile applications.

Why it matters

We think this launch marks a crucial shift in Google's AI playbook toward practical engineering rather than benchmark vanity.

For developers, modularity is the real story here. Instead of forcing everyone into an expensive, one-size-fits-all API call, offering domain-specific models like Codey alongside smaller on-device footprints gives builders actual architectural flexibility. If you are building tools with strict latency, cost, or privacy constraints, this family structure is exactly what you want to see.

For the wider industry, PaLM 2 is Google's line in the sand against OpenAI. Bard finally gets the brain upgrade it desperately needed to stay competitive against ChatGPT. Furthermore, tying the model tightly to proprietary TPU v4 silicon gives Google an internal cost structure that competitors relying entirely on Nvidia GPUs will struggle to match.

The catch is execution and transparency. Google did not let third-party evaluators kick the tires before the announcement, so we still do not know how PaLM 2 handles hallucination or edge cases in the wild. A 20-token-per-second mobile model sounds promising, but until it ships to consumer devices, it remains an impressive demo rather than an everyday reality.