Meta and Qualcomm team up to bring Llama 2 to your phone

While Nvidia rode the generative AI wave to a massive 220% stock surge this year, mobile chip giants like Qualcomm were left watching from the sidelines, gaining just 10% and lagging well behind the Nasdaq.

Now, Qualcomm is making its move to bring the AI party straight to your pocket.

The chipmaker announced a partnership with Meta to run its brand new open-source large language model, Llama 2, directly on Qualcomm-powered phones and PCs starting in 2024.

Squeezing the cloud into your pocket

Up until now, massive AI models have basically lived inside warehouse-sized server farms packed with power-hungry GPUs. Running them in the cloud is wildly expensive, which is why everyday usage usually requires costly subscriptions or API bills.

Qualcomm's counter-strategy is on-device AI. By shifting computation to local hardware, companies can slash operating expenses while delivering faster, latency-free virtual assistants that work without an internet connection.

Meta's open-weights approach makes this possible. Unlike OpenAI's GPT-4 or Google's Bard, which keep their model weights tightly guarded secrets, Meta lets anyone inspect and modify Llama 2. That allows engineers to trim the fat, compress the model into a lightweight footprint, and shove it onto mobile silicon.

Qualcomm's chips already pack tensor processor units designed specifically for AI math, and the two companies have deep ties from designing chips for Meta's Quest headsets.

Still, physics is physics. A mobile processor running on battery power simply cannot match the raw horsepower of a rack of cutting-edge data center GPUs, a reality underscored by Qualcomm's previous demos where running Stable Diffusion locally was painfully sluggish.

Why it matters

For everyday users, on-device models mean real privacy and zero ongoing API costs. Your personal assistant can read your messages and organize your life without beaming your private data to a corporate server farm. While we have to wait until 2024 to see real devices, this opens up massive opportunities for app developers who have been priced out of cloud-based LLMs.

For the broader industry, this pushes the AI arms race beyond cloud infrastructure and into consumer hardware. Qualcomm is effectively forcing Apple, MediaTek, and PC makers to accelerate their edge AI roadmaps, turning model compression and energy efficiency into top-tier competitive battlegrounds.

We think this is Qualcomm's smartest possible play. Competing with Nvidia in data centers is a lost cause, but Qualcomm already owns billions of consumer endpoints. The real question is whether Qualcomm's silicon can deliver snappy, ChatGPT-like experiences without cooking your battery, or if it will feel like a clunky tech demo running in slow motion.