LM Studio gets a massive RTX speed boost

If you have ever tried running a large language model entirely on your own machine, you know the frustration: slow model loading, high latency, and fans spinning at maximum speed just to generate a few tokens per second.

NVIDIA just announced a major performance boost for LM Studio, one of the most popular free apps for running LLMs offline. With the rollout of LM Studio version 0.3.15, the app officially taps into CUDA 12.8 to unlock serious speedups across GeForce RTX PCs.

Squeezing llama.cpp for every drop of speed

At the heart of LM Studio is llama.cpp, the beloved open-source runtime that makes running local models on consumer silicon possible. NVIDIA teamed up with both the LM Studio and llama.cpp communities to inject direct GPU optimizations.

First up is CUDA Graph support. By grouping multiple GPU operations into a single CPU call, the runtime cuts down CPU overhead significantly, boosting model throughput by up to 35%.

Second is the integration of Flash Attention CUDA kernels. Transformer models typically choke on memory when dealing with long context windows. Flash Attention optimizes how attention is computed, delivering up to an extra 15% throughput boost without requiring more memory or compute.

Hardware support stretches across the board. The CUDA 12.8 runtime supports everything from GeForce RTX 20 Series GPUs all the way up to upcoming NVIDIA Blackwell-class chips. On compatible drivers, LM Studio handles the upgrade automatically.

Better controls for local agents

The 0.3.15 release also focuses heavily on developers hooking local LLMs into real workflows.

It introduces granular control over function calling through the tool_choice parameter. Developers can now force an external tool call, disable it, or let the model decide dynamically. That makes structured agent pipelines and local RAG workflows dramatically more predictable.

There is also a redesigned system prompt editor tailored for longer and more complex instructions. Since LM Studio serves OpenAI-compatible API endpoints, users regularly plug it into VS Code for local coding assistants or connect it to Obsidian via community plugins like Text Generator and Smart Connections to query private personal notes completely offline.

The app continues to support open model families like Gemma, Llama 3, Mistral, and Orca across quantization levels from 4-bit up to full precision.

Why it matters

For privacy-focused developers, researchers, and creators, running models locally is no longer just a hobbyist flex—it is a daily workflow. If you own an RTX machine, updating to 0.3.15 and enabling Flash Attention with GPU offload is an easy, no-brainer win that delivers instant speed upgrades.

Strategically, this is classic NVIDIA at its best. Instead of trying to force users into a proprietary local runtime, NVIDIA stepped in to optimize the open-source tools the community already loves, like llama.cpp. By ensuring that community software runs blisteringly fast on GeForce cards, NVIDIA makes sticking with RTX hardware the only practical choice for running local AI.

That said, local AI still faces hard hardware ceilings. Flash Attention helps, but consumer VRAM limits remain an unforgiving wall for larger models, and manual toggling of runtimes and layer offloading reminds us that local LLMs are still somewhat geared toward power users.