Berkeley and Stanford researchers build GEPA to optimize LLMs without costly RL

Training an AI system to master a specialized task usually involves a brutally expensive routine: run it thousands of times, slap a simple numeric score on each attempt, and let reinforcement learning slowly brute-force a solution.

A team of researchers from UC Berkeley, Stanford, and Databricks just came up with a much smarter shortcut called GEPA.

Instead of treating systems like black boxes guided by bare-bones scalar rewards, GEPA uses an LLM's own reading comprehension to diagnose its own errors and rewrite its instructions. The method beats established approaches while slashing trial runs by up to 35 times.

Fixing prompts with plain English, not math scores

Modern enterprise AI rarely boils down to a single prompt. Teams are increasingly building "compound AI systems" that link models to code interpreters, databases, and multi-step agent workflows.

Optimizing these setups with reinforcement learning methods like GRPO—the technique famously leveraged in models like DeepSeek-R1—is punishingly inefficient. Because the optimizer only sees a sparse score like 7/10, it often requires tens or hundreds of thousands of trial rollouts to inch in the right direction. When every single run burns API tokens or invokes costly tools, that trial-and-error math falls apart quickly.

As co-author and UC Berkeley doctoral student Lakshya A Agrawal pointed out, RL is simply impractical for most teams due to cost and complexity, leaving developers stuck tweaking prompts by hand. Furthermore, top-tier frontier models cannot be fine-tuned directly anyway.

GEPA, short for Genetic-Pareto, replaces sparse rewards with detailed natural language feedback. Because reasoning steps, tool outputs, and execution errors can all be serialized into text, an LLM can read the entire trace of what went wrong.

If a coding agent breaks, the model inspects the compiler error trace directly and naturally concludes that the prompt needs to mandate a specific library version. It mutates the prompt intelligently rather than guessing blindly.

Dodging dead ends with specialist pools

GEPA also tackles another classic optimization trap: getting stuck in local optima by only chasing a single highest-scoring candidate.

The framework adopts Pareto-based selection, maintaining a diverse pool of "specialist" prompts. Even if a prompt doesn't achieve the highest average score across the board, it is kept if it outperforms everything else on a particular difficult edge case.

By pulling from this diverse lineup of winning strategies, the system explores solutions that generalize far more reliably across unpredictable inputs.

Agrawal noted that the real leverage comes down to "feedback engineering"—extracting the richest possible diagnosis out of every single rollout.

Why it matters

We think GEPA exposes a practical reality that gets drowned out by the frontier RL hype: for most real-world teams, prompt-level optimization on compound systems is vastly more economical than model fine-tuning.

If you are an engineer stringing together autonomous agents or data extraction pipelines, this technique is worth watching closely. It allows you to dramatically boost performance on top-tier proprietary models without renting custom GPU clusters or managing RL infrastructure.

Slashing required trial rollouts by up to 35 times is not just an academic milestone; it directly slashes enterprise API bills and shrinks iteration cycles from weeks to hours.

That said, GEPA's success hinges entirely on the quality of your execution traces. If your agent fails silently or your tools return vague error messages, the LLM has nothing useful to reflect on. But where rich logs exist, letting the model read its own diagnostic notes looks far smarter than waiting for thousands of blind reward signals.