Google frontier model PM lead shares playbooks for the AI era

Writing code with AI is getting cheaper by the day, but building the wrong product still carries a steep price. In a recent conversation on Lenny's Podcast, Google frontier model product lead Tulsee Doshi unpacked how product management has to change when working with unpredictable models.

Hands-on evals and the 60/20/20 balance

Doshi pointed out that frontier model training rarely behaves like traditional software engineering. Capabilities often emerge unexpectedly, meaning product managers cannot simply rely on public benchmark leaderboards to spot viable features. You have to actually use the models.

Instead of overcomplicating evaluation systems early on, she advises teams to curate 5 to 20 core prompts that represent what they care about most. Defining what "good" looks like by hand sets an authentic quality bar. She also warned that when upgrading to a newer model, teams need to re-evaluate their system prompts and guardrails, since old patches written to hide previous model flaws might end up handicapping newer capabilities.

To balance current delivery against future upside, Doshi proposed a 60/20/20 resource framework: dedicate roughly 60% of effort to capabilities that work reliably today, 20% to capabilities likely to land soon, and 20% to exploring early, unproven signals.

Why it matters

We think this framework cuts straight through the two biggest traps in AI development: either freezing a product around today's flaws or betting the entire company roadmap on the promise that tomorrow's model will solve everything.

While some developers on social media were quick to dismiss the advice by citing Gemini's hit-or-miss product record, the underlying methodology is sound. Rather than chasing generic benchmarks, curating a tight test suite of a dozen proprietary prompts is the most practical habit any builder can adopt right now.