OpenAI gives GPT-6 prompt caching a major speed and discount boost

If you've ever built a long-running AI agent, you know the painful economics of multi-turn workflows: your agent spends hours refactoring a repo or synthesizing research, repeatedly feeding identical tool definitions and massive context histories back into the API. The token meter spins constantly, and latency compounds with every step.

OpenAI rolled out a major overhaul to prompt caching for the GPT-6 family. The upgrade delivers higher cache hit rates by default, offers up to a 90% discount on cached input tokens reused within a 30-minute window, and brings a suite of developer controls to manage cached context. According to GitHub Copilot, implementing the system cut fresh token processing by 50%.

Slaying the cache-miss mystery

Historically, optimizing API caching felt like guessing in the dark: one tiny payload tweak could quietly invalidate megabytes of context, leaving you with an unexpected spike on your monthly invoice.

OpenAI is tackling this with two new visibility tools. The Prompt Caching Dashboard lets teams track hit rates over time and inspect token composition charts to see exactly what percentage of inputs were served from cache.

Alongside the dashboard, a new diagnostics utility helps catch misses. If a cache miss occurs, the tool compares the failed call with a recent response and outputs the exact culprit—such as an altered tool schema or updated settings—along with the count of affected tokens so you know what broke reuse.

Surgical controls for reasoning and tools

Beyond monitoring, OpenAI introduced granular switches to keep cache lines warm as agent contexts evolve.

On GPT-6, developers can now adjust reasoning effort on the fly without blowing away existing cache. By appending a configuration_update, you can dial up reasoning effort for complex edge cases or dial it down for simple follow-ups while preserving reusable prefix context.

OpenAI also added explicit cache breakpoints and a prewarming feature, which lets applications load shared instructions and bulky documentation into cache at startup before the user even submits a prompt. To prevent accidental misses when tool needs change, OpenAI recommends keeping schema definitions fixed and toggling availability via allowed_tools or tool_choice parameters instead of deleting definitions.

Why it matters

We think this release signals a crucial shift from treat-the-API-as-magic to treating frontier models as manageable production infrastructure.

For developers building autonomous coding agents or enterprise research copilots, this is an immediate, practical win. Reusable context is the single biggest line item in persistent workflows; a 90% discount and a 30-minute window mean teams running multi-step agent loops can slash run costs without sacrificing model depth. If you are running long agent chains, refactoring your prefixes to take advantage of this is worth doing right away.

What makes this update particularly smart is that OpenAI addressed the fragility problem. Caching used to break at the slightest change in metadata. By introducing prewarming, explicit breakpoints, and miss diagnostics, OpenAI is effectively turning prompt engineering into traditional systems caching.

Across the industry, this puts heavy pressure on rival frontier providers. As agentic tasks stretch from seconds to hours, raw token pricing matters far less than effective cached throughput, and competitors will have to match these deep developer ergonomics to keep enterprise agent workloads on their clouds.