OpenAI’s new trick to make GPT-4o edits faster

If you have ever watched an AI model slowly re-type hundreds of lines of unchanged code just to tweak a few variables, you know the frustration.

OpenAI just rolled out Predicted Outputs, a new API capability designed to dramatically reduce latency for gpt-4o and gpt-4o-mini.

Skipping ahead with reference text

The feature works by allowing developers to pass in a reference string alongside their prompt.

Basically, when you ask an LLM to update an existing document or file, most of the original content stays identical. By giving the model that reference text upfront, it does not have to generate every single token from scratch and can breeze through the unchanged parts.

OpenAI highlighted three core use cases where this speeds things up: updating a blog post inside a document, iterating on prior model responses, and rewriting code in an existing file.

Why it matters

Editing existing code and text is arguably the most common workflow for developer tools and AI writing assistants. Waiting for an entire file to stream back just to view a minor patch has always been a bottleneck.

We think this is a smart, targeted optimization that directly benefits code copilots and collaborative editors. If you build developer tooling, this should make your in-line edits feel noticeably snappier.

The catch is that it only supports gpt-4o and gpt-4o-mini, and you need an accurate reference string to feed it. If you are generating net-new content from scratch, it will not speed up your pipeline.