OpenAI's new Decisions API cuts latency 10x

If you've ever built an AI agent, you know the pain of waiting two seconds just to have a giant model tell you 'yes' or 'no.' You end up paying for generated tokens you didn't even want, just to route a simple ticket.

OpenAI wants to fix that plumbing problem. The company just made its Decisions API generally available, powered by GPT-6 Luna. It's built specifically for tiny, frequent questions with bounded answers, clocking in at up to 10x faster than calling GPT-6 Luna through the standard Responses API.

Stripping out the output bloat

The setup is refreshingly straightforward. You define a question, provide the allowed answers, pass in your context as text, images, or native JSON, and get back one of three clean results: a probability, a predefined choice, or a weighted score.

OpenAI doubled the batch capacity to 400 questions per request. Even better, the pricing model kills the output tax entirely. Input runs at $0.10 per million tokens, falling to $0.01 per million if cached, with zero charge for output tokens.

There's also native integration with GPT-Live for real-time voice apps. That means a digital avatar can pick its next facial expression while listening to you, or a support bot can decide whether to escalate a call mid-sentence without stalling the conversation. Regional data residency in the US and EU, zero data retention for eligible users, and extra capacity across Asia are also in place.

Why it matters

We think this is one of the smartest developer moves OpenAI has made lately. Most workflow bottlenecks in agentic systems don't come from deep reasoning; they come from endless minor classifications like intent routing, spam checks, and tool selection. Having frontier-level intelligence act as a near-instant micro-switch solves a massive architectural headache.

Killing output fees is also a shrewd business play. OpenAI is acknowledging that when developers just need structured decisions, making them pay for JSON formatting tokens is pure friction. By optimizing inference specifically for bounded choices, they make running agents on OpenAI infrastructure radically cheaper and faster than stitching together smaller open-source models.

The catch is vendor lock-in. Once you weave an application's internal routing and decision trees tightly around a proprietary endpoint, migrating away becomes a nightmare. Still, if your agents are currently bogged down by sluggish branching logic, testing this in your pipeline is an easy call.