OpenAI adds WebSockets to its API, cutting agent lag by 40%

For the longest time, the slowest part of using an AI coding agent was waiting for the model to think. Now that hardware has gotten blisteringly fast, developers ran into a funny bottleneck: the API itself was slowing them down.

OpenAI rolled out WebSocket support for its Responses API, cutting agentic workflow latency by up to 40% and slashing time to first token by roughly 45%.

The API overhead bottleneck

When a coding agent like Codex fixes a bug, it doesn't just answer once. It reads your codebase, edits files, runs tests, and sends tool outputs back and forth over dozens of API calls.

Under standard HTTP, every single turn required establishing a fresh connection and resending the entire conversation history, tool definitions, and context from scratch. When flagship models like GPT-5 and GPT-5.2 outputted around 65 tokens per second, that background overhead was easy to ignore.

That changed with GPT-5.3-Codex-Spark. Running on specialized Cerebras chips, the model hits over 1,000 tokens per second with bursts up to 4,000. Suddenly, the API servers processing requests became the bottleneck, forcing users to wait on CPUs before they could even hit the GPUs.

Ditching HTTP for persistent state

Instead of switching to a complex gRPC setup, OpenAI went with WebSockets to keep integration seamless for developers.

The new mode maintains a persistent connection and caches conversation state directly in memory. When an agent fires a follow-up request, it simply passes a previous_response_id. The server reuses previously rendered tokens, tool schemas, and model routing logic, meaning safety classifiers only have to inspect the newest input rather than the entire history.

The speedups landed quickly across major coding tools. Vercel reported a 40% latency reduction in its AI SDK, Cline saw multi-file workflows speed up by 39%, and Cursor noted OpenAI models running up to 30% faster.

Why it matters

This update highlights a quiet shift in AI engineering: raw model inference is getting so fast that classical networking protocols are becoming the real drag.

For developers building with agents, adopting this is a no-brainer. Multi-turn tasks that used to take several minutes can now complete significantly faster, making automated debugging feel much less like an agonizing waiting game. The integration barrier is low since OpenAI deliberately kept the familiar response.create payload instead of forcing an entirely new API paradigm.

We think this also puts pressure on competitors like Anthropic and Google. As ultra-fast inference hardware proliferates, competing purely on model speed won't cut it anymore; the surrounding transport pipelines will have to be rebuilt to match.