GPT-4 is quietly closing its speed gap with GPT-3.5
Portkey just revealed that GPT-4 latency has steadily dropped over the past three months, effectively cutting 99th-percentile wait times by more than half. While complex prompts still spike processing, median latencies now sit comfortably below 1ms per token. It is still pricier, but speed is no longer the bottleneck holding teams back.