IBM launches z17 mainframe to bring AI straight to enterprise data

While everyone is busy debating cloud GPUs, the machines that quietly run the world's banks just got a massive AI upgrade.

IBM just announced the z17, the next generation of its iconic enterprise mainframe. Developed over five years with input from more than 100 enterprise clients, the new system is built around a simple premise: don't move your data to the AI; bring the AI directly to where the data lives.

Sub-millisecond inferences on Telum II

At the heart of the z17 is IBM's new Telum II processor, featuring a second-generation on-chip AI accelerator. With a 40 percent increase in cache, higher clock speeds, and extra compute capacity, the z17 can churn through more than 450 billion inferencing operations per day at a sub-one-millisecond latency.

That translates to 50 percent more daily AI inference operations than the previous z16. In practice, this means financial institutions can now score 100 percent of transactions for fraud in real-time, rather than relying on batch jobs or sampling.

IBM is also preparing a companion expansion card called the Spyre Accelerator, arriving in Q4 2025 via PCIe. Working alongside Telum II, Spyre is designed to run generative AI workloads and agents—specifically IBM's Granite models—directly inside the mainframe.

AI operations and a refreshed OS

The software stack is getting a companion overhaul. IBM previewed z/OS 3.2, set for release in Q3 2025, which adds native support for hardware-accelerated AI, modern data access methods, and NoSQL databases.

For systems management, IBM announced Z Operations Unite, launching in May 2025. It formats metrics and logs using OpenTelemetry, letting operations teams troubleshoot system anomalies through an AI-powered chat interface via watsonx Assistant for Z.

Why it matters

Mainframes are often dismissed as legacy tech, but they still power the backbone of global banking, insurance, and healthcare. IBM's move here is shrewd: instead of convincing risk-averse institutions to pipe their crown-jewel data into third-party cloud environments, it brings low-latency AI inference directly onto the secure hardware they already trust.

For enterprise customers, zero-data-movement AI with sub-millisecond response times removes the compliance and latency headaches that usually stall enterprise AI deployments. The real-world use cases—ranging from loan risk mitigation to real-time retail crime detection—are practical and immediately valuable.

However, this remains a very expensive playground for deep-pocketed enterprises. And while day-to-day operations get easier with watsonx assistants, full generative AI capabilities still depend on the Spyre accelerator shipping late in the year. Still, IBM is proving that on-premise, highly secure infrastructure has a durable place in the AI era.