Intel unveils Gaudi 3 to take on Nvidia's H100

If you have spent the past year watching companies fight over every single Nvidia GPU they can get their hands on, Intel wants you to know there is another option on the table.

At its Vision 2024 event in Phoenix, Intel officially unveiled the Gaudi 3 AI accelerator. The entire pitch was crystal clear: take aim directly at Nvidia’s market-dominating H100 data center GPU.

Under the hood of Gaudi 3

Gaudi 3 builds directly on the architecture of Gaudi 2, but it makes a key leap by switching from TSMC’s 7nm process to a 5nm node.

Physically, the processor uses two identical silicon dies linked by a high-bandwidth connection. Across those two dies, you get 64 programmable tensor processor cores, eight matrix multiplication engines, and 96MB of shared central SRAM cache.

On the memory side, Intel packed in up to 128GB of HBM2e memory delivering 3.7 TB/s of bandwidth, alongside a hefty 900W TDP.

According to Intel, the real payoff is speed. In AI training workloads running OpenAI’s GPT-3 175B and Meta’s Llama 2 7B, Intel projects Gaudi 3 will be roughly 50 percent faster than an H100, reaching up to 1.7x faster in specific setups. For inference workloads running models like Llama 2 and Falcon 180B, Intel claims an identical 50 percent speed advantage.

Intel is also heavily pitching efficiency, claiming Gaudi 3 is up to 2.3 times more power-efficient than the H100.

Air-cooled samples are already shipping to partners, with general availability scheduled for Q3. Liquid-cooled variants are planned for Q4.

The moving target problem

Intel’s claims look formidable on paper, but there are a few important catches that the industry immediately picked up on.

First, Intel did not specify exactly which H100 configuration or system-level hardware it tested against, leaving some benchmark nuances in question.

Second, the H100 is effectively a moving target. While it remains the undisputed king of active data centers right now, Nvidia already has newer iron waiting in the wings. The H200 is slated to start shipping in Q2, and the Blackwell B200 was announced recently.

By the time Gaudi 3 hits volume deployment in the third and fourth quarters, Nvidia’s flagship lineup will look very different.

Pricing also remains a blank canvas. Intel did not disclose how much Gaudi 3 will cost. An H100 currently sells for an estimated $30,000 to $40,000, while rival AMD’s Instinct MI300 series reportedly commands roughly $10,000 to $15,000.

Why it matters

For enterprise tech leaders and AI engineers, chip supply has been a continuous bottleneck. Everyone from Microsoft and Meta to Google has either deployed custom silicon or worked on in-house designs just to avoid sitting on Nvidia waiting lists.

Gaudi 3 gives those desperate buyers a viable alternative right now. If Intel can price this competitively near AMD’s range and actually deliver steady supply, enterprise customers will gladly snap them up for model deployment and fine-tuning.

In our view, Intel's decision to benchmark against the H100 rather than wait for Blackwell is clever commercial pragmatism. Enterprises need working silicon today, not paper specs scheduled for next year.

The massive caveat, however, remains software. Raw silicon speed is only half the battle; Nvidia’s true monopoly lives in CUDA and its mature developer ecosystem. Convincing engineering teams to rewrite pipelines for Intel’s software stack will be the real hurdle, no matter how good the hardware benchmarks look.