Report: Microsoft building custom AI chips to ditch Nvidia reliance

Running cutting-edge AI is basically paying an endless tax to Nvidia right now.

With every major tech player scrambling for compute, Nvidia's flagship H100 GPUs have been spotted flipping on eBay for over $40,000 apiece. Industry estimates suggest OpenAI alone needs north of 30,000 older A100 chips just to commercialize ChatGPT.

Faced with a massive hardware tab, Microsoft is quietly preparing an exit hatch. According to a report from The Information, the company has been secretly developing its own custom AI silicon since 2019 under the codename Athena.

Inside Project Athena

Athena isn't a rushed reaction to the recent generative AI hype. Microsoft has been chipping away at the project for years, and work has recently picked up serious momentum.

Select employees across both Microsoft and OpenAI already have their hands on the silicon. They are reportedly running performance benchmarks with top-tier models like GPT-4 to see how the hardware holds up in real-world environments.

If tests go well, Microsoft reportedly plans to roll out the chips more broadly across internal teams and OpenAI as early as next year. The company isn't stopping at a single generation either, with a multi-generation roadmap already mapped out.

Whether Azure cloud customers will ever get direct access to Athena remains up in the air. Sources noted the in-house chip is not designed to wipe Nvidia out of Microsoft's datacenters entirely, but rather to slash operational costs as AI features roll out across Bing, Office 365, and GitHub.

Custom silicon isn't totally uncharted territory for Microsoft. The company has tinkered with ARM-based server chips for years and previously partnered with Qualcomm and AMD to design custom processors for its Surface lineup.

Why it matters

By moving into custom AI silicon, Microsoft is following the playbook written by Google with its TPUs and Amazon with Trainium. But having OpenAI as an in-house tuning partner gives Microsoft a rare shortcut.

We think this is less about beating Nvidia on raw performance and more about pure economic survival. Every time a user pings Bing Image Creator or drafts an email in Office with Copilot, Microsoft burns compute cycles. Nvidia's sky-high margins are Microsoft's direct operating expense. If Athena can handle high-volume inference at even a fraction of the hardware cost, it translates into billions saved annually.

It also sends a loud message across the industry: even Nvidia's biggest, most important customer is hedging its bets against GPU shortages and pricing power.

The real test will be software. Building hardware is hard, but building the compilers, runtime libraries, and networking stack to match CUDA is where most custom chips stumble. Getting OpenAI's engineers to dogfood Athena on live frontier models right now is probably the smartest way to make sure the chip actually works where it counts.