Apple's new Mac Studio can run 600B AI models locally

If you wanted to run a massive, hundred-billion-parameter open-source AI model locally, your options used to be loud server racks or thousands of dollars in commercial GPUs.

Apple just offered a surprisingly compact desktop alternative.

Apple introduced the redesigned Mac Studio, packed with the M4 Max and the brand-new M3 Ultra chip. Pre-orders kick off immediately, with shipments arriving March 12.

Half a terabyte of memory inside a tiny aluminum cube

The real showstopper here is the M3 Ultra. Apple built it using its UltraFusion packaging, linking two M3 Max dies across more than 10,000 high-speed traces with 2.5TB/s interprocessor bandwidth, cramming 184 billion transistors onto the board.

On the hardware front, it maxes out at a 32-core CPU with 24 performance cores, an 80-core GPU, a 32-core Neural Engine, and over 800GB/s of unified memory bandwidth. That CPU packs 50 percent more performance cores than any previous Ultra silicon.

Then there is the memory. The M3 Ultra configuration starts at 96GB and scales all the way up to 512GB of unified memory, paired with up to 16TB of internal SSD storage. It is the largest unified memory capacity ever shipped in a personal computer.

Because the GPU directly shares that memory pool, Apple claims the Mac Studio can run large language models with over 600 billion parameters entirely in memory. In LM Studio, Apple says token generation speeds on massive models are up to 16.9 times faster than the M1 Ultra.

For creators who do not need workstation-class scale, the M4 Max version is no slouch. It supports up to a 16-core CPU, a 40-core GPU, up to 128GB of RAM, and compiles Xcode code up to 2.1 times faster than the M1 Max.

Thunderbolt 5 and eight 6K displays

Connectivity gets a massive overhaul with Thunderbolt 5, delivering data speeds up to 120Gb/s. That is roughly three times faster than Thunderbolt 4, backed by custom dedicated controllers integrated straight into the silicon for each port.

With an M3 Ultra under the hood, the Mac Studio can drive up to eight 6K Pro Display XDR monitors simultaneously. You also still get a 10Gb Ethernet port, HDMI, front-facing SDXC card slot, and macOS Sequoia with Apple Intelligence.

Why it matters

For AI researchers, engineers, and boutique creative studios, this desktop solves a massive headache. Running massive frontier models locally used to require multi-GPU setups that demand industrial power delivery and dedicated air conditioning.

While Apple silicon does not match enterprise Nvidia clusters in pure raw matrix compute throughput, unified memory changes the game. Being able to fit a 600B parameter model on a quiet desktop machine for a fraction of enterprise server costs is a breakthrough for local AI research.

We think pairing the M3 Ultra with the M4 Max in the same lineup is a very practical move. Apple did not wait around for an M4 Ultra to address the high-memory void. Instead, it linked two M3 Max chips to immediately deliver half a terabyte of RAM and Thunderbolt 5.

If you are a video editor or developer, the M4 Max tier will deliver more speed than you will likely ever tap into. But if your goal is self-hosting massive local models without renting cloud clusters, the 512GB M3 Ultra stands completely in a league of its own.