Microsoft rolls out Windows ML for local AI

Shipping local AI features on Windows used to be a packaging nightmare. Developers had to write custom code for AMD, Intel, Nvidia, and Qualcomm silicon, then bundle bloated vendor runtimes into their installers—easily inflating app sizes by hundreds of megabytes just to avoid breaking on someone's setup.

Microsoft officially announced the general availability of Windows ML. First introduced at Build 2025, the built-in AI inferencing runtime is now ready for production use, aiming to turn Windows 11 into a unified, silicon-agnostic platform for on-device AI.

Offloading hardware headaches to the OS

Windows ML acts as a hardware abstraction layer built on top of ONNX Runtime (ORT). Developers can stick to familiar ORT APIs and standard ONNX model formats, or optimize raw PyTorch models directly using the AI Toolkit for VS Code.

The real relief comes in distribution. Instead of forcing developers to ship bulky runtime binaries, Windows now handles the delivery, registration, and maintenance of Execution Providers (EPs) directly through the operating system. When a user runs an app, Windows detects the hardware and pulls down the right EP dynamically, slicing dozens to hundreds of megabytes off installer footprints.

Silicon vendors are all aboard with dedicated execution providers. AMD integrated its Vitis AI EP across Ryzen AI NPUs, GPUs, and CPUs. Intel tied in its OpenVINO framework for Core Ultra chips. Qualcomm tuned its QNN EP for the Snapdragon X Series and promised support for the upcoming Snapdragon X2 platform. Meanwhile, Nvidia delivered a dedicated TensorRT for RTX execution provider, which it claims delivers over 50% faster inferencing on RTX GPUs compared to DirectML, reaching an installed base of over 100 million devices. Developers can also define execution policies to dictate whether a task prioritizes battery life on the NPU or peak speed on the GPU.

Early adopters show real-world utility

Major software vendors are already shifting real workloads onto the runtime.

Adobe is integrating Windows ML into Premiere Pro and After Effects, using local NPUs for accelerated semantic search in media libraries, audio categorization, and scene edit detection, with plans to migrate all on-device models to the framework over time. Wondershare's Filmora is tapping NPU acceleration across AMD, Intel, and Qualcomm chips to preview real-time body effects like neon rings and particle bursts.

Security and utility apps are leaning into local execution for privacy and responsiveness. McAfee is using Windows ML to run local detection for deepfake videos and social scams, while BUFFERZONE analyzes webpage threats without sending data to the cloud. Other implementations include Camo using NPUs for live webcam segmentation, Dot Vista handling offline voice navigation in clinical settings, and Topaz Labs running on-device photo enhancement.

Why it matters

We think the general availability of Windows ML represents a crucial pivot: the AI PC narrative is finally moving from empty NPU hardware hype to usable software plumbing. For over a year, PC makers have shouted about TOPS without giving developers a practical reason to build local features. By abstracting away the fragmented silicon landscape, Microsoft just removed the single biggest tax on desktop AI development.

Leaning on existing ONNX workflows instead of reinventing the wheel is a clever move. It lets desktop developers downsize their installers immediately and target millions of devices with a single build. If you maintain a Windows desktop app, testing Windows ML today is a low-risk, high-reward bet to cut cloud API costs.

The lingering challenge will be execution parity. While Microsoft promises rigorous compatibility certification, hardware bugs and inference discrepancies across different vendor EPs are notorious edge cases. Still, Windows finally has a credible answer to Apple's Core ML, and that alone lowers the barrier for desktop software to run locally by default.