Meta drops SAM 3 and SAM 3D with text-driven tracking and 3D reconstruction

SAM 3 and SAM 3D are Meta’s latest computer vision tools. SAM 3 lets you pinpoint and track objects across images and video using flexible natural language prompts, while SAM 3D can take a single 2D photo and turn it into a realistic 3D asset.

What it does: smarter tracking and one-shot 3D

Earlier vision models usually stumbled on anything beyond simple labels like car or chair. SAM 3 handles open vocabulary prompts out of the box, meaning you can ask for a red baseball cap or team it with multimodal LLMs for negative reasoning, like people sitting down but not wearing a red cap.

It handles video tracking seamlessly and allows text-based editing on the fly, like swapping a blue shirt for a red one in a single pass.

SAM 3D takes this a step further by reconstructing what it sees into 3D. The system splits the workload between SAM 3D Objects for items and scenes, and SAM 3D Body, which estimates human anatomy and body shape from flat photos. The resulting models can be dropped straight into game engines, AR setups, or video projects.

Access and open-source status

Anyone can try both tools today in the new Segment Anything Playground without setting up a local environment. You upload a clip or image, prompt the cuts, or spin the camera around a scene in 3D to rearrange objects and test effects.

When it comes to weights, SAM 3 is fully open-sourced alongside its code, benchmark datasets, and research paper. SAM 3D, however, is not fully open yet. Meta is currently showcasing it in the browser playground and rolling it into consumer features like View in Room on Facebook Marketplace.

Why it matters

If you build automated video pipelines, creative tools, or game assets, SAM 3 is an immediate upgrade over manual rotoscoping and rigid bounding boxes. Being able to isolate moving targets using descriptive text removes hours of tedious masking.

We think the real leap here is how fast 2D-to-3D workflows are maturing. Even with SAM 3D’s weights kept under wraps for now, turning a casual snapshot into an interactive 3D mesh proves that expensive multi-camera scanning rigs are quickly becoming optional for prototyping.