Meta turns flat photos into 3D models with SAM 3D

SAM 3D is Meta’s new collection of models designed to reconstruct interactive 3D assets straight from standard 2D flat photographs. The release includes two separate models: SAM 3D Objects handles everyday items and spatial scenes, while SAM 3D Body focuses on predicting human body shapes and poses.

What it does and how to try it

Most existing 3D generators stumble the moment you feed them a photo taken in messy real-world conditions, usually because they were trained exclusively on clean synthetic shapes on empty backgrounds. SAM 3D tackles everyday cluttered snapshots and turns selected subjects into detailed 3D meshes.

Meta has open-sourced both the model checkpoints and inference code. They are also preparing to release the SAM 3D Artist Objects (SA-3DAO) evaluation dataset, which provides paired real-world photos and ground-truth meshes to benchmark reconstruction in natural settings.

If you want to test the tool without cloning repos, Meta launched the Segment Anything Playground on the web. You can upload an image, select objects or people, and inspect the 3D reconstructions directly in your browser. The playground also bundles SAM 3, Meta's latest foundation model for image and video understanding.

Inside Meta's own apps, the system is already working in production. Combined with SAM 3, it powers the new View in Room feature on Facebook Marketplace, allowing shoppers to visualize home decor items inside their own living rooms before buying.

How Meta trained it

The fundamental wall in 3D AI has always been data. High-quality 3D ground truth is orders of magnitude scarcer than text or flat images. Prior models relied almost entirely on synthetic CAD-like files, leading to severe sim-to-real gaps when fed everyday camera shots.

Meta solved this by adapting the pre-training and post-training playbook from large language models. They treated synthetic 3D assets as the pre-training foundation, then built an active human-in-the-loop data engine to handle post-training alignment.

Building 3D meshes from scratch requires skilled 3D artists, but Meta recognized that rating and ranking generated meshes is much easier. Human evaluators scored candidate meshes produced by model ensembles, while only the toughest edge cases got routed to professional artists to patch blind spots.

Using this pipeline, Meta annotated nearly 1 million distinct real-world photos and produced roughly 3.14 million model-in-the-loop meshes. As the models improved, the data engine became better at producing candidates, creating an ongoing data flywheel.

Why it matters

For solo game developers, XR creators, and 3D hobbyists, SAM 3D is worth testing right away. Turning single real-world snapshots into textured geometric meshes removes the friction and hardware cost of multi-view photogrammetry.

Meta took the smartest path here: instead of waiting for natural 3D datasets to magically appear, they borrowed LLM post-training rankers to manufacture their own ground truth. Hallucinating unseen backsides from a single image will still hit limits on intricate occlusions, but it moves single-image 3D from isolated toy demos into usable creative workflows.