Fire3D turns casual photos into editable 3D worlds in under a minute

Fire3D is an open-source, feed-forward 3D reconstruction system that turns a single everyday photo or casual video clip into an interactive, editable 3D world in under a minute.

Traditional workflows make you choose between two bad options: spend hours running iterative per-scene optimizations, or manually crop out every chair and mug before feeding them into single-object generators. Fire3D collapses perception, segmentation, modal completion, and texturing into one unified feed-forward pipeline.

How it builds a room in 60 seconds

You feed Fire3D an unsegmented RGB image or a rough handheld video. A component called Pi3X extracts point maps and camera trajectories, while DINOv3 features help predict object count, 3D bounding boxes, and 6-DoF poses without human prompts or masks.

From there, Fire3D does not just mesh the visible surfaces. It uses a flow-matching model to hallucinate the unseen backsides of furniture and props, reconstructing amodally complete geometry alongside PBR materials.

To make this fit on a single GPU, the authors developed HC-VAE, a hierarchical compression architecture built on top of TRELLIS.2's SC-VAE. It slashes latent feature capacity by 32x. That compression allows Fire3D to run a 12-step flow-sampling schedule across up to 16 objects in parallel on one card.

In testing, a 60-frame video featuring 12 instances, including structural background surfaces, went from raw capture to textured assets in 58.4 seconds.

Speed, benchmarks, and release

The speed gap against existing pipelines is dramatic. Optimization-based setups like HoloScene often demand around 8 hours of GPU crunching for a single scene; Fire3D delivers in roughly 60 seconds with sharper geometric accuracy. Compared to a modular pipeline pairing Boxer, SAM2, and TRELLIS.2, Fire3D runs inference 6.52x faster while scoring higher across benchmark metrics.

Every detected object exports as an independently transformable mesh, ready to drop directly into Unreal Engine, Blender, or robotics simulators like IsaacSim and IsaacLab.

The authors have released inference protocols, pretrained checkpoints, and complete training code on the project page.

Why it matters

For 3D artists, indie game creators, and robotics researchers, Fire3D eliminates hours of manual scene breakup. You get editable foreground meshes positioned cleanly in a metric coordinate frame, generated fast enough to iterate during prototyping.

Keep in mind that the framework is not without limits. The authors acknowledge that physical stability and sim-to-real robotic transfers have not yet been systematically evaluated. Expect meshes to look ready for simulation, but you will still want to verify collision hulls and center-of-mass distributions before trusting them in production robotics.