Microsoft's new model turns images into 3D in 3 seconds

TRELLIS.2-4B is Microsoft's newly open-sourced, 4-billion-parameter 3D foundation model that turns a single image into high-fidelity 3D assets. Rather than producing crude point clouds or untextured meshes, it directly generates 3D meshes with full physically-based rendering materials, including realistic translucency, completely under the permissive MIT License.

What it does and how it works

Most existing image-to-3D pipelines rely on implicit fields like SDF or Flexicubes. Those approaches often fall apart when dealing with thin open surfaces, non-manifold structures, or complex geometric layers.

TRELLIS.2 takes a different path by introducing O-Voxel, a sparse voxel structure that encodes geometry and appearance simultaneously. It supports arbitrary topologies without lossy conversion and allows bidirectional transformation between meshes and voxels in milliseconds to seconds.

Under the hood, a 4B Flow-Matching Transformer operates over a sparse 3D VAE with 16x spatial downsampling. It shrinks a massive 1024³ resolution asset into roughly 9,600 latent tokens with virtually no perceptible loss in fidelity.

On an NVIDIA H100 GPU, the pipeline outputs a 512³ asset in just 3 seconds. Pushing the resolution to 1024³ takes around 17 seconds, and the maximum 1536³ detail level completes in about 60 seconds. It also supports shape-conditioned texture generation for existing raw 3D models.

Hardware specs and current limits

The local setup requirements are steep. The model is currently only verified on Linux and requires an NVIDIA GPU with at least 24GB of VRAM, with official verification on A100 and H100 accelerators. Software prerequisites include CUDA 12.4 and Python 3.8 or higher.

Keep in mind that TRELLIS.2-4B is an unaligned base model. Because it has not undergone RLHF or aesthetic tuning, visual styles strictly mirror the training set distribution, so you may need to experiment with reference inputs. Furthermore, raw generated meshes occasionally contain small topological holes, though Microsoft provides post-processing scripts to seal them for 3D printing workflows.

The weights and inference code are live on Hugging Face, where the model logged over 2.4 million downloads last month, alongside runnable Google Colab and Kaggle notebooks for quick testing.

Why it matters

For indie game creators, VFX artists, and digital asset designers, TRELLIS.2-4B moves image-to-3D generation much closer to a true production utility. Instead of handing off broken geometry that requires hours of manual retopology and material baking, it delivers export-ready GLB meshes with native PBR channels in seconds.

We think this is a genuinely impressive release from Microsoft. While the 24GB VRAM barrier keeps most entry-level desktop users on the sidelines, teams running cloud GPUs now have access to a blazing-fast, open-weight 3D generation engine that rivals proprietary closed-source alternatives.