PhysiFormer brings realistic physics and squishy dynamics to 3D meshes
Researchers just open-sourced PhysiFormer, a diffusion transformer that simulates real-world 3D mechanics directly on mesh coordinates. Instead of generating rigid, unresponsive shapes, it takes an object's initial mesh, velocity, and material stiffness to predict collisions and jelly-like deformations. It marks a solid step toward geometry-aware physics engines for robotics and graphics.
PhysiFormer is an open-source diffusion transformer designed to simulate realistic 3D mesh motion, collisions, and squishy deformations directly in world space.
While existing 3D generative models are great at churning out static assets, putting those assets into motion usually requires heavy manual rigging or traditional simulators. PhysiFormer bridges that gap: give it a 3D mesh, an initial velocity, and a material property, and it predicts exactly how the geometry moves and bends under real physical forces.
What it does
The model takes raw 3D mesh vertices, initial per-vertex velocities, and a simple material tag—rigid or elastic—as inputs. From there, it samples future vertex trajectories over time.
Drop a rigid ball, and it bounces cleanly while maintaining its geometry. Drop a rubber cube or a soft jelly-like shape, and PhysiFormer accurately simulates the squash, stretch, and collision response. It handles multi-object scenes out of the box and generalizes well to complex, unseen geometries that were never part of its training set.
Across benchmarks against autoregressive baselines, the model demonstrated significantly higher trajectory accuracy, better preservation of rigidity, and superior momentum consistency across more than 100,000 simulated training trajectories.
Under the hood
Instead of modeling physics as 2D video generation—which inherently tangles viewpoints, lighting, and occlusions—PhysiFormer works straight on 3D meshes in coordinate space, drawing inspiration from RenderFormer and Just Image Transformers.
It builds on a general-purpose Diffusion Transformer (DiT). The team factorized self-attention across time, space, and individual objects. This lets the network handle permutation-invariant multi-object physics without needing custom object tokens or learned latent autoencoders.
Crucially, PhysiFormer rejects step-by-step autoregressive rollout. Instead of repeatedly predicting the next frame and compounding drift errors, it generates the full temporal trajectory in a single denoising diffusion pass.
The authors have open-sourced the code and model weights on their project page, along with setup instructions for running local inferences. The full training dataset is slated for release soon.
Why it matters
If you work in 3D content creation, game physics, or robotics simulation, PhysiFormer is worth testing today. It gives static generative 3D meshes immediate physical utility without forcing developers through complicated manual rigging pipelines.
We think betting on raw 3D coordinate diffusion over pixel-space video models is a smart move. Pixel models hallucinate physics easily, but robotics and simulation stacks demand exact geometry and physical plausibility. While it currently only supports binary rigid and elastic labels, it establishes a very clean foundation for spatial intelligence in physical world models.
Sources (1)
- arxiv.org(opens in a new tab)Originalarxiv.org