Motion 3-to-4: Turn monocular video into 4D animated objects

Motion 3-to-4 is an open-source feed-forward framework that converts a standard monocular video into a high-quality, temporally coherent 4D dynamic object. Beyond simple 3D reconstruction, it can also extract character movement and transfer those dynamics into other video sequences.

Synthesizing 4D dynamic assets from a single monocular viewpoint has always been notoriously difficult. The primary bottlenecks are scarce training data and the fundamental ambiguity of disentangling true 3D surface geometry from motion seen from just one perspective.

Decoupling static shape from vertex motion

Instead of solving 4D generation in one monolithic step, Motion 3-to-4 breaks the pipeline into two manageable halves: static 3D shape generation and motion reconstruction.

Given an input video and an optional 3D reference mesh, the model anchors itself on a canonical mesh to learn a compact motion latent representation.

From there, it predicts per-frame vertex trajectories to reconstruct complete geometry that stays consistent across frames. Because motion is isolated into compact vertex trajectories, extracting and transferring a character's actions onto new targets becomes straightforward.

Handling variable sequence lengths

Handling arbitrary video durations is another frequent pain point in 4D synthesis. Motion 3-to-4 incorporates a scalable frame-wise transformer designed specifically to stay robust across varying sequence lengths.

Evaluations across standard benchmarks, as well as on a newly introduced dataset with precise ground-truth geometry, show that Motion 3-to-4 outperforms prior approaches in both spatial fidelity and temporal consistency.

The research paper and project demo page are already live, showcasing the underlying mechanics and visual results.

Why it matters

Generating usable 4D digital assets usually demands expensive multicam capture rigs or labor-intensive motion tracking. Motion 3-to-4 proves that a feed-forward model can pull structured dynamic meshes out of ordinary single-camera footage.

For 3D artists, animators, and game creators, the real standout feature is the clean decoupling of motion from geometry. Being able to steal motion from a quick video clip and map it onto 3D meshes without rigging from scratch is a massive shortcut.

Keep in mind that monocular depth will always have blind spots on occluded surfaces, so heavy acrobatic moves may still require manual cleanup. Even so, it is an impressive open tool for rapid animation prototyping that developers should definitely keep on their radar.