OmniLottie turns multimodal prompts into vector animations
Researchers just introduced OmniLottie, a framework that leverages vision-language models to generate lightweight Lottie JSON vector animations directly from text, image, or video prompts. Alongside the model, the team introduced MMLottie-2M, a benchmark dataset of two million annotated vector animations, making automated UI motion design far more practical.
OmniLottie is a multimodal generation framework designed to turn text, image, and video instructions directly into lightweight Lottie vector animations. Instead of rendering heavy raster videos, it outputs clean JSON files ready to drop straight into mobile apps and web frontends.
What it does
Most generative AI tools today produce pixels. While diffusion models can render stunning MP4 clips or GIFs, those files are bloated, non-interactive, and impossible to scale without pixelation, making them largely useless for real-world UI engineering.
OmniLottie changes that by targeting the Lottie JSON standard. It follows multimodal prompts and outputs code-native vector graphics complete with shapes, animation curves, and control parameters that look crisp at any resolution.
How it works
Raw Lottie JSON files are full of invariant formatting noise and rigid metadata that make standard language model training inefficient.
To solve this, the researchers created a dedicated Lottie tokenizer. It compresses raw JSON documents into structured sequences of parameterized commands representing shapes, animation behaviors, and control parameters.
This compact representation allowed the team to build OmniLottie on top of pretrained vision-language models, making it the first family of end-to-end multimodal Lottie generators. To train it, the researchers curated MMLottie-2M, a massive dataset featuring two million professionally designed vector animations paired with rich visual and textual annotations.
Early community feedback on Hugging Face points out that generating parameterized tokens instead of pixels is a smart pivot that keeps compute costs low while yielding immediately usable files.
How to access it
The research paper is now available on Hugging Face Papers under identifier 2603.02138, detailing the framework design, tokenizer specs, and evaluation metrics.
Why it matters
If you build digital products, turning motion ideas into usable code has always been a painful, manual step. OmniLottie points to a future where developers can generate lightweight, resolution-independent UI animations on the fly.
While most labs are busy chasing higher-resolution video diffusion, OmniLottie tackles an overlooked engineering headache. We think treating structural formats as parameterized tokens rather than brute-force code generation is a very practical path forward for generative design tools.
Sources (1)
- huggingface.co(opens in a new tab)Originalhuggingface.co