ByteDance drops Lance, an all-in-one vision model

Lance is an open-source unified multimodal model from ByteDance that packs image and video understanding, generation, and editing into a single architecture.

What it can do

Instead of chaining separate models for different visual jobs, Lance handles the entire workflow under one roof. You can generate images and videos from scratch, or toss in existing media and edit it using natural language.

Its video editing chops are especially interesting. You can swap out a background for flames, repaint a car, or add props through multi-turn chat. For instance, you can ask it to curl a character's hair, add a flower crown in the next prompt, and then change her hand gesture.

On the static image side, it generates styles from anime to photorealistic shots. It can also merge objects from multiple reference images into a cohesive scene while maintaining visual consistency.

Because understanding is native to the model, it can solve visual puzzles on the fly. Hand it an image of a maze and it generates a video walking through the solution, or feed it a video with questions and it reads charts and context with ease.

Under the hood and hardware specs

Rather than brute-forcing scale, Lance relies on a dual-stream Mixture-of-Experts (MoE) setup trained on interleaved multimodal sequences.

This architecture decouples the understanding and generation pathways while sharing context, preventing visual tokens from clashing. While its total size is estimated around 14B, it runs on just 3B active parameters.

The code and paper are publicly available with setup instructions for local deployment.

The catch is the hardware footprint. To handle these heavy multimodal sequences, you will need a GPU with at least 40GB of VRAM, putting it out of reach for standard consumer rigs.

Why it matters

Lance proves that small, tightly coupled architectures can handle complex multimodal pipelines without needing massive parameter counts.

If you are building workflows that require conversational, multi-step editing instead of one-shot generation, Lance is a compelling foundation. It cuts out the duct tape usually required to stitch vision-language models to separate diffusion pipelines.

That said, raw video generation quality is not industry-leading yet. We think Lance shines best as an interactive editing workbench rather than a final-pass rendering engine.