ByteDance open-sources Bernini for video generation and editing

Bernini is an open-source framework from ByteDance designed to handle both video generation and fine-grained video editing in a single pipeline. It couples a multimodal semantic planner with a DiT-based renderer, allowing users to move between text, images, and video without jumping across fragmented workflows.

Most video generators treat prompt following as a brute-force guessing game. Bernini tackles the issue by having an MLLM plan out what should change across time and space before passing that structural plan to the diffusion renderer. The setup shifts AI video closer to real directing instead of pure random generation.

What it does

Bernini unifies six common vision workflows into a single model interface: text-to-image, image-to-image, text-to-video, and standard video-to-video, along with reference-guided variants for both editing and generation.

In practice, it can swap local elements, strip subtitles and watermarks, transfer styles, and insert new objects guided by reference images. ByteDance's internal blind-evaluation arena scores the model's editing accuracy in the top tier among commercial engines. Early community notes point out, however, that while the instruction following and editing logic are sharp, raw visual fidelity and texture look pretty average compared to dedicated cinematic generators.

How to run it

The team dropped complete inference code, training code, and model checkpoints across GitHub and Hugging Face. You can choose between full Bernini and the dedicated Bernini-R renderer.

Bernini-R includes a 1.3B model fine-tuned from Wan2.1-1.3B. It performs close to the larger 14B variant on simple tasks like watermark removal and style transfer, though it visibly lags behind on intricate human figures. For setup, it requires Python 3.11, CUDA 12.6, and pinned builds of PyTorch and Diffusers, alongside the Apache-2.0 licensed Open-VeOmni dependency.

Hardware-wise, ByteDance recommends Hopper cards like H100 or H800 to leverage FlashAttention-3, but it falls back to FlashAttention-2 or standard SDPA on A100s and other CUDA GPUs. You can spin up an out-of-the-box Gradio demo script locally, with an optional flag that taps an OpenAI-compatible vision endpoint to refine your prompts before rendering.

Why it matters

If you build automated media pipelines or need repeatable video touch-ups, Bernini is worth setting up. Being able to run watermark removal, local asset insertion, and reference-guided edits on an open-weight 1.3B model gives developers practical controls that closed APIs rarely expose cleanly.

Across the AI video space, raw text-to-video generation is already a crowded race. By open-sourcing an architecture focused on multi-step semantic planning and localized editing, ByteDance is pushing open tooling toward practical post-production control, even if raw aesthetic polish still needs another pass.