PermaVid fixes AI video editing amnesia

Anyone who has tried editing generative video knows the frustration of AI amnesia. You tell the model to swap a character's jacket to leather, and two seconds later the jacket turns back into cotton, or the character's face warps into someone else entirely.

Generative video models struggle with editing consistency because their context memory clumps everything together. When an edit alters visual appearance or layout, existing stored contexts quickly become outdated or corrupted.

Released on June 15 alongside its research paper, PermaVid takes a direct swing at this problem by introducing a disentangled context memory framework that separates how a scene looks from how it is physically shaped.

Decoupling RGB look from 3D depth

Instead of treating every frame as a flat collection of pixels, PermaVid splits its working memory into two distinct banks.

The first is an RGB context memory. This bank tracks semantic appearance, including textures, lighting, and colors, while implicitly preserving viewpoint observations.

The second is a dedicated depth context memory. It completely ignores color and styling, focusing strictly on preserving 3D geometric layout and spatial structure.

Working alongside an edit-aware retrieval and update mechanism, the model fuses these multimodal memory streams during generation. The result is surgical control across edits.

If you apply a global style transfer, turning a city street into a watercolor painting, PermaVid locks the 3D depth memory so buildings and perspectives stay rock solid while the new aesthetic flows across frames. If you perform a local edit, swapping out an object on a table, it updates the object's appearance context without destroying the surrounding scene geometry.

The project has open-sourced its code, complete with documentation for users to set up and run the framework locally.

Why it matters

Generative video will never become a reliable production tool if creators cannot make iterative tweaks without breaking the rest of the clip.

For digital creators and VFX artists, PermaVid points toward actual usability. Right now, editing an AI video usually means rerolling seeds or masking frames by hand in external software. Giving the model long-term spatial memory brings us closer to a true timeline where edits persist across subsequent camera moves.

Architecturally, we think PermaVid makes a clever design choice. Instead of simply scaling parameter counts and hoping the neural net figures out physics on its own, it pulls in explicit geometric depth priors to anchor spatial relationships. Decoupling structural consistency from visual styling prevents edits in one domain from corrupting the other.

That said, running dual memory banks and depth fusion locally naturally introduces extra compute and retrieval overhead. The fidelity of the edits will also heavily depend on how cleanly the underlying depth estimator captures tricky scenes. Still, curing AI video amnesia is an essential milestone before generative models can graduate into professional editing suites.