InstructAV2AV syncs audio and video edits with a single prompt
Researchers just introduced InstructAV2AV, an end-to-end framework that edits video visuals and accompanying audio at the same time using simple text instructions. It handles tasks like swapping characters, altering dialogue, and adding sound effects without manual masking, all while keeping the original background intact.