Sony's MMHNet generates synced foley audio for 5-minute videos
Sony researchers just introduced MMHNet, a video-to-audio framework that combines hierarchical networks with Mamba architecture to generate realistic audio for clips over five minutes long. It solves the long-video sync puzzle by training on short snippets while generalizing to long scenes, keeping sound effects tightly locked to visual hits and cuts.