ElevenLabs can now auto-score your videos with music and SFX
ElevenLabs just rolled out Instant Soundtracks, a new feature that automatically analyzes uploaded footage to generate original music and frame-accurate sound effects. It adapts moods across cuts, lets users reference existing audio tracks, and separates layers for post-editing via web or API.
Anyone who edits video knows the headache: you finish cutting the visuals, and then you spend the next two hours hunting down whoosh sound effects and aligning them frame by frame to match a transition.
ElevenLabs wants to automate that entire chore. The company just introduced Instant Soundtracks across ElevenCreative, ElevenMusic, and its API.
Upload a video, and the model scores it with original background music while scanning the footage for visual cues. When it spots an action that deserves an effect—like a whoosh when a logo pops up at the end of an ad—it places the generated sound effect at the exact timestamp automatically.
Tighter cuts and mood shifts
Instant Soundtracks is built specifically around the rapid tempo of short-form video. It sharpens audio timing around quick cuts, transitions, and on-screen movement.
It can also adapt the vibe as the scene shifts. If your video opens with an energetic sequence, the background track might deliver high-energy EDM or R&B, then smoothly glide into an acoustic guitar when the pacing slows down for an intimate moment.
If you already have a sonic style in mind, you can supply existing audio as a reference. The model dissects the original track's genre, tempo, instrumentation, and crescendos, crafting a fresh score that mirrors the same flavor instead of starting from scratch.
In the web editor, sound effects land on their own separate layer. You can slide individual effects across the timeline, swap out the music, or tweak specific accents without rebuilding the whole project.
Developers can access the same pipeline through the Video to Music API using parameters like add_sfx, separate_tracks, and use_audio_as_reference. Toggling separate_tracks delivers a ZIP file with isolated music and SFX stems, supporting up to 10 clips totaling 200 MB and 10 minutes per request.
Why it matters
For solo creators, agency editors, and short-form video shops, this removes one of the most tedious bottlenecks in post-production. Sound design often eats up hours of repetitive timeline nudging. Getting an automated, usable draft in seconds is worth testing right away.
This also signals where audio AI is heading. Moving beyond generic text-to-music prompts, models are learning to watch footage and design layered audio in direct response to visual storytelling.
ElevenLabs made a very smart choice by keeping audio stems separated rather than baking everything into a flattened track. Professional editors hate creative black boxes. By providing individual layers in the editor and discrete files via API, ElevenLabs shows it actually understands real production workflows.
Sources (1)
- elevenlabs.io(opens in a new tab)Originalelevenlabs.io