VocalRender turns raw sheet music into realistic singing

VocalRender is a new open-source singing voice synthesis system that turns symbolic scores directly into 48 kHz expressive singing audio.

Traditional vocal synthesis usually forces you to micro-manage exact durations for every phoneme or draw complex pitch curves by hand. VocalRender skips that entirely. It only asks for what a composer naturally writes down: lyrics, MIDI note pitches, note durations, and tempo.

What it does

All you need to feed it is a score containing lyrics, MIDI notes, and a clean 2 to 8 second reference audio clip to set the target timbre.

The system uses an interleaved token sequence to bind words and notes together. It handles melisma, where a single lyric syllable stretches across multiple notes, automatically and naturally. Instead of relying on a rigid duration predictor, an autoregressive transformer sketches out the prosody and timing, while a localized diffusion model fills in pitch nuances, timbre details, and local acoustic textures.

There are two checkpoints available: the standard VocalRender and VocalRender-Pro. Both share the exact same architecture initialized from the speech model VoxCPM2, but Pro was trained on a larger corpus of real singing vocals. Third-party comparisons show it stays locked onto target pitches far better than older generators like Vivo 2 or Soul X, while capturing realistic breath and vocal inflections.

How to run it

The whole project is licensed under Apache-2.0, with full data preprocessing, training routines, and inference code made public.

Each model checkpoint is roughly 9.5 GB, meaning it fits inside most modern consumer GPUs with enough VRAM. You can clone the repository, install the environment, pull the weights from Hugging Face, and run an included script against three bundled test tracks without changing any config files.

If you prefer a visual interface, the repo includes a Gradio-based WebUI with a built-in piano roll editor. It can import ABC and MusicXML files, letting you swap reference voices and preview tracks right in the browser. The only real caveat right now is that the pre-trained checkpoints are focused on Chinese vocals; singing in other languages will require training your own checkpoint using the provided data pipelines.

Why it matters

For independent musicians and producers, VocalRender eliminates the single most painful part of making vocal demos: spending hours aligning phonemes on a grid. You can write a melody in MIDI, write the words, drop in a vocal sample you like, and render a 48 kHz scratch track in minutes.

Its hybrid autoregressive plus diffusion design hits a sweet spot between natural phrasing and acoustic clarity. While cross-lingual support still demands some DIY training, having the weights, code, and preprocessing pipeline fully open makes this one of the most practical local audio tools released this year.