SymphonyGen orchestrates full symphonies from simple chord sketches

SymphonyGen is an open-source AI framework designed for contemporary cinematic orchestration. Instead of trying to write dozens of instrument tracks all at once, it takes simple chord sketches and expands them into complete, multi-instrument orchestral arrangements.

Writing symphonic music with AI has always been notoriously messy because models easily lose track of harmonic coherence when juggling strings, brass, and percussion across hundreds of bars. SymphonyGen tackles this by splitting the problem into two distinct stages.

How it works

First, a dedicated model generates a beat-quantized harmony skeleton. This works just like a composer sketching out a short-score on a piano to lock down chord progressions and musical direction.

Next, a 3D hierarchical Transformer decodes the sketch along three axes: Bar, Track, and Event. It fleshes out the outline by assigning specific notes and parts to individual instruments. You can let the system compose from scratch or feed it a custom MIDI file to re-orchestrate an existing tune around its original chord structure.

To keep the composition from sounding chaotic, the team post-trained the model using GRPO reinforcement learning paired with CLaMP 3 perceptual audio rewards. During inference, it uses dissonance-averse sampling to actively penalize tonal clashes and harsh intervals.

What is released

All code, preprocessing scripts, and checkpoints are available on GitHub and Hugging Face, with the accompanying paper accepted at ISMIR 2026.

The release includes four checkpoints: a standalone 12-layer harmony model, a 33-layer two-stream 3D symphonic model, and two GRPO-tuned variants. Local audio rendering is handled through headless MuseScore.

The framework is lightweight enough to run locally and gives you granular control, such as masking out piano instruments entirely or visualizing dissonance adjustments across pitch logits.

Why it matters

For indie game devs and media composers, SymphonyGen serves as a remarkably practical orchestration assistant. Instead of manually voicing 30 orchestral tracks, you can lay down basic triads and get a full cinematic draft in seconds.

Compared to earlier black-box models like SymphonyNet or NotaGen, decoupling the harmony skeleton gives musicians direct steering wheel control over the progression. The dissonance filters also solve the random clash problem common in symbolic music AI.

Keep in mind that SymphonyGen outputs MIDI data rather than finished studio masters. While MuseScore provides quick audio previews, you will still need to route the MIDI tracks through your DAW and premium sample libraries to get production-ready dynamics.