Cactus packs a full speech-to-text model into just 16.9 MB

Whistle is an ultra-compact speech recognition model packed into a single 16.9 MB file, handling transcription, word timestamps, and speech embeddings directly on-device.

While most speech-to-text models require several gigabytes of space, heavyweight Python environments, and dedicated GPUs, Cactus Compute built Whistle to run purely on the CPU with zero external dependencies. It loads from a single .cact container straight into a lightweight C++ engine.

What it does

Whistle tackles three core jobs locally on your hardware.

First is transcription. It handles 16 kHz mono audio across seven languages: English, German, French, Spanish, Italian, Dutch, and Polish. It processes up to 30 seconds in a single pass, uses windowing for longer clips, and automatically identifies the language unless you explicitly set one.

Second is word-level timestamps. Aligned straight from the decoder attention, every emitted word comes with its start time, end time, and confidence probability.

Third is speech embeddings. It can output encoder representations at one row per 80 ms frame without running the full decoder.

It also supports keyword biasing through an Aho-Corasick automaton, letting you boost domain-specific terms during search. If the audio loudness falls below a set threshold, the engine immediately returns empty text without wasting cycles on beam search.

Speed, specs, and benchmarks

Despite weighing just 16.9 MB, Whistle holds its own against far heavier models. In tests across 86,174 utterances, it beat the 145.3 MB Whisper base model on LibriSpeech clean and other, SPGISpeech, Earnings-22, and the FLEURS average. Independent hands-on tests note that it decodes roughly six times faster than Whisper base.

Latency is another highlight. While Whisper pads every clip to 30 seconds, Whistle scales with actual duration: time to first token sits at 5.9 ms for a 5-second clip, 11.1 ms for 10 seconds, and 36.3 ms for 30 seconds.

Under the hood, an 8-block encoder processes frames at 80 ms intervals, while an 8-block decoder leverages gated cross-attention. Key and value projections are calculated just once per clip and cached, meaning a 5-beam search incurs almost no redundant audio computation.

How to run it

Getting started takes minimal effort. The C++ engine ships prebuilt for seventeen targets, including macOS, Linux, iOS, Android, watchOS, Windows on ARM, RISC-V, MIPS, and WASI.

For Python developers, a simple pip install gets you going. Processing 16 kHz WAV files requires no extra libraries, while microphone input needs just the optional mic package. You can run needle whistle playground to test your mic directly from the terminal, or run needle whistle compare to benchmark it against Whisper and Moonshine side by side.

Model weights are available on Hugging Face, while the engine binaries and full source code live on GitHub under Cactus-Compute/needle3.

Why it matters

We think Whistle's biggest win is stripping away the overhead of modern AI deployments. By cutting the footprint down to under 17 MB and ditching GPU requirements, it turns speech recognition into a drop-in utility for microcontrollers, wearables, and edge robotics that previously could not justify the hardware cost.

Its boundaries are clear: it only supports seven languages right now and caps single outputs at 320 tokens, so it is built for snappy voice commands rather than transcribing two-hour boardroom meetings. But for developers who need fast, offline, and private voice capture on constrained hardware, this is one of the easiest recommendations of the year.