Stability AI launches open-source language model Stable LM

Stable LM is Stability AI's new suite of lightweight, open-source language models designed to run locally on consumer hardware while generating text and code.

The initial Alpha release includes 3B and 7B parameter models, with larger variants ranging from 15B to 65B parameters slated to roll out later.

How it was built

The secret sauce behind getting capable performance out of such small architectures comes down to dataset scale.

Stable LM is trained on a new experimental dataset built on top of EleutherAI's The Pile, but expanded to three times the size at 1.5 trillion tokens. Stability AI previously helped open-source foundational models like GPT-J, GPT-NeoX, and Pythia alongside EleutherAI, laying the groundwork for recent projects like Cerebras-GPT and Dolly-2.

Alongside the base checkpoints, Stability AI also released instruction fine-tuned research models. These models were tuned on five recent open-source conversational datasets: Alpaca, GPT4All, Dolly, ShareGPT, and HH.

Specs, licenses, and where to try it

The licensing terms depend on which checkpoint you pull down.

The base models are released under the CC BY-SA-4.0 license, allowing developers to inspect, modify, and deploy them for commercial or research purposes without locking private data into proprietary services.

The fine-tuned conversational models use a non-commercial CC BY-NC-SA 4.0 license, aligned with Stanford's Alpaca rules and strictly intended for research.

Because these models sit at 3B and 7B parameters—a far cry from 175B monsters like GPT-3—they can run locally on consumer GPUs and edge hardware.

All code and weights are live right now on Stability AI's GitHub repo. The team also announced an upcoming technical report, a crowd-sourced RLHF initiative, and ongoing collaboration with the Open Assistant community.

Why it matters

If you are an independent developer or working with strict data privacy constraints, Stable LM is worth firing up immediately. It gives you a way to build local AI workflows without writing checks to proprietary cloud providers or streaming user data across the internet.

Compared to competing lightweight models, its 1.5 trillion token training foundation and commercially friendly base license make it an attractive starting point. That said, we think users should remain realistic about its capabilities: at 3B and 7B parameters, it will hit clear walls on complex multi-step reasoning unless fine-tuned tightly on domain-specific tasks.