Replit's new 3B code model punches above its weight

Replit Code V1.5 3B is a compact, open-source language model specifically trained for code completion. With just 3 billion parameters, it is designed as a lightweight base model that anyone can fine-tune for custom coding workflows without restrictive commercial limitations.

If you want an on-device or self-hosted code completion engine that runs fast and does not send code to third-party proprietary APIs, this is worth a look.

What it does

The model was pre-trained on 1 trillion tokens of code, drawing from permissively licensed repositories in The Stack dataset alongside developer content from StackExchange.

To keep quality high, Replit filtered the data heavily using syntax parsability checks, code quality heuristics, and toxic content removal.

It covers the top 30 programming languages on Replit, paired with a custom 32K token vocabulary tailored for efficient multi-language code representation.

In standard evaluations, it leads the pack among 3B-parameter models on HumanEval and Multi-PLe. Replit notes that after fine-tuning on public code, it even outperforms substantially larger alternatives like CodeLlama 7B.

Under the hood and how to run it

To balance rapid inference with output quality, the model utilizes Grouped Query Attention alongside Flash Attention Triton kernels and ALiBi positional embeddings.

During training, the team applied modern techniques including the LionW optimizer, learning rate cooldown schedules, and QKV clipping for stability.

You can grab the weights directly on Hugging Face under the repository replit/replit-code-v1_5-3b. Loading it requires standard Hugging Face transformers code with trust_remote_code=True.

Replit also outlines Triton GPU kernel optimizations in its repository README, making it easy to maximize token generation throughput.

Why it matters

We think Replit Code V1.5 3B hits a genuine sweet spot for edge and private cloud deployments.

Larger models like CodeLlama 7B require more VRAM and introduce higher latency for inline completion, where sub-second feedback is everything. At 3B parameters, this model runs cheaply while beating larger baselines once tuned on your own repositories.

Keep in mind that this model is tuned specifically for completion rather than conversational chat or high-level architecture planning. For autocompleting functions inside an IDE, however, it provides incredible bang for your compute buck.