Taalas hardwires Llama into silicon to hit 17,000 tokens/s

Waiting for an LLM to spit out words line by line might soon feel like listening to a dial-up modem.

AI chip startup Taalas unveiled the HC1, a radical AI accelerator that physically bakes the Llama 3.1 8B model directly into hardware. Instead of treating software models and hardware accelerators as separate layers, Taalas hardwired the neural network into the silicon itself.

The result is staggering: the HC1 cranks out up to 17,000 tokens per second. That is roughly 40 times faster than Nvidia's flagship B200 and about 10 times quicker than Cerebras hardware, while reportedly costing 20 times less to manufacture and consuming just a tenth of the power.

Smashing the memory wall

Running large language models on traditional GPUs usually runs headfirst into the memory bandwidth bottleneck. Processors have to ferry model weights between external memory and compute cores continuously, burning precious time and electricity.

Taalas bypasses this entirely by unifying storage and compute on a single die at DRAM-level density. Fabricated on TSMC's 6nm node, the massive 815 square millimeter chip packs 53 billion transistors. Designed for 2.5kW server systems, it is engineered for serious enterprise racks rather than low-power edge gadgets.

Ridiculously fast, but strictly single-purpose

In online demo tests, the speed is borderline disorienting. A simple prompt can hit nearly 20,000 tokens per second, while standard conversational responses hover between 15,000 and 16,000 tokens per second. An entire 14-chapter book outline generated in just 0.064 seconds.

The catch is flexibility. Because the neural network is etched into the silicon, you cannot suddenly swap Llama for DeepSeek or Qwen. Taalas notes that users can still adjust context window sizes and fine-tune behavior via LoRA adapters, but the base foundation is permanent. Also, keep in mind that 8B models still hallucinate, just thousands of times faster.

Taalas plans to release a second HC1 chip focused on a mid-sized reasoning model in Q2, followed by its next-generation HC2 architecture near the end of the year.

Why it matters

For voice agents, real-time robotics, and high-concurrency enterprise services, eliminating inference latency changes everything. Conversations become truly conversational when the model answers before the user can even finish blinking.

Across the industry, Taalas represents the absolute extreme of ASIC specialization. Nvidia won the datacenter by selling flexible general-purpose computing platforms, but Taalas bets that once model architectures settle down, hardwired silicon will blow general GPUs out of the water on cost and power.

We think this is a high-risk, high-reward bet that makes a ton of sense for established workloads. The obvious danger is model obsolescence before silicon ships. But if your enterprise stack already runs millions of calls through a standard lightweight open-weight model every day, turning that software into a dedicated chip could slash operating expenses down to virtually nothing.