Etched bets $120M on an ASIC built purely for Transformers

What if the secret to beating Nvidia isn't building a better general-purpose GPU, but building a chip that refuses to do almost anything at all?

That is the audacious bet behind Etched, a two-year-old AI hardware startup that just raised $120 million in a Series A funding round led by Primary Venture Partners and Positive Sum Ventures. Instead of fighting Nvidia on its own turf, Etched is putting all its chips on a dedicated ASIC called Sohu, designed to do one single job: run transformer models.

Burning transformers into silicon

Modern GPUs are marvels of flexibility. They can train massive LLMs, process computer vision tasks with convolutional neural networks, or run emerging architectures like state space models. But Etched argues that flexibility comes with a massive performance tax for inference.

Etched COO Robert Wachen told The Register that by literally burning the transformer architecture into the silicon and stripping away every piece of unnecessary logic, the startup can unlock an order-of-magnitude leap in efficiency. Etched claims Sohu will deliver a 20x performance advantage over Nvidia's H100.

On paper, the claims are jaw-dropping. The company says Sohu can pump out 500,000 tokens per second running Llama 70B, claiming an eight-chip Sohu server can replace an entire rack of 160 Nvidia H100s.

The real secret sauce lies in batch sizes. For high-traffic consumer chatbots like ChatGPT or Copilot, serving thousands of simultaneous queries on standard GPUs usually causes severe performance degradation when batch sizes scale past 32 or 64. Etched claims Sohu can handle batch sizes in the thousands without breaking a sweat.

The paper tiger caveat

Before anyone starts canceling their Nvidia orders, there are massive caveats to keep in mind. Sohu hasn't taped out yet. Right now, it exists purely in emulation running on FPGAs, with physical silicon likely still nearly two years away.

There is also the memory bottleneck. High-throughput inference requires blistering memory bandwidth and enormous capacity. Sohu packs 144GB of HBM3 across six stacks, yielding an estimated bandwidth of roughly 4TB/sec. That lags behind Nvidia's H200 and Blackwell, as well as AMD's MI300X series.

Furthermore, running massive batch sizes eats up enormous amounts of memory for the key-value (KV) cache. With an eight-Sohu node holding around 1.1TB of total memory, real-world model capacity could run into a hard wall.

Why it matters

We think Etched represents both the smartest economic gamble and the riskiest architectural bet in hardware today. Right now, inference unit economics are breaking the back of every generative AI deployment. If Sohu delivers anywhere near its promised batch throughput, it could radically slash the cost of running hyper-scale AI products.

However, building an ASIC is an all-or-nothing game. If the AI research world shifts away from pure transformers toward alternative architectures like state space models or novel recurrent setups, Sohu instantly becomes expensive paperweights. In a field evolving as fast as AI, betting the farm on an architecture freezing for two years requires nerves of steel.