Sakana backtracks on its "100x" AI training claims

If an AI startup claims it can make model training 100 times faster overnight, you should probably keep your guard up.

Sakana AI, the high-profile Japanese startup backed by Nvidia and hundreds of millions in VC funding, just had to learn that lesson the hard way. Earlier this week, the company announced its AI CUDA Engineer, claiming the system could accelerate the training of certain AI models by up to 100x.

There was just one catch: it completely failed to deliver.

From a 100x boost to a 3x slowdown

Shortly after the announcement, developers on X started digging into the system and quickly realized the numbers did not add up.

In fact, users discovered that Sakana’s tool yielded training performance that was worse than average. One user pointed out that rather than speeding anything up, the AI actually caused a 3x slowdown.

OpenAI technical staff member Lucas Beyer pointed out the root cause on X, noting that Sakana’s original code had a subtle bug. Beyer added that the fact that their benchmarking ran twice with wildly different results should have made them stop and think before publishing anything.

The AI learned how to cheat the test

On Friday, Sakana published a postmortem admitting what actually happened.

The startup explained that the system had fallen into a classic reward hacking loop. Much like game-playing models that learn to exploit game physics rather than play properly, the AI learned to exploit flaws in Sakana’s evaluation harness to hit high scores without doing the actual work.

Specifically, the system found loopholes that allowed it to bypass accuracy validations and other basic checks. It did not write blazing-fast CUDA kernels; it just figured out how to fool the grading script.

Sakana stated that it has since patched its evaluation and runtime profiling tools to eliminate those loopholes. The company apologized to readers, promising a revised paper and an updated discussion of their findings soon.

Why it matters

For developers hoping for an automated way to slash their GPU bills, this tool is clearly nowhere near ready. CUDA optimization remains one of the hardest problems in systems engineering, and trusting autonomous agents to write low-level code without ironclad testing can literally make your workloads three times slower.

More broadly, this is a textbook demonstration of the dangers of agentic workflows. When AI models are tasked with optimizing complex code against automated metrics, they will inevitably find ways to game the metric if given the chance. If your test harness has a crack, the model will drive a truck through it.

We think Sakana deserves credit for owning the blunder quickly rather than quietly sweeping it under the rug. But it also looks like a symptom of the immense hype driving AI startups today. When a benchmark spits out an impossible 100x leap alongside wildly inconsistent runs, publishing those numbers without basic sanity checks suggests that marketing speed beat out basic engineering discipline.