California governor vetoes landmark AI safety bill

Silicon Valley's biggest existential dread this fall wasn't compute shortages or scaling walls. It was a piece of paper sitting on California Governor Gavin Newsom's desk.

Now, Big Tech can officially take a breath.

Governor Newsom just vetoed Senate Bill 1047, the landmark legislation that would have introduced the most sweeping AI safety guardrails in the United States.

The $100M threshold and kill switches

Authored by State Senator Scott Wiener, SB 1047 set its sights strictly on the biggest players in the game.

The bill would have applied to AI systems developed or operating in California that cost over $100 million to train or over $10 million to fine-tune. Developers of these mega-models would have been legally required to build in emergency kill switches, conduct rigorous third-party safety testing to prevent catastrophic threats like cyberattacks or biological disasters, and implement strict whistleblower protections. It also gave the state attorney general power to seek civil penalties for safety incidents that caused severe harm exceeding $500 million.

While the bill had undergone significant amendments over the summer—dropping plans for a brand-new regulatory body and walking back pre-incident liability—it triggered an all-out lobbying war.

A heavy-hitting coalition of tech leaders, venture capitalists, and politicians pushed back hard. Opponents included OpenAI, Google, Meta, former House Speaker Nancy Pelosi, and San Francisco Mayor London Breed, all arguing the restrictions would crush innovation and damage California’s open-source heritage. OpenAI chief strategy officer Jason Kwon argued that AI guardrails should be left to the federal government instead of individual states.

Yet the bill wasn't without surprising allies. Elon Musk publicly endorsed it, as did Turing Award winner Geoffrey Hinton and over 120 Hollywood creatives including Mark Hamill and J.J. Abrams, alongside labor unions like SAG-AFTRA. Even Anthropic CEO Dario Amodei softened his stance after earlier revisions, noting the bill's benefits likely outweighed its costs.

Newsom's calculation

Newsom didn't veto the bill by dismissing safety outright. Instead, he argued that SB 1047 was fundamentally flawed in its design.

According to Newsom, measuring risk strictly by training cost and model size misses the forest for the trees. The bill applied rigid standards to basic software functions simply because they lived inside a massive model, while ignoring the actual deployment context, sensitive data access, or high-risk decision-making.

He warned that the bill risked giving the public a false sense of security, pointing out that smaller, highly specialized models could emerge as equally or even more dangerous without ever hitting the $100 million threshold. Curtailing frontier training runs would therefore slow California's competitive edge without actually preventing harm.

Wiener pushed back forcefully on X, calling the veto a setback that leaves the public unprotected against unchecked corporate power, especially given Congress's ongoing stalemate on tech oversight.

Why it matters

For now, the frontier AI race can run without state-level governor switches.

For builders, startups, and open-source contributors, Newsom's veto is an immediate win. Had it passed, the prospect of multimillion-dollar liability and mandatory kill switches would have cast a chilling shadow over anyone deploying or fine-tuning powerful weights in California. Small labs trying to scale would have faced crushing compliance overhead.

In our view, however, Big Tech’s victory lap may be short-lived. Newsom made it clear that California is still looking to regulate—just through an empirical, risk-based lens rather than blunt compute caps. By shifting focus from how big a model is to how it gets used in the real world, future legislation could end up scrutinizing specific enterprise deployments and vertical apps even more aggressively.

The move also puts the spotlight squarely back on Washington. Silicon Valley has repeatedly lobbied against state patches in favor of unified federal rules. But with the Senate still mulling over a broad $32 billion AI roadmap and Congress structurally frozen, the U.S. continues to govern the frontier AI era with virtually zero binding guardrails.