New York passes RAISE Act to rein in frontier AI disasters

Silicon Valley thought it had killed state-level AI regulation when California vetoed SB 1047 last year. New York just proved that assumption dead wrong.

State lawmakers in Albany passed the RAISE Act on Thursday, marking a major legislative push to prevent frontier AI models from triggering catastrophic disasters. The bill specifically targets catastrophic scenarios like events resulting in more than 100 deaths or injuries, or causing over $1 billion in damages.

The legislation now sits on Governor Kathy Hochul's desk. She has the power to sign it into law, request amendments, or issue a veto.

Championed by AI pioneers Yoshua Bengio and Nobel laureate Geoffrey Hinton, the RAISE Act would establish the nation’s first legally binding transparency standards for frontier AI labs if enacted.

Targeting the $100 million club

The RAISE Act is laser-focused on the largest players in the industry, including OpenAI, Google, Anthropic, and foreign entities like DeepSeek and Alibaba.

Its rules only apply to companies deploying AI models trained with over $100 million in compute resources that are made accessible to New York residents. That single criterion effectively shields virtually all early-stage startups and university labs from compliance headaches.

Covered companies must publish detailed safety and security audits for their frontier systems. They are also legally required to report critical safety incidents, including unaligned model behaviors or stolen weights, directly to state officials.

Failing to meet these obligations carries teeth: the bill grants the New York attorney general the authority to seek civil penalties of up to $30 million.

Co-sponsor and State Senator Andrew Gounardes defended the urgency of the bill, arguing that the window to put sensible guardrails in place is closing fast while the most knowledgeable experts sound the alarm.

Silicon Valley pushes back again

To avoid the fate of California's SB 1047, lawmakers retooled the RAISE Act to address the tech industry's harshest criticisms.

According to Nathan Calvin, general counsel at Encode who worked on both bills, New York dropped the controversial "kill switch" mandate entirely. The bill also protects downstream developers who fine-tune open weights from being held liable for catastrophic harms.

Still, venture capitalists and tech executives are furious. Andreessen Horowitz general partner Anjney Midha slammed the bill on X, calling it "yet another stupid, stupid state level AI bill" that handicaps the US while global rivals sprint ahead.

Anthropic has not taken an official stance yet, but co-founder Jack Clark voiced concerns on X that the bill might be overly broad and hurt smaller companies. Senator Gounardes pushed back immediately, arguing that Clark's critique misses the mark since small teams fall well below the $100 million compute threshold.

Some critics warn that AI labs might simply geo-block New York, mimicking Big Tech's playbook in the European Union. However, co-sponsor and Assemblymember Alex Bores dismissed that threat, noting that New York has the third-largest economy in the US and the reporting burden is too light to justify leaving that revenue behind.

Why it matters

While the Trump administration and Silicon Valley continue to prioritize breakneck acceleration, New York is asserting that safety oversight cannot wait for federal consensus.

For builders and regular users, the immediate burden looks minimal. By stripping out downstream liability and setting a $100 million compute floor, the bill avoids punishing the open-source community or everyday developers who build on top of foundational models.

We think New York played this much smarter than California did. By ditching the sci-fi "kill switch" trope and focusing strictly on basic transparency and catastrophe reporting, lawmakers created a bill that is far harder for Big Tech to dismiss as unworkable.

The real risk lies in regulatory fragmentation. If individual states start rolling out bespoke reporting mandates, compliance overhead could quietly tilt the playing field even further toward well-funded tech giants. But if Hochul signs the bill, the era of zero mandatory disclosures for America's most capable AI systems is officially over.