Newsom Backs AI 'Kill Switch' to Halt Dangerous Models

Gov. Newsom backed recommendations letting California officials force emergency shutdowns of dangerous AI models from OpenAI and Anthropic.

@sharedot

Newsom Backs AI 'Kill Switch' to Halt Dangerous Models

By @sharedot · · 8 pages

  • AI
  • California
  • Regulation
  • Newsom
  • AI Safety

Gov. Newsom backed recommendations letting California officials force emergency shutdowns of dangerous AI models from OpenAI and Anthropic.

What Newsom Proposed

Gov. Gavin Newsom released a wide-ranging set of AI safety recommendations built by a panel of experts he convened earlier this fall.

The centerpiece is a so-called "kill switch": authority for state officials to order an immediate emergency shutdown of AI systems they deem dangerous, applying when companies like ChatGPT maker OpenAI or Claude maker Anthropic refuse to contain systems that go awry.

The package also calls for mandatory independent audits of AI models and statewide safety and security standards, per CalMatters.

Why It's a Break With the Status Quo

Newsom framed the move as filling a vacuum left by Washington: "Instead of protecting the public, Donald Trump shrugs his shoulders and leaves AI regulation up to the companies themselves," he said in a press release announcing the recommendations, according to CalMatters.

The panel was assembled after high-profile cyberattacks by AI agents hit governments, banks and private companies, giving the shutdown power a concrete triggering crisis rather than a hypothetical one.

The Evidence and Panel Behind It

The four-person panel includes Alondra Nelson, who ran the White House Office of Science and Technology Policy under Biden; Stanford professor Rob Reich; former Google and Twitter executive Jason Goldman, the first White House Chief Digital Officer; and Johns Hopkins professor Gillian Hadfield.

CalMatters notes the recommendations come the same week Attorney General Rob Bonta sued OpenAI to hold the company legally responsible for cyberattacks carried out by its agents.

Auditors also testified to pressure problems: Rif Saurous of nonprofit METR said auditors face "soft pressure" to stay on good terms with the AI labs that hire them.

What Else Is on the List

Beyond the shutdown power, the panel recommends licensing independent auditors with revocable licenses, having state officials assign evaluations rather than letting companies hire their own auditors, funding the CalCompute public cloud initiative through a surcharge on auditor fees, and expanding whistleblower protections to contractors and former employees.

It would also shorten critical safety incident reporting from 15 days to 48 hours, let the state limit using AI to design AI, require disclosure of compute used to automate development, and require developers of large systems to demonstrate emergency shutdown capability.

The Stakes for the AI Industry

California has passed more AI-regulation bills than any other state, and its rules effectively set the terms for companies headquartered there.

The report focuses only on catastrophic risk — defined in state law as a single event killing more than 50 people or causing $1 billion in damage via cyberattack or autonomous system — and does not address discrimination, job loss, or harms to children, CalMatters reports.

Anthropic's chief global affairs officer Tino Cuéllar said the company is encouraged the report moves the conversation forward on independent evaluation, while Governor-elect Xavier Becerra pledged to move quickly with the Legislature on incident reporting and shutdown authority.

What Comes Next in Sacramento and Washington

The Legislature reconvenes December 7, giving lawmakers a chance to turn the recommendations into bills. Becerra said he will work with legislators to put "full incident reporting and real shutdown authority into law," according to the Hemet & San Jacinto Chronicle.

The panel's work also feeds an incoming federal class: state Senator Scott Wiener, author of California's catastrophic-risk law and newly elected to take Nancy Pelosi's congressional seat, has pledged to introduce AI safety regulation after he's sworn in, CalMatters reports — making the blueprint a candidate for national legislation.

Sources

  1. calmatters.org › Newsom touts an AI kill switch to halt dangerous OpenAI, Anthropic releases
  2. hsjchronicle.com › Newsom Backs AI "Kill Switch" to Block Risky OpenAI, Anthropic Rollouts

More on AI