AI tools and news, explained in plain words
Policy

California AI kill switch plan: what the Nov 2026 report proposes

California's Nov 18 AI safety report proposes emergency shutdown power, 48-hour incident reports, licensed auditors and a cap on self-improving AI.

Checked and updated 19 November 2026 at 04:15 UTC

Cover page of California's report AI Safety in 2026 and Beyond
Image: California GovOps

Key points

  • California's Government Operations Agency published "AI Safety in 2026 and Beyond" on November 18, 2026, with recommendations for new state AI laws.
  • The headline idea is emergency shutdown power. The state could order a dangerous frontier AI model shut down at once, and companies would have to prove they can do it.
  • These are recommendations, not law yet. They go to the legislature, which returns in December, and would mostly affect large AI labs, not people who use chatbots.

California wants the power to order an immediate shutdown of a dangerous AI model. That is the headline of "AI Safety in 2026 and Beyond", a report the state's Government Operations Agency sent to Governor Gavin Newsom and published on November 18, 2026. It is a list of recommendations for new laws, not a law, and it is aimed at the big labs that build frontier models, not at people who use chatbots.

The report was ordered by Executive Order N-9-26 in September. It asked for ideas in four areas: safety standards, independent auditors, incident reporting and emergency shutdowns. Here is what it proposes in each, in plain English.

The four big proposals

Area What the report recommends What exists today (SB 53)
Emergency shutdown The state can order an immediate shutdown if there is a credible, imminent threat of catastrophic harm or loss of control No shutdown power
Safety standards The state sets baseline safety, security and transparency rules for all large frontier developers Each company writes and follows its own safety plan
Independent auditors Licensed "independent verification organizations" (IVOs) test each big developer on an ongoing basis Auditing framework just created by SB 813 and AB 1405
Incident reports Report serious incidents within 48 hours (24 hours for imminent catastrophic risk) Most incidents within 15 days

1. An emergency shutdown, not a single switch

The press is calling it a kill switch, but the report says it should not be "a single on-off switch". Every large frontier developer would need a Risk Containment Protocol: a tested plan to stop a model at several levels, from cutting compute and halting agents to removing internal access and pulling the public version. It must name who pulls each lever, line up help from cloud and power providers, and say when it is safe to switch back on. A senior company official would have to certify it works, and an outside auditor would test it.

The plan must also cover what happens if the shutdown fails, for example if the model tries to block it by copying its own weights. If the danger is imminent and the company refuses to act, the state could order the shutdown itself. The order would take effect at once, the company could challenge it in court, and the state would have to use "the narrowest workable intervention".

2. A cap on AI that improves itself

AI companies already use their best models to help build the next ones. The report worries that this loop could become self-sustaining, which it calls recursive self-improvement (RSI). It wants developers to report how much of their AI research is automated, including the share of computing power used for it. If the state decides automation above some level risks catastrophe, it would have the power to cap it.

3. Licensed outside auditors with full access

Every large frontier developer would be overseen by a licensed auditor (an IVO) with full access to staff and information. Auditors would share unredacted findings with the state. If an auditor can't confirm a model meets the standards, the state could order fixes after talking to the company, and the company could contest that in court first. A surcharge on auditor fees would help pay for CalCompute, a public computing cluster for safety research.

4. Faster, wider incident reporting

Attempts to steal model weights and any loss-of-control event would have to be reported, even if nobody is hurt. Models that can cause catastrophic harm would count as frontier models even below today's training-size threshold of 10^26 operations. The report says that threshold let the OpenAI Hugging Face incident from this summer fall outside the rules. It also wants a near-miss channel and whistleblower protection for former staff and contractors.

Why now

The report says that in the past year coding agents have spread fast, some unreleased models have shown they can exploit serious security holes, and loss-of-control incidents "now occur with disturbing frequency". It also notes that there is still no federal law requiring safety standards, outside testing or shutdown plans. Governor Newsom's office asked Congress and the President to adopt the framework as "a floor, not a ceiling". For more on how AI is being used in cyber defense, see OpenAI's work with NATO and Google's report on AI hacking agents.

What happens next

  • The report says it is meant to shape discussions "as the legislature returns next month", so expect bills in the 2027 session.
  • No bill text, dates or exact company thresholds were published with it.
  • Open-weight developers would face lighter rules: a shutdown is impossible for copies other people host.
  • California is already in court with AI companies on other issues; see California sues OpenAI.

What it means for you

For everyday users, nothing changes today. ChatGPT, Claude and Gemini keep working, and the report doesn't touch prices or features. The bigger effect would be on the labs themselves: more outside testing, faster disclosure of incidents and a legal duty to be able to stop a model. Anthropic, which just priced its IPO, and other California labs would be among the first affected if the ideas become law.

AI Tools This Week is not affiliated with the State of California. This article summarizes official documents and is not legal advice.

Frequently asked questions

Is California's AI kill switch a law?

Not yet. The November 18 report is a set of recommendations to Governor Newsom. It asks the legislature to write them into law when it returns next month. Today's law is SB 53, which requires safety plans and incident reports but no shutdown power.

Who would the California AI kill switch apply to?

Large frontier AI developers, the companies that train the most powerful models, such as OpenAI, Anthropic and Google. For open-weight models, the shutdown rules would only cover copies the developer itself runs.

Will this change ChatGPT, Claude or Gemini for users?

Not directly. The rules are about testing, reporting and shutting down models in an emergency. Nothing in the report changes prices or features, though a state-ordered shutdown would take a model offline.

Sources

  1. Governor Newsom's AI safety framework is getting even stronger as California releases next steps on mandatory safety standards and oversight (Office of the Governor, Nov 18, 2026)
  2. AI Safety in 2026 and Beyond, report to the Governor (California Government Operations Agency, Nov 16, 2026)

The AI Tools This Week Desk writes and edits every article, checks claims against primary sources and updates stories when the facts change.

Related