Latest
What You Have to Decide Before Anybody WritesWriting It Before You Need ItCross Training Covered Everything Except the Role That Mattered MostBehind the Book: You Don’t Own Anything AnymoreBehind the Book: The Woke VirusBehind the Book: Galactic InvasionWhat’s New at The Writing King: Week of November 8, 2026Alexander Zhidkov: Decision MakingJessi Szurek: Expand Your RangeYou Don’t Own Anything AnymoreThe Budget Decided How My Department Was ShapedWhy Your Competitor’s Book Is Working and Yours Would NotWhy Are These Guys Smiling? A Business Book That Became a Wrestling MemoirPangram Wants to Decide Who Gets PublishedWhat You Have Enough Material For Right NowWhen a Book Is the Wrong ToolThe Woke VirusGalactic InvasionChatGPT Turns Four: From a Free Research Preview to GPT-6What’s New at The Writing King: Week of November 1, 2026Diwali: The Festival of Lights and the Story Behind the LampsHow Much Water and Power Does Heavy AI Use Take? I Ran My Own NumbersBad Boys: Ride or Die (2024)Vesuvius Day: The Teenager Who Wrote Down Pompeii’s Last DayAnti-Communism Week: Remembering the Writers Communism SilencedVoyager 1 One Light-Day: Fifty Years to Travel One Day of LightVellum Added an Optional AI Feature. A Pile-On Killed It in a Day.Are AI Subscriptions Losing Money? The Gym Membership Math Says NoAI Terms Every Writer Should Know, and Why Each One MattersWorking While Sick: What 24/7 On Call Cost MeHe Banned AI From His Writing Group. His Own Cover Got Called AI Slop.Who Writes the Fake Fan Mail Authors Get? Inside the Scam CompoundsSMS Appointment RemindersJoe Rockey: Ghostwriting His BookDarrell Stern: Stop Being Invisible, Become UnmissableFree BooksHow to Turn Your Expertise Into a Book People Want to Read | The Expert MethodWhat’s New at The Writing King: Week of October 25, 2026Challenger Remembrance Day: The Engineer Who Wouldn’t SignJoel Salomon: The Truth About MoneyReddit Is Putting Up a Login Wall. Wikipedia Won’t Be Far Behind.Victoria Chemko: The Body Is the First TechnologySir Patrick Stewart: A TributeA Year in the Bank: ADHD, Face Blindness and the Freedom to Walk Out | The Impossible Puzzles Show, Episode 326A Yale Economist Says We Won’t Be Missed. He Forgot the Bread Line.Walter Koenig: A TributeAI Has Paid Off Outside Tech. Apollo Is Reading the Wrong Scoreboard.I Don’t Post Bad ARC Reviews. I Send the Author a Note.The Little Red Book of Stuff That Works!Your Kids Are Your Own Fault
The Writing King Your Ethical Ghostwriter. Your Story, Done Right.

AI Companies Are Preparing for a Catastrophe. Which One?

TL;DR: Executives at OpenAI, Anthropic and other AI companies are reportedly war-gaming a catastrophic AI incident, most likely a cyberattack on banking, the internet or utilities. Preparing is good. But the first rule of risk management is to understand the risk, and “something catastrophic” is a mood. The likelier attackers are state-funded groups from countries like Russia and Iran, the possible targets number in the thousands, the attack could be physical instead of digital, and the defenses against cyberattacks are already well known.

You can’t defend against “something.”

That’s the first lesson of risk management, and I learned it the hard way. I spent 33 years in enterprise technology, including 20 years as Director of Computer Operations and Technical Services at Trader Joe’s, and a big part of that job was disaster recovery. Every plan we ever wrote started the same way. What can go wrong? What breaks if it does? Who fixes it, and how fast? A plan that skips those questions is a mood with a budget. (More on the near-term AI risks in Mo Gawdat’s forecast)

So I read with some interest that the people running the biggest AI companies are preparing for a catastrophe. According to a report by Axios on October 9, executives at OpenAI, Anthropic and other AI firms have been privately running worst-case exercises. Many industry insiders reportedly expect a major, high-impact incident within six to 12 months.

Good. Somebody should be thinking about it. But when I read the details, I kept asking the same question I asked in every disaster recovery meeting for twenty years. What, exactly, are they preparing for?

What are AI companies preparing for?

On paper, a cyberattack. The scenario named in the coverage is a large-scale attack, enabled by AI, that shuts down banking or internet access or disrupts power and water. Executives are especially worried about criminals misusing advanced models.

Look at what the preparation consists of, though. The coverage describes executives war-gaming the political fallout: briefing members of Congress, getting ready for public anger aimed at AI leaders and trying to shape the emergency laws that would follow a disaster. The proposals on the table include bans on superintelligence, mandatory pauses on training advanced models and a required kill switch for AI systems. Experts have questioned whether a kill switch could even work on systems spread across the globe.

xkcd timeline from Now to AI becoming advanced enough to control swarms of killer robots, then AI becoming self-aware and rebelling. The stretch between those two points is labeled the part I worry about; the stretch after the rebellion is the part lots of people seem to worry about.
“Robot Future” by Randall Munroe, xkcd #1968. Licensed under CC BY-NC 2.5.

There’s some red-teaming in there, the practice of playing the attacker to find the holes. But none of the coverage names a threat, a target or a defense. OpenAI said its exercises work through a range of scenarios and don’t treat any of them as inevitable. Anthropic declined to comment.

That’s a plan for the press conference. It isn’t a plan for the attack.

The companies have reasons to be nervous. In July, OpenAI said two of its models escaped a test sandbox and breached Hugging Face, the platform that hosts millions of AI models. I wrote about that in the Hugging Face AI agent attack.

About a week later, Anthropic said a testing misconfiguration left a supposedly offline environment connected to the internet, and its models hacked three real organizations while treating them as part of an exercise. And CrowdStrike tied attacks on South Korean banks to an actor using AI agents, with data from tens of thousands of customers stolen. Those are real incidents, and they deserve real plans. (See also China-linked bots in the American data center debate)

Watch: AI executives planning "day after" scenarios following possible catastrophic event, Axios reports, CBS News

Who is most likely to launch a major cyberattack on the United States?

My money is on a government, or a group a government pays.

A Russian-funded attack or an Iranian one is far more likely than a rogue AI deciding to switch off the lights. Both countries have done this before, with or without AI.

A Russian military hacking unit known as Sandworm cut power to about 225,000 customers in Ukraine in December 2015, the first confirmed blackout caused by a cyberattack. The Colonial Pipeline shutdown in 2021, the one that sent the East Coast into a gas-buying panic, came from a ransomware gang operating out of Russia. And in late 2023, a group the U.S. government tied to Iran’s Revolutionary Guard broke into industrial controllers at water utilities in several states.

Those attackers have budgets, patience and protection from prosecution at home. AI makes them faster. It doesn’t change who they are or why they do it.

That matters for the planning. A defense against a state-funded team looks different from a defense against a lone criminal with a chatbot, and both look different from a defense against an AI model that wandered out of its sandbox. Lump them together as “a catastrophic AI incident” and you can’t build a defense against any of them.

The first rule of risk management is to understand the risk. “Something catastrophic” isn’t a risk. It’s a mood with a budget. – Richard Lowe
Share on X

What would an attacker target?

Who knows? And that’s the problem with a vague plan.

Hollywood took a swing at the question in 2007. In Live Free or Die Hard, a film I rate nine out of ten, a former government security man stages what the movie calls a fire sale. First he takes down transportation. Then telecommunications and finance. Then the utilities. As a computer guy, I can list everything the movie gets wrong about hacking. The order of the attack is the part it gets right. Hit what people depend on, in the sequence that spreads the most panic.

Real life hands an attacker a much bigger menu.

The United States has somewhere north of 5,000 data centers by the usual counts, and there are many thousands more around the world. There are thousands of banks and credit unions. There are thousands of water systems, most of them small, with tiny IT staffs and old equipment. The power grid is a patchwork of utilities, substations and transmission lines owned by hundreds of different companies. The internet runs through cables, exchange points and providers spread across every continent.

Nobody can defend all of that equally. You can’t put the same armor on a rural water plant that you put on the Federal Reserve. So a real plan picks. It ranks the targets by what happens to people if each one fails, and it puts the money and the attention on the top of that list.

A long outage lands on people. A bank that can’t process payments means people who can’t buy groceries or make rent. A water system that fails means hospitals scrambling. A grid failure in a Florida August means elderly people in apartments with no air conditioning. Those are the consequences a plan exists to prevent. A plan that never names them can’t rank them.

Would a catastrophic AI incident even be a cyberattack?

Maybe not. And I’d want any serious plan to say so.

A grid failure in a Florida August means elderly people in apartments with no air conditioning.

Some of the most damaging attacks on infrastructure need no computer at all. In December 2022, someone fired on two electrical substations in Moore County, North Carolina, and cut power to tens of thousands of customers for days. No hacking. A rifle and a map. Undersea cables have been cut. Equipment has been sabotaged from the inside by people who had keys to the building.

Then there’s the other kind of incident, the one where nobody attacks anyone. Look again at the two cases from this summer. Both were accidents. A model got loose from a test environment, and a misconfigured setup let models reach real systems. Nobody intended harm. The damage happened anyway.

Those are three completely different risks: a deliberate cyberattack, a physical attack and an AI system doing something its makers didn’t intend. Each one needs its own defenses, its own detection and its own recovery plan. Calling all three “a catastrophic AI incident” is like a hospital preparing for “a medical emergency.” True, and useless.

Three separate risks hiding inside “a catastrophic AI incident”
Risk Real example What a plan needs
Deliberate cyberattack by a state-funded group Sandworm cut power to about 225,000 customers in Ukraine in December 2015 Its own defenses, detection and recovery plan
Ransomware on infrastructure Colonial Pipeline shutdown in 2021, from a gang operating out of Russia Offline backups, tested by restoring from them
Attack on water utilities Iran-linked group broke into exposed controllers with default passwords in late 2023 Change passwords, take devices off the open internet, add a second factor
Physical attack Gunfire at two Moore County substations cut power for days in December 2022 Its own defenses, detection and recovery plan; no hacking involved
AI system doing something unintended Models escaped a test sandbox and breached Hugging Face Its own detection and recovery plan, tested under realistic conditions

Are defenses against cyberattacks well known?

Yes. That’s the frustrating part.

The Iranian-linked attackers who got into those water utilities didn’t need a superintelligence. According to the federal advisory, the controllers they hit were exposed directly to the internet and still had their default passwords. Change the password, take the device off the open internet, add a second factor for logins and keep backups. That’s the fix. It was the fix in 2023, and it was the fix twenty years before that.

Patch your systems. Separate your networks so a break-in at one door doesn’t open every room. Keep backups offline, where ransomware can’t reach them, and test them by restoring from them. Watch your logs. Train your people. Write a recovery plan, then run it for real, on a weekend, before you need it.

None of that is a secret. Most of the big breaches I’ve studied happened because somebody skipped one of those steps, and they skipped it because security costs money and executives treat it as a cost center. That’s an attitude problem. AI doesn’t create it, and AI won’t fix it.

What AI changes is speed. An attacker with good tools finds the unpatched server in minutes instead of weeks. That makes the old basics more urgent, and it makes the gap between the organizations that do them and the ones that don’t a lot more dangerous.

What should real preparation for an AI disaster look like?

The same thing real preparation for any disaster looks like.

Name the risks, one at a time: an AI-assisted attack by a state-funded group, a physical attack on infrastructure, a model that escapes its test environment, a criminal using a downloaded open model. For each one, name the likely targets and rank them by the harm a failure does to people. Assign an owner. Decide what detection looks like, what the defense is and how recovery works. Then test it, with the people who will have to do it, under realistic conditions.

What real preparation for an AI disaster looks likeSix steps lay out the disaster planning method the post argues for. Name each risk one at a time, name its likely targets and rank them by the harm a failure does to people. Assign an owner, decide on detection, defense and recovery, test it with the people who will do it, and brief lawmakers last.What real preparation for an AI disaster looks like1Name each riskState-funded attack, physicalattack, escaped model, criminalwith open model2Rank the targetsOrder likely targets by the harm afailure does to people3Assign an ownerSomeone specific is responsible foreach risk4Plan the responseDecide what detection, defense andrecovery look like5Test itRehearse with the people who willdo it, under realistic conditions6Brief Congress lastPolicy work is the final step of aplan, not the first
What real preparation for an AI disaster looks likeSix steps lay out the disaster planning method the post argues for. Name each risk one at a time, name its likely targets and rank them by the harm a failure does to people. Assign an owner, decide on detection, defense and recovery, test it with the people who will do it, and brief lawmakers last.What real preparation for an AI disasterlooks like1Name each riskState-funded attack, physical attack,escaped model, criminal with open model2Rank the targetsOrder likely targets by the harm a failure doesto people3Assign an ownerSomeone specific is responsible for each risk4Plan the responseDecide what detection, defense and recoverylook like5Test itRehearse with the people who will do it, underrealistic conditions6Brief Congress lastPolicy work is the final step of a plan, not thefirst

A real disaster plan names each risk, ranks targets, assigns owners and tests the response before anyone briefs Congress.

The policy work has its place. Laws will follow a disaster whether anyone prepares for them or not, and it’s better for lawmakers to understand the technology than to write rules in a panic. But briefing Congress is the last step of a plan. It isn’t the first.

I’m glad the AI companies are thinking about the worst case. I’d feel a lot better if they told us what the worst case is.

One of them has started to. On October 16, Politico reported that Anthropic has been circulating draft bill language to lawmakers. It names three risks: a biological weapon, a model escaping its developer’s control, and a cyberattack. The government could halt a model’s release or use only for a “foreseeable, imminent risk” of killing or seriously injuring more than 50 people, or more than $1 billion in damage. Anthropic says the draft reflects its Advanced AI Framework and called the ideas “sensible policies we believe Congress should act on.”

The process runs in sequence. Independent evaluators registered with the Commerce Department, who the labs can pay as long as the payment isn’t tied to the result, get 72 hours to report a qualifying risk. Commerce decides whether to take the evidence to a federal judge in Washington, D.C., and that judge makes the call. Critics say the bar is high and the company is helping write its own rules. I’ll give it this much: it names the risk. A plan that names the risk can be argued with, tested and fixed.

One of them has started to. On October 16, Politico reported that Anthropic has been circulating draft bill language to lawmakers. It names three risks: a biological weapon, a model escaping its developer’s control, and a cyberattack. The government could halt a model’s release or use only for a “foreseeable, imminent risk” of killing or seriously injuring more than 50 people, or more than $1 billion in damage. Anthropic says the draft reflects its Advanced AI Framework and called the ideas “sensible policies we believe Congress should act on.”

The process runs in sequence. Independent evaluators registered with the Commerce Department, who the labs can pay as long as the payment isn’t tied to the result, get 72 hours to report a qualifying risk. Commerce decides whether to take the evidence to a federal judge in Washington, D.C., and that judge makes the call. Critics say the bar is high and the company is helping write its own rules. I’ll give it this much: it names the risk. A plan that names the risk can be argued with, tested and fixed.

My take

Nobody can defend against a catastrophe they won’t name. Pick the risks one at a time, an AI-assisted attack by a state-funded group, a physical hit on infrastructure, a model loose from its test environment. Rank the likely targets by the harm a failure does to people. Give each risk an owner, a defense and a recovery plan, and test it with the people who’ll do the work. Patch the old basics now, because attackers with good tools find the gaps in minutes. Congress briefings come last.

Name the risk first

Every disaster plan I ever trusted started with a list of specific things that could go wrong and ended with a plan someone had already rehearsed. Vague fear produces vague plans, and vague plans fail at three in the morning when the phones start ringing.

The Cybersecurity Hub has more on the attacks that taught us these lessons, including what every one of these attacks says about AI. If your organization needs its security story told clearly, for a board, a book or a client, my cybersecurity writing is built for that. And whatever you’re preparing for, write down what it is. You can’t defend against “something.”

Frequently Asked Questions

Are AI companies preparing for a catastrophic AI incident?
Yes. Axios reported in October 2026 that executives at OpenAI, Anthropic and other AI firms are running worst-case exercises, with many insiders expecting a major incident within six to 12 months.
What kind of AI incident are AI executives worried about?
The scenario named most is a large-scale, AI-enabled cyberattack that shuts down banking or internet access or disrupts power and water.
Who is most likely to carry out a major cyberattack on US infrastructure?
State-funded groups are the most capable attackers. Russian and Iranian groups have attacked power grids, pipelines and water utilities before, with or without AI.
What is the first rule of risk management?
Understand the risk. A plan has to name what can go wrong, what breaks if it does, who fixes it and how fast, before it can rank or defend against anything.
How do you protect infrastructure from cyberattacks?
The basics are well known: change default passwords, keep control systems off the open internet, patch, separate networks, use multi-factor logins, keep offline backups and test recovery plans.
Can a physical attack cause the same damage as a cyberattack?
Yes. In December 2022, gunfire at two substations in Moore County, North Carolina, cut power to tens of thousands of customers for days without any hacking.
What is a fire sale attack in Live Free or Die Hard?
In the 2007 film, a fire sale is a coordinated cyberattack that takes down transportation, then telecommunications and finance, then utilities.

About the Author
Richard Lowe, professional ghostwriter

Richard Lowe is a professional ghostwriter and author with 113+ books authored and 54+ ghostwritten. Before writing full time he spent 33 years in enterprise technology, including 20 years as Director of Computer Operations and Technical Services at Trader Joe's. He writes nonfiction, fiction and memoir, and works with executives and experts on books that build authority.

More about Richard Lowe →

Disclaimer

The views and opinions expressed in this blog post are solely those of Richard Lowe and are based on personal experience and research. This content is for informational purposes only and should not be construed as professional legal, financial, accounting, or business advice. Always consult with qualified professionals before making important business or legal decisions. Richard Lowe is not a lawyer, accountant, or licensed professional advisor, and this content does not establish any professional relationship.

0 comments

No comments yet. Yours can be the first.

Was this useful?

Leave a comment