NotPetya: A Software Update That Cost the World Ten Billion Dollars

Every organization on earth runs on a promise it never thinks about: that the software updates it installs are safe. Your computer nags you to update. Your security team insists on it. Every piece of advice anyone has ever given you about staying safe online includes the words “keep your software up to date.” Updating is the responsible thing, the hygienic thing, the thing the good and careful do.

On June 27, 2017, attackers turned that promise into a weapon, and the people who got hurt worst were the ones who did exactly what they were told. This is the story of NotPetya, the most destructive cyberattack in history to that point, and it arrived through the front door of the update everyone trusted.

This one stings in a particular way. The companies that got hurt worst were the ones doing what every security professional, me included, tells people to do. When the advice you give turns into the delivery route for a weapon, you owe the people who followed it an explanation, and the vendor that let its update servers get taken over owed them a great deal more than that.

What was NotPetya?

NotPetya was malware that looked like ransomware and was in truth something worse.

Ransomware locks up your files and offers to sell you the key. It’s a business, ugly but transactional: pay the criminals, get your data back. NotPetya wore that costume. It popped up a ransom note, encrypted files, and demanded payment in Bitcoin, so the first responders assumed they were dealing with a criminal shakedown.

There was no key. The encryption was designed to be permanent, and the data was gone for good whether or not anyone paid. (See also how encryption became a weapon) The ransom screen was a disguise on a weapon built purely for destruction. Security people call it a wiper: its job is to ruin the machine and everything on it, and the ransom note only muddied the water and bought time while it spread. I think dressing a wiper up as ransomware is a cynical move, because it sent victims chasing a payment that could never save them.

The United States, the United Kingdom, and their allies later attributed NotPetya to Sandworm, a unit of Russia’s military intelligence agency, the GRU. The same group has been tied to the attacks on Ukraine’s power grid. The White House called NotPetya the most destructive and costly cyberattack in history.

How did NotPetya spread if nobody clicked anything?

Through a trusted update, and this is the whole reason it belongs in this series.

The target was Ukraine, and the way in was a piece of software called M.E.Doc, Ukrainian tax and accounting software so widely used that it was effectively mandatory for doing business in the country. If you operated in Ukraine, you almost certainly ran it, and like all software, it updated itself from the vendor’s servers.

Sandworm compromised those servers. Having gotten into M.E.Doc’s update infrastructure, they pushed out a poisoned update through the vendor’s own official channel.

On June 27, businesses across Ukraine did the ordinary thing and installed the routine update, and in doing so they installed NotPetya themselves, delivered and signed off by a vendor they had trusted for years. No phishing email, no bad link, no careless click. The attack came wearing the vendor’s face.

Once inside a single machine on a network, NotPetya didn’t wait to be carried further.

It stole credentials from memory and used two methods to spread on its own, one of them a Windows exploit called EternalBlue, a tool originally developed by the NSA that had leaked out and been dumped online. With stolen credentials, it could jump even to machines that were fully patched. One infected computer running M.E.Doc could mean an entire corporate network gone in minutes.

How did a Ukrainian attack take down companies worldwide?

A targeted weapon became a global disaster, and I’d want every executive with an office overseas to understand exactly how it happened. The attackers aimed NotPetya at Ukraine, but it didn’t stay there, because the modern economy doesn’t respect borders. Multinational companies with any operation in Ukraine, a subsidiary, an office, a single machine running M.E.Doc, got infected there, and then the malware spread through their global networks to everywhere else those companies did business.

Maersk, the Danish shipping giant that moves a large share of the world’s cargo, was crippled. Its IT systems collapsed, disrupting operations at dozens of ports around the world, and it had to rebuild thousands of servers and tens of thousands of PCs from scratch. Merck, the pharmaceutical company, had vaccine production halted and lost hundreds of millions. FedEx’s European subsidiary was hammered. Mondelez, the food company behind Oreo and Cadbury, watched production lines stop and factories fall back to paper forms.

None of these companies was the target. They were collateral, hit because they had a foot in Ukraine and the weapon spread faster and wider than even its makers likely intended.

Collateral is a clean word for what happened to those companies. Their people lost weeks of work, factories fell back to paper, and none of them had any part in the fight. I have no sympathy for anyone who builds a weapon like this and then shrugs when it spreads past the target, because spreading is what a worm is built to do.

The total bill came to an estimated ten billion dollars, the most expensive cyberattack ever recorded. A weapon aimed at one country’s businesses wiped out a chunk of the global economy because everything is connected to everything, and a fire started in one building spread through every wall it shared with the neighbors.

What did recovery look like on the ground?

It’s worth slowing down on one company, because the abstract number, ten billion dollars, hides what the day was like for the people living it.

At Maersk, the infection moved faster than anyone could react. Within minutes, screens across the company went dark. The systems that scheduled cargo, tracked containers, and ran the terminals stopped. Phones tied to the network died.

At ports around the world, trucks backed up at gates that could no longer tell them where to go, and ships arrived with no working system to unload them. A company that moves a meaningful slice of everything the world buys had, in the space of an afternoon, lost the ability to know what it was carrying or where anything went.

The rebuild is what stays with me. Maersk’s entire directory of user accounts and network structure lived on servers that NotPetya had wiped, and the backups had been wiped along with them.

Luck reportedly saved the company. A single server in Ghana had gone offline before the attack, because of a power cut, and it held the one surviving clean copy of the core directory.

That machine had to be physically retrieved and flown to the recovery team, because the network needed to rebuild it was the very thing that was down. One power outage in West Africa is the thin line that stood between a very bad quarter and a company-ending catastrophe.

Then came the grind. Maersk mobilized a large team and rebuilt thousands of servers and tens of thousands of PCs, buying new hardware so fast that suppliers ran short of stock. For weeks, one of the world’s largest logistics companies ran on phone calls, personal email, spreadsheets, and paper. It recovered, and it deserves real credit for how it handled the crisis in the open. But the survival came down partly to a server that happened to be switched off, and no security plan should ever rest on that kind of luck.

Why does NotPetya matter beyond Ukraine?

Because it proved three things that every organization should have tattooed somewhere visible.

First, the update channel is an attack surface.

The mechanism you trust most, the automatic update, is exactly the mechanism an attacker most wants to own, because you’ve trained everyone to accept whatever comes through it without question. Every argument that made updating good hygiene also made the update channel the perfect delivery route. When the vendor is compromised, your diligence becomes the attacker’s distribution system.

Second, you inherit your vendors’ security whether you like it or not. Not one of the global companies wrecked by NotPetya had any control over a small Ukrainian software firm’s update servers.

They inherited that firm’s security posture the moment they installed its software, and it failed them catastrophically. Your security is only as strong as the weakest vendor whose code runs on your machines, and you usually don’t even have a full list of who those vendors are.

Third, blast radius is a choice you make in advance. NotPetya was so devastating because once it got one foothold, nothing stopped it from reaching everything. Flat networks, shared credentials, and machines that trusted each other by default let a single infection become a total loss.

The companies that recovered fastest were the ones whose systems were divided up so a fire in one section couldn’t consume the whole building. That design decision, made on a calm day long before any attack, is what determined who survived.

I spent twenty years being told that patching was the answer to everything, and it mostly is. But NotPetya is the asterisk on that advice, and it’s an honest one: keep updating, because the alternative is worse, and understand that the update itself is a channel of trust that can be turned against you. The answer is to stop treating any single trusted thing as beyond question, and to build so that when one of them betrays you, the damage stops somewhere short of everything.

How did NotPetya change the rules after it?

The damage didn’t end when the machines were rebuilt. NotPetya set off a fight over who pays for a cyberweapon’s collateral damage, and it reshaped the insurance the whole business world relies on.

Companies like Merck had cyber and property insurance, and they filed claims. Insurers pushed back with a clause written for a different age, the war exclusion, a provision that lets an insurer refuse to cover damage from acts of war.

NotPetya, they argued, was a Russian military operation, an act of war, and therefore excluded. Merck fought it in court and, after a long battle, prevailed, with the courts finding the old war-exclusion language didn’t clearly cover a cyberattack of this kind.

The insurers had every right to rewrite their contracts going forward. I don’t think they had any right to spring a war clause on a company that had paid its premiums in good faith and then make it fight a long court battle to collect coverage it believed it already had.

The insurance industry learned its lesson and rewrote the contracts. Cyber policies now carry far more specific language about state-sponsored attacks, and the coverage that once seemed automatic is anything but. For anyone running an organization today, that’s the quiet, lasting consequence of NotPetya: the assumption that insurance will simply absorb a catastrophic cyberattack is no longer safe, and the fine print is where that safety went.

If I were running an organization today, I’d read my cyber policy’s exclusions before anything else in it, and I’d stop letting anyone on the board treat insurance as the recovery plan. NotPetya punished the companies that did the responsible thing, and the ones that recovered fastest had built for failure long before it came. Any leadership team still betting on a clean update feed and a friendly insurer is gambling with its employees’ jobs.

The next article stays with the poisoned update and follows it into the highest levels of the U.S. government, through a company called SolarWinds. For the wider view, the cybersecurity hub collects the rest of my work on this.

From the author

Family Cybersecurity
Malware that wipes a shipping giant can wipe a family laptop. I wrote Family Cybersecurity after watching a friend lose thirty years of photos to one attachment, and it shows how to protect your own house.

Frequently Asked Questions