TL;DR: In 1980, a water district in a snow-locked region had a problem nobody could solve with the technology of the day. The remote pumping stations were cut off from central command for weeks at a time every winter, and the water had to keep flowing. The team built a system of distributed autonomous controllers in Pascal that could think for themselves, knew their limits, and failed safely when conditions exceeded what they’d ever seen. The system ran for decades, and I apply the same approach to AI on books.
This story matters to me because it’s the cleanest proof I know that the AI argument is older than most of the people selling it. Every successful AI rollout since has followed the shape this team built in 1980, and every failed one has broken the rule they refused to break. Executives with far better tools keep relearning that rule at their customers’ expense, and they have no excuse.
Northern California, winter 1980. A small control room in a concrete building somewhere in the snowbelt. Outside, three feet of fresh snow on the ground, more coming down, the kind of weather that closes roads and shuts down the cell network nobody had yet.
Inside the building, a young engineer sits in front of a CRT screen running green monochrome text. The screen is showing flow rates from a pumping station forty miles away. The station is currently inaccessible. The road is closed. The phone line is down, because in 1980 these stations communicated by leased phone lines and the lines fell with the trees.
The pumping station is running anyway. It’s running because three years earlier somebody was told the existing system would fail in exactly this kind of weather. That system needed a human operator at central command for every routine decision. They had time to fix it before winter, and they used it.
What problem did the snow-locked water district have?
The water district covered hundreds of square miles. Tanks, dams, pumping stations, sluice gates, all scattered across mountain terrain that nobody could reach in winter. Central command monitored flow, adjusted pump rates against demand, opened and closed gates to balance the system, and answered any equipment that misbehaved.
The system was designed to fail safe instead of faking competence.Share on X
For nine months of the year, this worked. For the other three months, central command lost the remote stations one by one as roads closed and phone lines fell. The system had to keep running without the humans who normally ran it.
The old solution had been to set each remote station to a winter mode before the snow came. A fixed schedule. No adjustments. Demand changed. A pump failed. A tank started overflowing. The station kept doing what it had been told in October, until somebody could physically reach it in March.
Winter mode is what a lot of companies are doing with AI right now.
They configure it once, point it at customers, and walk away until something breaks. The district at least had three feet of snow as an excuse. A company that sets up its chatbot and stops watching it has no excuse at all, and its customers are the ones standing in the overflow.
This worked badly. Tanks overflowed. Pumps ran when they didn’t need to. Equipment failed because nobody was watching it. By spring there was always damage to repair and water that had been wasted or lost.
The new solution had to be something different. The remote stations had to decide on their own when they couldn’t reach a human. They couldn’t decide everything, because some decisions needed information only central command had.
The system they built
They wrote it in Pascal. Each remote station got its own controller, its own program, its own job. The program could read every sensor at the station. It knew the normal flow rate, the seasonal variations, the historical patterns from years of data. In dozens of routine situations, it knew what to do because the team had walked through every routine situation with the human operators before writing the code.
And, critically, the system knew its own limits.
When conditions at a station went outside what the controller had ever seen, it didn’t guess or extrapolate. It did one of three things: it reverted to a safe default that wouldn’t damage equipment, it sent an alert to central command if the lines were up, or it logged the situation and waited.
The system was designed to fail safe and never to fake competence. I wish every AI product shipping today carried that one design decision.
The engineer watching the green screen that winter night could see flow rates from a station forty miles away. The controller there had judged conditions to be inside the normal envelope, and it was running on its own. If the station had met something unfamiliar, the screen would have shown a flag instead of a number. The engineer would have known a decision was waiting for a human.
What did the water district’s system never try to do?
It never tried to replace the human operators.
The system was designed to fail safe and never to fake competence.
The operators still did the planning, still set the seasonal envelopes, still made the calls on anything that required judgment about the district as a whole. The system handled the routine, predictable, repetitive parts of the job when the operators physically couldn’t be in the room. The operators handled everything else, all year round.
This wasn’t the cheap solution. The cheap solution would have been to write a controller that tried to do everything the human operators did, in their absence, and call it a day. That controller would have failed the first time conditions at a remote station went outside its training, because it would have generated a confident response to a situation it had no business responding to, and the response would have been wrong, and the wrong response at a water district can flood a town or empty a reservoir.
So the system was built around what the technology could do, and nobody asked it to pretend. The technology could run a known pattern reliably. It couldn’t make judgment calls about anything it hadn’t seen before. The design respected that limit and built the rest of the system around it.
What happened next
The system ran. It ran that winter. It ran the next winter. It ran every winter for the next two decades, with periodic upgrades, until the industry moved to newer hardware and protocols that did the same job more elegantly.
The engineer who wrote it didn’t think of any of this as artificial intelligence. The phrase was just a control system. A program that could handle the boring routine work of running a remote station and knew when to ask for help.
Forty years later, every successful AI rollout I’ve seen has followed the same shape.
The system handles the routine. The humans handle the exception. The system fails safe when it’s outside what it knows. The humans stay in charge of judgment. The technology gets used for what the technology can do, not for what an executive press release wishes it could do.
| Design question | 1980 Pascal controllers | Failed rollouts today |
|---|---|---|
| Routine work | Each station handled routine situations mapped with operators | Confuse handling the routine with replacing the humans |
| Unfamiliar conditions | Safe default, alert central command, or log and wait | Confident response with no signal it is out of its depth |
| Human role | Operators kept planning and judgment calls all year | Humans fired; the system is expected to do every job |
| Reaching a human | A flag showed a decision was waiting for a person | No off-ramp; callers trapped for five minutes |
| Outcome | Ran every winter for two decades | Klarna hiring humans back |
And every failed rollout has broken the same rule
People who thought the system could handle every call designed the doctor’s office chatbot that traps the patient for five minutes. It can’t. It can handle the routine. The chatbot trapped the patient who knows what they want and just needs a human because nobody built the off-ramp.
I hate those chatbots. The off-ramp to a human costs money, so somebody cut it, and the patient who needed a person pays for that decision in minutes on hold. A system with no way to hand a caller to a human protects the company’s payroll and nothing else.
The fintech company that said its OpenAI chatbot was doing the work of 700 customer service agents was run by an executive who thought the system could do every job.
It can’t. It can handle the routine 70 percent. The remaining 30 percent destroyed the brand, and the company is now hiring humans back. I walked through that story in detail in Why Most AI Rollouts Fail.
Every Klarna in 2026 is making the mistake the water district team didn’t make in 1980. They’re confusing “the system handles the routine” with “the system can replace the humans.”
The water district team, working with technology that today’s executives would consider primitive, understood the difference. Today’s executives, with technology those engineers could only have dreamed about, don’t.
My take
Build AI the way that water district team built its controllers. They ran the routine, knew their limits, and failed safe when conditions went past anything they’d seen. Humans kept the judgment. That system ran for two decades. Most failed rollouts today break that one rule and expect the system to replace the people. Give the machine the routine, give a human the exception, and build the off ramp before you need it.
What does a 1980s water district teach about AI?
The technology has gotten more powerful. The lesson hasn’t changed.
Any system that runs without human judgment in the loop will eventually face a situation outside what it was trained on. Whether that system is a Pascal controller at a remote pumping station or a large language model handling customer service for a Fortune 500 company, the failure mode is the same. The system has no signal that it’s out of its depth. It produces a confident response to a situation it’s no business responding to. Somebody pays.
The fix is the same in both centuries. Design the system to know its limits. Design the workflow to keep humans in the loop wherever the cost of a single failure exceeds the convenience of automating the routine. Build for what the technology can do, not for what the press release wants it to do.
I apply the same rule to AI and books. AI is a tool, and if you treat it like one you’ll do fine. Let it do your work for you and you’ll have problems. My writing handbooks use AI prompts to help a writer build a stronger protagonist or a tighter plot, and they stop there, because deciding what the book means is the author’s job.
The water district team solved this in 1980, in Pascal, on hardware a modern phone would outrun without noticing. Executives who fire their people and hand the whole job to a chatbot in 2026 are ignoring a lesson that’s been sitting in plain sight for over forty years, and I’ve got no sympathy for them when the bill arrives. Their customers deserve better, and so do the people they fired.
For the deeper version of the story, told as a first-person memoir about what the engineer was thinking in that control room, see What I Learned in Pascal in 1980 That Every AI Executive Should Know.
Machines that know their limits make people more capable. The Birth of the Augmented Human is my book on using AI as a force multiplier for human thinking.
