TL;DR: In 1980, I was a young engineer writing Pascal for a water district that lost contact with half its infrastructure for weeks every winter. We built a control system that ran without humans when humans couldn’t physically reach the equipment. The system knew its limits and failed safely when conditions exceeded what it had seen how I use AI on a book. That project taught me something I have watched every executive in 2026 fail to learn, which is that the technology you can deploy is always smaller than the press release wants it to be, and the difference between the technology that works and the press release that didn’t is the question of whether the people who built it understood what it couldn’t do.
In 1980, I was sitting in front of a green monochrome screen at 2 in the morning, writing Pascal code that was going to control a pumping station forty miles away in the middle of a snowstorm I would never see.
I was young. I had no idea I was working on what people would call artificial intelligence in forty years. I thought I was writing a control system. Specifically I thought I was writing the controller for a remote pumping station in a water district that had a problem nobody had solved with the technology available, which was that every winter, for weeks at a time, half their equipment was unreachable.
I want to tell you what I was thinking that night, because I have spent the last four decades watching the same problem reappear in larger forms with bigger budgets, and the lesson I learned at 2 a.m. in front of a CRT in 1980 has never stopped being the lesson.
What I was thinking
I was thinking about a man I had met three weeks earlier, who had worked at the water district for thirty years. For more, see why every writer should embrace AI as a digital assistant.
He had told me, in a tone of voice I have never forgotten, that the existing automation at the remote stations was unsafe. For more, see augmented beats replaced. every time.. Every winter at least one tank overflowed, at least one pump damaged itself, at least one valve got stuck because the timer kept it open through a temperature drop the timer wasn’t smart enough to register. He could not get to the stations to prevent any of it. By the time the roads opened in spring there was always damage to repair. Sometimes the damage was expensive. Once it had flooded a small town. He told me that the night before they had to evacuate the town, he hadn’t slept.
He wanted the new system to not do that.
That was what I was thinking about at 2 a.m. on a Tuesday, writing Pascal. The man who couldn’t sleep the night before they evacuated the town. The man who, by the way, was going to be the one using my system to do his job. He was going to be the operator at central command, watching the green screen at headquarters, when the lines went down and the remote stations had to make their own decisions.
What I was writing was a program that was going to take some of his decisions away from him, in the hours when he couldn’t make them, and give them back to him in the hours when he could.
What I was not thinking
I was not thinking that I was going to replace him.
I want to be clear about this, because the contrast with what people in 2026 are doing is so absolute that I find it hard to describe without sounding like I’m exaggerating. It never occurred to me, or to anyone on the project, that the right thing to do was to build a system that did what he did, and then fire him.
The system existed because there were hours when he couldn’t be in the room. That was the entire reason for the system. The hours when he could be in the room were his. The hours when he couldn’t be in the room were the system’s, and the system was designed to do as little as possible while still keeping the water flowing. It was conservative. It made the smallest decisions it had to make. It logged everything for him to review when the lines came back up. It deferred to him in every situation where it could.
This was the design. The technology was not trying to be a person. The technology was trying to be a competent stand-in for a person, in the hours when the person was unreachable, doing the minimum required to keep the system safe.
The contrast with what executives are deploying in 2026 is that they’re deploying systems that try to do everything the human did, all the time, with the human removed. That isn’t a system. That’s a fantasy with a corporate-communications budget. I tell that story in my web development journey.
What I learned afterward
The system shipped. It ran that winter. It ran the next winter. It ran for two decades. It made small decisions when it had to, deferred large decisions to humans, and failed safe when it encountered conditions outside what it had been trained on, which happened periodically and which the team had explicitly designed for.
I went on to other projects. I watched the industry develop. I watched the technology get more sophisticated, more capable, more powerful. I watched the language change. Control systems became automation. Automation became smart systems. Smart systems became expert systems. Expert systems became machine learning. Machine learning became artificial intelligence. The technology got faster, the buzzwords got grander, and the budgets got larger.
The fundamental problem, the problem that man at the water district described to me with the tone of voice I have never forgotten, did not change.
The problem is that any system that operates without human judgment in the loop will eventually face a situation outside what it was trained on. The system has no way to know that it’s outside its training. It produces a confident response to the new situation. The response is wrong. Somebody pays.
This is true in 1980 Pascal. This is true in 2026 large language models. The mechanism is identical. The cost scales with what you let the system control. The man at the water district understood this. The executives who deployed Air Canada’s chatbot did not. The executives who deployed Klarna’s customer service chatbot did not. The medical practice that deployed the chatbot that traps the patient for five minutes every call did not. They all built systems that tried to do what the humans did, in the hours when the humans were absent, without the constraint we had in 1980, which was the explicit understanding that the system was a stand-in, not a replacement.
The lesson I would teach if I could
I have been trying, in different forms, for years, to teach the lesson I learned at 2 a.m. in front of a green CRT in 1980. The lesson is this.
The technology you can deploy is always smaller than the press release wants it to be.
The press release is a marketing document. The technology is what it is. They are not the same thing. The press release will tell you the technology can do everything the humans did. The technology cannot. The technology can do, at most, the routine portion of what the humans did, in the hours when the humans are unreachable, with appropriate fail-safe defaults for the situations outside its training. That’s what it could do in 1980. That’s what it can do in 2026.
The companies that deploy the technology according to what it can actually do are the companies whose systems run for two decades. The companies that deploy the technology according to what the press release said it could do are the companies whose systems get reversed within a year. The pattern is identical across forty years of technology shifts. The buzzwords change. The pattern does not.
If you read What a Snow-Locked Water District Taught Me About AI, you read the third-person version of this story, the parable. This piece is the same story, told as a memoir, because the lesson I learned was not abstract. It was specific, and it was given to me by a specific man, on a specific project, in a specific year, and I have never forgotten it.
What I want you to do with this
I want you to think about whatever AI rollout is on your desk right now.
I want you to ask yourself whether the technology you are about to deploy is being deployed according to what it can actually do, or according to what the press release said it could do. I want you to ask yourself who is going to be doing the work the technology cannot do, after the deployment. I want you to ask yourself whether you have built in the explicit fail-safe defaults for the situations outside the technology’s training, the way we built them into the Pascal controllers in 1980, because the technology in 2026 is more sophisticated than the technology in 1980 in many ways, but the failure mode is exactly the same.
And I want you to think about the man who couldn’t sleep the night before they had to evacuate the town. He was right about what the technology needed to do, and he was right about what it didn’t need to do. He understood the work, he understood the limits, and he was specific about both. The systems that work get designed by people like him. The systems that fail get designed by people who never met him, or anyone like him, and who don’t understand that he exists, and who deploy the technology as if the man does not need to be in the room.
He does. He always does. Every successful automated system in the next forty years will have a person like him somewhere in the design. Every failed one will be missing one. The Birth of the Augmented Human is the longer version of what that looks like across professions, but the version I know best is one engineer, one operator, and one Pascal program, in a winter I never saw, forty years ago.
The buzzword has changed since then. The rule has not.
The Guides That Get Your Book Written, Published, and Sold
Four short, practical guides on writing, publishing, and selling your book, plus the occasional note when there's something worth your time. No fluff, no daily inbox clutter. Drop your email and they're yours.
We use MailerLite to manage our list and send these emails. Your address is used only to send you what you signed up for. We will not sell it, share it, or use it for anything else, and you can unsubscribe anytime.
Frequently Asked Questions