The Writing King Your Ethical Ghostwriter. Your Story, Done Right.

The Water Kept Flowing When Nobody Was Watching

This entry is part 1 of 2 in the series The Operations Room
TL;DR: One question will tell you whether your AI agent project is going to work, and it takes an afternoon to answer. The question is not what the agent can do. It is what the agent is allowed to do when it cannot reach you. Nine out of ten agent pilots never make it into production, and I think that unasked question is most of the reason why. I worked on a system in the 1980s that answered it, and that system ran alone through entire winters.

Write down what your agent is allowed to do without a human, and you will know more about your project than any demo will tell you.

Nine out of ten enterprise AI agent pilots never reach production. That number comes from Deloitte’s 2026 research. The follow-up reporting is worse, because companies that did deploy have started pulling their agents back out.

Every explanation I read blames the model. Not enough context. Bad prompting. Wrong framework. Wait for the next release.

I do not buy it. I think most of these systems were built without anyone deciding what the thing should do when it was on its own.

Why do most AI agent projects fail in production?

The demo runs on a good day. Somebody is watching. The connection is up, the data is fresh, and the question is one the system was tested against.

Production is not a good day. Production is the morning a service is down, the data is four hours old, and the question is one nobody thought of.

The current reporting points at observability as the gap. Traditional monitoring watches whether software is running. It does not watch what an agent decided or why, so failures stay invisible until somebody downstream notices the damage and starts asking questions.

Observability tells you a problem happened. It does not stop the problem from happening.

The real cause is older and simpler. Nobody wrote down, before the build, what the system was permitted to do without supervision. So it does whatever the model produces, and nobody knows what it will do when it cannot reach a human.

What does it mean for an AI agent to be autonomous?

Autonomy is not a feature you buy. It is a decision somebody makes before anything gets built, and it has almost nothing to do with how clever the thing is.

A system is autonomous when somebody has written down what it may decide alone, what it must wait for, and what it does when waiting is not an option. A capable system with no such document is not autonomous. It is unsupervised.

The industry has spent three years talking about capability and almost none on permission. That is backwards. I worked on a system built the other way, and it ran for months without anyone watching it.

How can a system keep working after it loses its network connection?

In the 1980s I led a project for a water district in New Haven, Connecticut. It is the most interesting project I have ever worked on.

The system covered several thousand remote devices scattered across the region. Sensors reading levels and flows. Mechanisms that opened gates and dams, drained tanks, and moved water where it needed to go. All of it reported to a central server, and the operations staff made the calls from there.

That much is a normal control system. One thing made it different.

New Haven gets snow. A lot of it, and it stays. The communications ran over phone lines, because in the 1980s that was the entire list of options, and everyone involved understood those lines would go down in winter. Not might. Would.

Nobody treated that as a risk to be mitigated. It was a fact to be designed around.

And the water still had to flow. A city does not accept a service interruption because the phone company is having a rough quarter. Tanks fill. Gates open. People turn on taps in January and expect water.

We had a system that would routinely lose contact with every human being responsible for it, for weeks and sometimes months, and it could not stop working during that time.

That is a harder constraint than anything I have seen in an AI project. Nobody could say they would monitor it closely, because for weeks at a time there was no way to monitor it at all.

Can a behavior profile replace an AI model?

A behavior profile can replace a model for a narrow job, and at New Haven it did. Every remote station held a profile of its own behavior.

Each unit held a picture of what normal looked like at that specific location, built from the previous few years of water flows, weather, and the other variables that mattered where it happened to stand. It knew what February usually required. It knew what a heavy snow year did to demand.

When the line dropped, the station stopped waiting for instructions it was not going to get. It started deciding from that picture instead. What to let flow, what to hold, what to open, what to leave alone.

It ran on parameters and state tables rather than on anybody’s judgment, because nobody’s judgment could reach it.

There was a genuine genius on that project who designed it. I led the work. He built it, and I have been borrowing from that design for forty years.

People sometimes ask whether that counts as AI. I do not think it does. There was no model, no training run, and no learning of any kind. It was a profile, a set of parameters, and a decision table.

The system was not intelligent by any standard. It was still autonomous, because somebody had decided in advance what it could do alone.

What should an AI agent do when a service it depends on is unavailable?

Ask what your agent does when a service is unavailable, and most of the time nobody knows.

The question never came up. The agent calls a service, the service was there in testing and in the demo, and everybody moved on.

Then one day the service is not there, and the agent does something. It retries. It retries differently. It produces a confident answer built on nothing, or it loops until somebody notices the cost.

The current reporting describes this in detail. Tool calls fail at meaningful rates in production because enterprise interfaces were never designed to be called by machines at machine speed. Rate limits nobody documented. Format inconsistencies. Behavior that only appears under real load.

None of that is unusual. Every system that depends on another system has always worked this way.

The New Haven design started from that condition rather than discovering it in production. The connection was assumed to fail, so most of the engineering went into what happened after it failed.

We built the failure mode first and the normal mode second. I have not seen an AI project do that.

How do you decide what an AI agent can do without approval?

The New Haven answer was specific and it was settled before anyone wrote a line of code.

Under normal conditions the operations staff decided. That was the design. People with context and judgment made the calls, because those people were sitting right there and could be asked.

The stations decided only when the staff could not be reached. And what they were permitted to decide was bounded by the profile, which meant they could act inside the range of normal and could not invent anything new.

That is the distinction most AI projects get wrong. The station was not trusted to be clever. It was trusted to keep doing the ordinary thing while nobody was available to ask for something unusual.

Compare a bounded station with an agent handed broad system access, a general-purpose model, and no written boundary. That agent is being trusted to be clever. Some night it will be clever about the wrong thing with nobody there.

The prevention is a document, written before the build, that says what this thing may do alone. No vendor sells it.

What happens when an AI agent makes a wrong decision nobody catches?

The failure is rarely dramatic. Dramatic failures get fixed the same day.

An agent makes plausible decisions that are wrong, and nobody catches them for a while, because the wrong output looks like the right output. The rollback reporting describes this repeatedly. The system does not crash. It produces bad answers that look fine.

Then somebody finds it. Now the company has two problems, and the second one is that nobody trusts the system anymore, including the people who pushed for it.

Most rollbacks come from lost confidence rather than from an outage. Confidence takes far longer to rebuild than software.

Do small companies need AI governance?

A large enterprise has a governance function. Somebody’s entire job is writing down what systems may and may not do, and a committee meets about it.

A company with forty people has none of that, and gets told constantly to move fast and sort it out later.

The smaller company has one advantage. One person can keep the entire picture in their head. The decision about what an agent may do alone can be made by someone who understands the business, in an afternoon, on a single page.

That page is worth more than any governance platform on the market. Almost nobody writes it, because writing down limits feels like slowing down.

The companies that skip the page are the ones in the ninety percent.

What can a 1980s control system teach an AI project?

We delivered the New Haven system. That was the job, and then the job ended.

I never found out how it performed in a real winter. I would still like to know. I have wondered about it for close to forty years.

The company I built it for does not exist anymore. The system almost certainly outlived it, because water districts do not replace working infrastructure on a vendor’s schedule.

I did not take a technique away from it. Nobody needs the specifics of a decision table from the 1980s.

What I took was the order of operations. Decide what the thing may do alone. Decide what it does when it cannot ask. Then build it.

Every agent project I read about does those three steps in reverse, then blames the model.

The Guides That Get Your Book Written, Published, and Sold

Four short, practical guides on writing, publishing, and selling your book, plus the occasional note when there's something worth your time. No fluff, no daily inbox clutter. Drop your email and they're yours.

We use MailerLite to manage our list and send these emails. Your address is used only to send you what you signed up for. We will not sell it, share it, or use it for anything else, and you can unsubscribe anytime.

Frequently Asked Questions

Is an AI agent the same thing as automation?
No, and the difference is where the decision lives. Automation follows a rule somebody wrote down. An agent chooses what to do, which means the rule about what it may choose has to live somewhere else, and that rule is what most projects never write.
Should a small company deploy AI agents at all?
Yes, for bounded work with a clear failure mode. The trouble starts when an agent gets a long task, broad access, and no written limit on what it may do without asking. That combination fails at about the same rate whether the company has forty people or forty thousand.
Who should write the boundary document for an AI agent?
Somebody who understands the business consequences, not the person building the agent. The document says what the agent may decide alone, what it must escalate, and what it does when a dependency is unreachable. In a small company that is usually the owner or the operations lead.
Did machine autonomy exist before modern AI?
Industrial control systems have run unsupervised for decades using behavior profiles and decision tables instead of models. They were not intelligent by any current standard and they were autonomous, because somebody decided in advance what they were permitted to do alone.
Can better prompting fix an agent that fails in production?
Rarely, because prompting changes what the agent says rather than what it is permitted to do. If the failure happens when a service is unreachable or the data is stale, no wording fixes it. The fix is a written boundary and a defined behavior for the failure case.
How long can a system safely run without supervision?
As long as everything it is permitted to do stays safe with nobody checking. That is a question about the boundary rather than about the technology. A narrow boundary can run for months. A broad one can become a problem within the hour.

📁︎ Technology

🏷︎ AI🏷︎ IT Operations🏷︎ Technology

📝 Disclaimer

The views and opinions expressed in this blog post are solely those of Richard Lowe and are based on personal experience and research. This content is for informational purposes only and should not be construed as professional legal, financial, accounting, or business advice. Always consult with qualified professionals before making important business or legal decisions. Richard Lowe is not a lawyer, accountant, or licensed professional advisor, and this content does not establish any professional relationship.