TL;DR: An edge case is any situation outside the pattern an AI was trained on. Machine learning systems are structurally incapable of handling them, because the system has never seen them before and has no judgment about what to do when it doesn’t know. Edge cases cost a wrong answer at small scale, a lawsuit at medium scale, and a death at large scale.
The only fix is a human in the loop wherever the cost of a single failure exceeds the convenience of automation. More on how I keep a human in the loop. Map your edge cases before you deploy, before a customer or a regulator finds them for you.
An edge case is the part of the job your AI doesn’t know how to do.
Not the part it does badly. The part it’s never seen before, has no example of in its training data, and has no way to recognize that it’s in trouble. The part where the system gives an answer with full confidence and no idea that the answer is wrong.
Edge cases are a structural feature of every machine learning system that’s ever shipped. They’re also the reason most AI deployments fail outright or slowly hurt the business that paid for them.
AI rollouts that treat edge cases as a rounding error are negligent. The vendors want you staring at the 95 percent the system handles. I care about the one customer it’s going to fail, because that customer is a real person with a real problem, and the company that deployed the bot leaves them alone with it and books the savings.
What is an edge case in AI?
Machine learning works by finding patterns in examples.
You show the system a million pictures of cats labeled “cat” and a million pictures of dogs labeled “dog,” and it learns to tell them apart with high accuracy. The system is matching patterns. When a new picture comes in, the system asks: does this look more like the cat pattern or the dog pattern? For more, see mit research shows ChatGPT weakens your brain and AI and writing.
An edge case is the part of the job your AI doesn’t know how to do.Share on X
That’s the entire mechanism. It works beautifully when the new picture is, in fact, a cat or a dog.
The edge case is the picture that’s a fox, a ferret, a stuffed animal, a cat-shaped cloud, or a dog in a cat costume. The system has never been shown those examples. It still has to give an answer. So it picks one, with full confidence, and gets it wrong.
Now scale that mechanism up to the actual jobs companies are deploying AI for. A customer service bot trained on thousands of routine billing questions meets an angry customer whose card was charged twice for a canceled flight.
The bot has no example of what to do. A legal AI trained on standard contracts hits a clause it’s never seen. A medical AI trained on common presentations hits a rare condition that mimics a common one. Each system gives an answer, with full confidence, and gets it wrong.
I’d pull that customer service bot the first time it told a double-charged traveler something confident and wrong. The customer doesn’t care that the model scored well on the easy questions. They care that they’re out the money and the machine won’t admit it’s lost.
Why doesn’t training fix AI edge cases?
The obvious response is: just train the system on more examples. If the AI is failing on angry customers, train it on more angry customers. If it’s failing on rare medical conditions, train it on more rare conditions. This works, partially, on the cases you can predict. The problem is that the edge cases you can predict aren’t really the edge. The actual edge is the case nobody on the team thought to put in the training set, because nobody could imagine it.
And the world produces those constantly. New edges arrive constantly. Every new product. Every new customer demographic. Every new regulation. Every new fraud pattern. Every new emotional state a real human walks in with on a Tuesday morning. You can’t train your way out of this problem. You can only build the system to recognize when it’s outside what it knows and hand the job to a human.
It’s the only fix. Anybody who tells you more training data will close the gap is selling you something, and it’s the executives who believed them who end up explaining the failure to a board or a judge.
The cost at three scales
The reason this matters is that the cost of an unhandled edge case doesn’t stay constant. It scales with what you put the AI in charge of.
Small scale: a wrong answer
At the bottom of the stack, an edge case is a wrong answer. A chatbot tells a customer the wrong shipping date. A search assistant invents a citation that doesn’t exist. A spreadsheet AI mislabels a column. The customer gets frustrated, the analyst catches the mistake, life goes on. Annoying, recoverable, and you measure the cost in time. This is the level most people think AI failures live at. They don’t.
Medium scale: a lawsuit
The next level up, an edge case is a legal exposure. Air Canada deployed a customer service chatbot that invented a refund policy that didn’t exist. A customer relied on the chatbot’s promise.
The company refused to honor it. The customer took them to court, and the court ruled the company responsible for what its chatbot said. The “the chatbot was wrong” defense didn’t work.
The medium-scale edge case looks like this. A hallucinated answer your AI gave to a customer, with full confidence, that becomes a binding promise you have to either honor or fight in court. Multiply that across the volume of interactions a customer service AI handles, and the numbers get ugly fast.
The defense is the ugliest part of the Air Canada case. A company put a bot in front of its customers, the bot made a promise, and the company’s first move was to argue it wasn’t responsible for what its own bot said. If you won’t stand behind what your chatbot tells people, you had no business letting it talk to them.
I get into the failure mode of confident invention in AI Hallucination: A Survival Guide for People Who Publish Under Their Own Name. The mechanism is the same whether the output is a customer service answer or a chapter of your book.
Large scale: a death
At the top of the stack, an edge case is a person’s life.
Self-driving cars are the most public version of this.
The system handles every routine driving scenario it trained on. The edge case is the pedestrian whose silhouette the model has never seen. The bicycle wobbling in a way nobody’s ever shown it. The traffic backup caused by something it can’t categorize. The cost of the unhandled edge case at this scale is a human being.
Medical AI sits in the same category. So does any system that decides who gets a loan, who gets a job interview, who gets parole, who gets removed from a no-fly list. The person on the wrong end of those decisions doesn’t get to cross-examine the model, and I find that indefensible. When the system is confidently wrong about somebody who doesn’t fit its training data, that person can lose a job, a home, years of freedom, or a life.
What do edge cases mean for an AI rollout?
From the podcast
Ben “The Automator” Christensen has a name for the mechanism that catches the case nobody planned for, on Leaders and Their Stories:
If you could have your system raise its hand when it’s not sure about the outcome, or it experiences some type of weird error, that’s what bringing human in the loop is.The system doesn’t have to solve the edge case. It only has to know that it’s met one.
Before you deploy any AI system into a real workflow, you have to answer one question. What does the cost of a single failure look like, and who pays it? If the answer is “the user has to retry the search,” fine. Ship the system, let it fail occasionally, and move on. The cost is small, the cost lands on the user, and the user has another search engine they can use.
When the answer is “the company has to honor an invented refund policy,” you have a problem the law has already decided. Build a human in the loop for any answer that creates an obligation. The cost of that human’s time is less than the cost of the lawsuit you’ll eventually lose.
Where the answer is “a person gets hurt,” you can’t deploy this system without a human in the loop, full stop. The cost of getting this wrong is something money can’t get back, and no amount of training data ever solves the next edge case the world will produce.
Klarna-style rollouts all make the same mistake: they answer the cost question with the 95 percent the AI handles and never price the other 5 percent. That 5 percent is where your angriest customers, your legal exposure and your worst headlines live, and a company that won’t staff for it has decided those people don’t count.
What is the rule for AI edge cases?
Keep a human in the loop wherever the cost of one unhandled edge case exceeds the convenience of automating the routine ones. And that’s the whole rule. The work of adopting AI well is the work of mapping where that line falls for every system you deploy. Most rollouts fail because nobody did the mapping. They benchmarked the AI against the easy cases, took the savings, and waited for the hard ones to arrive.
The hard ones always arrive, and I think any leader who deploys AI without deciding in advance who catches them is being reckless with other people’s money, time and safety. Do the mapping before launch, put a person wherever one failure costs more than the automation saves, and stop treating the edge cases as somebody else’s problem.
