The Cracks in AI Search Nobody’s Warning You About

In the first part of this series I argued that AI search does not kill your website, it raises the stakes on having a good one, and I meant every word. The opportunity is real. I would not have spent months of my own time chasing it if it were not. But I would be lying to you by omission if I stopped there, because the same machines that create that opportunity are broken, and almost nobody in the business of selling AEO will tell you how.

I will. I went deep into this for my own business, and a lot of what I found was ugly. These are the cracks you need to know about before you build, because some of them are merely frustrating and some of them can do real damage to your name while you are not looking.

You cannot see how you are doing, or why

In old-fashioned search, you could at least see the board. There were tools that told you where you ranked for a given term, whether you were moving up or down, what was working. It was imperfect, but it was legible. You could form a theory, make a change, and watch what happened.

AI search gives you almost none of that. There is no reliable rank tracker for whether ChatGPT recommends you. The criteria these engines use to choose a source are opaque, and they are opaque even to the people who do this for a living. I could see, sometimes, that I showed up in an answer. What I could never see was why that source got chosen over another, or what to change to show up more. You are optimizing in the dark, working from guesses and indirect signals, with no scoreboard to tell you if any of it landed.

It gets stranger. Ranking well in traditional search does not guarantee you get cited in the AI answer. A page sitting at the top of Google can be completely absent from the AI’s response, while a lower-ranked page gets named again and again. The two systems do not move together the way you would expect, which means the playbook you spent twenty years learning does not reliably transfer.

There is no “the AI.” There are dozens, and they disagree

Here is the thing that quietly makes every other flaw worse. People say “the AI” as if it were one thing. It is not. ChatGPT is one machine. Gemini is another. Perplexity, Copilot, Claude, Grok, and whatever launches next month are all separate, and that is before you count the fact that each one ships new versions that behave differently from the version before it. You are not optimizing for a system. You are optimizing for a crowd of them, and they do not agree.

They pull from different sources. They weigh authority differently. They make things up at different rates. One leans hard on Reddit and reviews, another favors established publications, another quietly prefers whatever it was trained on and barely looks at the live web at all. A change that makes you more visible in one can do nothing in a second and actively bury you in a third. There is no single lever, no shared rulebook, no version of “do this and you will show up.” You do the work, then watch it land in one place, miss in another, and backfire in a third, with no way to tell in advance which will happen.

For a business owner this is exhausting in a specific way. You cannot pick one and ignore the rest, because your potential client might be asking any of them, and you have no idea which. So you are forced to chase a dozen moving targets at once, each opaque, each slow, each wrong in its own way, none of them telling you the rules. That is not a search engine you can learn. That is a field of them, all drifting, and you are trying to be the right answer in every one at the same time.

I did not arrive at that from theory. I ran the exact same question, a request for a factual profile of me with sources, cold through every major engine on the same day. The spread was not subtle. One of them, Copilot, pulled my current numbers, found the right material, and gave a clean profile with no hedging. Two others handed back a mix of current and years-old figures and openly flagged that they could not reconcile them. One repeated a flat error I will get to in a moment. And one, Gemini, generated an answer and then erased it in front of me before it finished, producing nothing at all. Five engines, the same question, the same day, and a range that ran from accurate to useless. There is no “the AI” to satisfy. There is a committee that does not agree, and you have to win over all of them.

It is slow to learn and slow to forget

Here is the one that surprised me most. AI search is slow in both directions, and both directions hurt.

It is slow to pick up new or corrected information, and slow is generous. Here is the one that has beaten me for years. I am a working professional with ten years of changes behind me. My book count goes up. My rates change. The kind of work I take shifts. That is what a real career looks like from the outside: the facts move, because the person is alive and still working. The machines cannot handle it. When I ran my profile test, the gap was right there in the numbers. The current, correct figure for what I have written sat on my own pages, and one engine found it cleanly. The others reached past it and pulled a book count I passed years ago off scattered third-party pages, then served it up as current. Two of them surfaced the old number and the right number side by side and admitted they could not tell which was true. I have corrected my own source again and again, and the dead figures keep crawling back into the answers because they are sitting on enough other pages to outvote me. It is the hardest problem I have hit in my own AI presence, and I have not solved it. A static fact is easy. A fact that changes the way every living person’s facts change is something these systems are simply bad at holding onto.

And it is slow to let go of bad information. The freshness problem cuts the other way too. Content that is not kept current decays out of the answers, and stale errors can sit in the machine’s picture of you long after you have fixed the source. So you are punished twice: slow to be believed when you are right, slow to be cleared when the machine is wrong. The only real defense is to keep your home base current and consistent enough that, whenever the machine does finally update, it updates toward the truth. This slow rot has a name worth knowing, because drift is the failure mode almost nobody is watching for.

When it is wrong about you, there is no one to call

This is the flaw that should worry a business owner most, and it is the one people understand the least until it happens to them.

AI engines get things wrong. Not occasionally, structurally. They generate confident, fluent, completely false statements, and the rate is not trivial. Depending on the topic, studies put the rate of fabricated output anywhere from a few percent to more than a quarter of responses, and it climbs higher on niche subjects, which is to say, on exactly the specific, lesser-known topics that describe most individual businesses and professionals. The more particular your niche, the more room the machine has to make something up about you.

Now the part with no good answer. When the machine invents something false about your business, there is no edit button. No form. No support line. No “report this” link that goes anywhere real. It states the falsehood to whoever asked, and you usually never find out it happened. This is the same wall as the stale book count, except now the wrong fact is not just outdated, it is invented. In my own profile test, three separate engines credited me with the same book. I did not write it. It is not mine. One engine making that mistake is a hallucination. Three engines making the identical mistake means there is a page somewhere out in the crawled web that has wrongly tied my name to that title, and every engine that found it repeated the error with total confidence. There is nowhere to send the correction, no one to email, no lever to pull. I own my source and can make it say anything I want. The machines kept saying what they had picked up somewhere else. This is worse than a simple mistake, because the machine does not just invent things, it distorts the real data it has about you.

This is not a hypothetical risk. A small solar company in Minnesota found that an AI engine was repeatedly and falsely telling people it had been sued by the state. The company tied real client cancellations and lost sales to it. They did not do anything wrong. The machine simply manufactured a damaging falsehood and told it to their potential customers, and there was no easy button to make it stop. Think about what that does to a business that runs on trust, which is every business.

There is no appeal, and “the AI said it” is not a defense

Search had a feedback loop, clumsy as it was. You could dispute things, file requests, work through a process. AI search effectively has none. There is no court of appeal for what the model believes about you. You cannot challenge it, cross-examine it, or force a correction. The documented recourse, when there is one at all, is a lawsuit, which is an absurd and unaffordable answer for a small business that just wants a falsehood taken down.

And do not assume the company behind the machine will absorb the blame. The accountability tends to flow toward the business, not away from it. When Air Canada’s own chatbot invented a refund policy that did not exist, the airline tried to argue it was not responsible for what its bot said. A tribunal disagreed and held the company liable. The lesson cuts in an uncomfortable direction: “the AI said it” did not protect the business when the AI was its own. You should not count on it protecting you when the AI belongs to someone else and is talking about you.

It can be corrupted on purpose

This is the one that genuinely unsettled me, because it turns a passive problem into an active threat.

The same openness that lets you build authority with a strong home base also lets a determined bad actor poison the well. Researchers have demonstrated this in controlled studies, and the results are sobering. By planting carefully constructed content where the machine would find it, they made a target product several times more likely to be recommended. In head-to-head tests, seeding the right material made one option two to eight times more likely to be chosen over a competitor. Worse, they showed you can attack in the other direction too: content engineered to disparage a rival, to label it unsafe or untrustworthy, can train the machine to quietly suppress that competitor in its answers.

Sit with what that means. A competitor willing to spend the effort, or a stranger who simply wants to hurt you, can plant false signals built to inflate themselves and bury you. There is no referee. No one is checking. The machine swallows what it finds and repeats it with the exact same confidence it uses for the truth, and it cannot tell the difference. The system has no immune response. Anyone who tells you the AI playing field is fair is telling you something they have not tested. It is not fair, it is not policed, and it can be bought.

If you want the broader context on how these systems get unreliable and how the errors compound, I have written more about the mechanics over on my fiction site, including a closer look at what the AI writing research actually says. The short version is that the failure modes are real, measurable, and not going away on their own.

It takes your work and sends no one back

There is a quieter flaw that costs you in a different way. The machine can absorb your content, your insight, the article you spent hours writing, and repeat the substance of it in an answer without sending anyone to you. You did the work. The machine delivered it. The person got what they needed and never learned your name, never visited your site, never had the chance to become a client. The credit, the click, and the relationship all evaporate into a summary. You become an uncredited supplier to a system that gives you nothing back for it.

You cannot audit what it read

And underneath all of it, you cannot see the inputs. You do not know which sources the machine weighed, which it trusted, which it ignored, which bit of stray nonsense it happened to pick up about you. You are guessing at what the machine has read, which means you are optimizing against a black box. Every other flaw on this list gets harder because of this one. You cannot fix what you cannot see, and you cannot see almost any of it.

So what do you actually do

If I have done my job, you are now appropriately wary and wondering whether this is worth it at all. It is. I have not changed my mind from part one. AI search is where a growing share of decisions are getting made, and opting out is not a strategy. But you go in clear-eyed, and you build the one defense that actually works against this whole list of problems.

That defense is the same home base I spent the first part of this series arguing for. You cannot control the machine. You cannot make it fast, make it honest, make it fair, or make it accountable, and you cannot chase every one of the dozen engines on its own terms. What you can control is one thing: how consistently the truth about you appears across the web, anchored to a strong, accurate, owned source. That single move is the one thing that works on all of them at once, because a clear, deep, consistent record is the closest thing there is to a signal every engine respects. When the machine is slow, a consistent home base is what it eventually updates toward. When the machine is wrong, a corroborated record is what makes the falsehood harder to sustain. When someone tries to corrupt the picture, a deep and consistent footprint is what is hardest to drown out. Every one of these flaws punishes businesses with thin, scattered, inconsistent presences, and rewards the ones who built something solid early and kept it current.

The honest takeaway is not doom. It is this. AI traffic is worth pursuing, and the businesses that win at it will not be the ones chasing a hack or buying a service that promises to game the system. They will be the ones who built something real, made it the clearest and most authoritative source about themselves, and made it easy for the machines to get them right. The flaws are exactly why that work matters. In a system this opaque, this slow, and this corruptible, being the strongest honest signal in the room is the only move that holds up. That is what I actually tell clients about AI when they ask whether they should worry about it.

Frequently Asked Questions