The Web Got Fenced: What AI Search Costs Small Sites

The open web I grew up on let anybody get a link by writing something good. That’s gone. Visibility now depends on being mentioned in places that charge admission, a rigged game for every small writer who can’t pay. The web was built so anyone could publish and anyone could find it. Nobody had to be vouched for. That property was never written down as a principle. It was a side effect of how discovery worked: a volunteer directory editor, a ringmaster, a crawler following a link. Every one of those would consider a page nobody had heard of. It lasted about twenty-five years.

It’s gone, and the thing that replaced it asks a different question. The old one was whether a page was good. The new one is whether anyone will vouch for it, and that shift shuts out the people I care most about, the ones who write well and have nobody with a budget in their corner.

I watched a smaller version of this happen once before, from inside. I ran web rings through the 1990s and was there when a corporation took the system apart and the community fought back for years. We won that fight. Rings died anyway, because the road moved. That’s the pattern worth carrying into what follows.

How did links used to work as votes for websites?

PageRank had a democratic story attached to it, whatever its faults. A link was a vote, and a small site with something good could accumulate votes from people who found it useful. People gamed it constantly, and Google spent twenty years swatting them. The theory still pointed at merit.

It also had a property people underrate now. A link cost nothing to give. Any page could vote for any other page, and a page with no budget could receive a vote from a page with an enormous one. Money helped, because money bought content and outreach and eventually links outright. Money was never the only currency.

Nothing about the current model points at merit. The requirement is corroboration: other entities saying who you are, across platforms that mostly charge, or reward volume, or both. Yelp. Trustpilot. G2. Capterra. Clutch. Trade directories. Press wires. An AI system reads the consensus across those sources and repeats it.

Look at the list for a second. Every one of those is a gate somebody else works, and most of them collect rent at the gate. That’s a spend filter wearing the costume of a trust signal.

Watch: Google AI Overviews Kills Web: Publishers Report 92% Traffic Loss (EU Files Antitrust) 🚨, Lexicon Labs

How does AI search decide which websites to cite?

Mostly by looking at what other sites say about you, and only secondarily at what your own site says. The industry has a phrase for it, off-page corroboration, and the practitioners who get results are open about the mechanics.

Your own page can claim anything. A page you don’t control saying the same thing counts for more than your own claim.

In practice that means a shopping list. A listing on the review platform that dominates your category. Reviews on it, in volume, recent enough to look alive. A presence in whatever directory the model reaches for when someone asks about your field. Mentions in publications the system already trusts. An entity record it can attach all of this to, so the mentions resolve to one identity instead of scattering across a dozen half-matches.

Then the same claim, phrased the same way, repeated across all of it. The consistency is the point. A machine reconciling forty sources is looking for agreement, and variation reads as uncertainty.

I listened to two well-known search professionals debate this recently, and the frank part of the conversation was the money. One described a client whose monthly retainer went from ten thousand dollars to seventy thousand for the same category of work under a newer label. That’s the market pricing the gates.

That price tells you who the new system is built for. A funded company can absorb it. A novelist with a blog, a retired engineer with a hobby site or a two-person consultancy can’t, and I doubt anybody designing these systems has lost a minute of sleep over them.

The case for corroboration is stronger than I want it to be

I should argue the other side properly, because the engineering logic is sound.

A system that answers questions in prose carries a burden search results never did. A list of ten blue links pushes the judgment onto the reader, who can see the source and decide. An answer in a paragraph has already decided. Whatever it says becomes what the person believes, and there’s no second opinion visible underneath it.

That raises the cost of being wrong enormously.

A search engine that ranks a bad page has shown you a bad page. A system that synthesizes a bad page has told you something false in its own voice. The pressure to be right pushes hard toward sources that many independent parties agree on, because agreement is the only proxy for truth that a machine can compute cheaply.

People reason the same way. A stranger recommends a plumber and you weigh it lightly. Four unconnected neighbors recommend the same plumber and you call him. Nobody thinks that’s unfair when a person does it.

The difference is what agreement costs to obtain. Your neighbors aren’t selling their opinions. The platforms sitting between a small business and its corroboration are running a business, and the price of admission is set by them. I accept the engineering. What I won’t accept is that the only road to being believed now runs through companies that charge for it, and that small sites are expected to pay without complaint.

Every gate collects a toll

Look at what a full corroboration footprint requires and the shape becomes clear.

Review platforms are the obvious tier. Most let you claim a listing free and then charge for anything that makes the listing perform. Directories in professional fields often charge for membership before the listing exists at all. Industry association listings usually require dues. Press coverage is either earned through relationships that took years, or bought through wire services that will publish almost anything for a fee.

Awards deserve their own paragraph. A meaningful share of the awards that show up as corroboration are pay-to-enter, and some are pay-to-win in everything but name. They exist because a badge on a website and a mention in a listing feed the consensus. The machine has no way to tell a juried prize from a purchased one, so both count.

Add these together and the cost of being visible is no longer the cost of making something good. It’s a recurring subscription to being believed. A one-person business with a real specialty pays the same tolls as a funded competitor, from a fraction of the revenue, and the funded competitor can simply buy more of the same signals.

Why does a smaller search index hurt small websites?

Something structural sits underneath the corroboration question, and it may matter more.

One practitioner described a site getting a few hundred clicks from search while nine of its pages were fetched tens of thousands of times as citations. The machines were reading it.

The people weren’t coming. His read was that these systems aren’t searching the live web for each answer. They build a working table from a small slice of the index, then read from the table.

I have no way to verify that from the outside, and he presented it as an observation from his own data. If it’s even roughly right, the consequence is severe. A site outside that slice never enters a ranking contest at all. Nothing reads it. There’s no page two to climb to.

The economics make it plausible. Indexing the whole web is expensive and refreshing it constantly is worse. If a small fraction of pages answers the overwhelming majority of questions, then building and maintaining only that fraction is the rational engineering choice. Nobody has to decide to exclude the model train page. The budget decides.

There’s a second layer that behaves differently. What a model absorbs during training isn’t the same as what it fetches while answering. Training gives it language, general facts, and the brands repeated so often they became ordinary nouns. Retrieval gives it whatever is current. A small site has almost no path into the first and a narrow, contested path into the second.

Why does manufactured consensus work on AI search?

Because the system rewards what’s corroborated, and corroboration can be built. This is the second-order effect, and it’s worse than the unfairness.

The same conversation covered the working methods openly.

Press release blasts across four or five wire services. Force-indexing the resulting URLs. Buying cheap pop-under traffic through a redirect link to fake a virality spike, six thousand hits in two hours to push a page up. In the gambling vertical, deliberate poisoning of what the models hold about a competitor. One participant paused twice to say search engineers might be listening.

The demonstration that stuck with me was smaller and much more revealing. Two of them ran an experiment where one published a press release claiming he’d replaced the other as the leading figure in his field. Within hours the AI systems repeated it as fact. The other spent the rest of the day publishing counter-material, and by evening the story had flipped back. Nothing true had changed. A title with no meaning moved twice in twelve hours because two people took turns feeding the machine.

None of that improves anything for a reader.

It’s the skill the incentives select for. When visibility depends on consensus and consensus is for sale, the market stops caring about quality. That’s where we are. I wrote about the structural cracks in AI search from a different angle, and this is the same fault line seen from above.

Who loses when AI search replaces search engines?

Abstraction hides the specific casualties, so I’ll name them. The hobbyist with twenty years of accumulated knowledge about a narrow subject. No business, no reviews, no directory, no reason for any platform to list him. His pages are the best material on the internet for his topic and there’s no mechanism by which a model finds out.

The small nonprofit whose entire budget goes to the thing it exists to do. Every dollar spent on corroboration infrastructure is a dollar not spent on the work, and the funded organizations in the same field have staff for this.

The regional expert whose reputation lives in conversations, referrals, and a professional community that never wrote anything down. Corroboration exists in abundance and none of it is machine-readable.

The academic or the retired practitioner publishing out of interest instead of commerce. No incentive to build a commercial footprint, so no footprint, so no citations.

What connects them is that the thing they have is knowledge, and the thing the system measures is commercial presence. Those overlap sometimes. They’re not the same, and nothing in the current architecture can tell the difference. That’s the loss I’d put in front of anybody building these systems. The best page on a narrow subject is often written by someone with no marketing budget and no interest in getting one, and a machine that can’t find that page gives people worse answers while sounding completely sure of itself.

The loop closes on itself

Then it grows, which is why it’s hard to reverse.

A site that gets cited gets read. Being read produces mentions, and mentions produce more corroboration. That produces more citations. Meanwhile the site nobody cites gets no traffic, generates no mentions, accumulates no signals, and slides further from the slice that gets read at all.

Search had this problem too, and it was real, but a link from anywhere could break the cycle. One well-placed mention could pull a site into visibility.

In a corroboration model the entry requirement is a set of platform presences, and platform presences don’t arrive by accident. Somebody has to build them deliberately. That means somebody has to fund them.

Over time the gap between the visible and the invisible hardens into a wall, and a site on the wrong side of it can’t climb over it with better work.

What can a small website do about AI search?

Less than I’d like, and more than nothing at all.

Be legible. State plainly who you are, what you do, and who you serve, on a page built for that job.

Machines doing the indexing run cheap and do no reasoning, so they never draw a conclusion you leave implicit. If your credentials imply expertise, write the sentence that says so. The machines won’t make the inference for you.

Be consistent. The same description in the same words everywhere you appear. A machine reconciling your identity across sources treats variation as uncertainty, and uncertainty as a reason to cite somebody else. This costs nothing and most sites get it wrong. Take the corroboration that’s free. Professional association listings you already pay dues for. Directories in your field that don’t charge. Interviews, podcasts, and mentions you earn by being worth talking to. That’s the same mechanism the paid version buys, earned slowly and cleanly. Answer questions in the form people ask them. Headings phrased as the actual question, answered directly underneath in a passage that stands alone. A page written as one long argument gives a retrieval system nothing to lift. A page with clear answers to real questions gives it something to quote.

Build the thing that can’t be purchased. A body of specific work that exists nowhere else is the one asset the machinery can’t manufacture. Original research, a documented method, an archive built over years, an account of something you lived through. It won’t put a small site above a funded one. It gives that site a reason to be cited when it does surface, which is what a citation is now worth.

Refuse the fabrication playbook. The press release blasts and the traffic spikes get tuned to whatever the detectors look for this quarter. It’s a cat-and-mouse game, the mice are well paid, and honest sites get trampled in the middle. The people selling those tactics will sell you next quarter’s version too, so you’d be renting a position and paying rent forever. I won’t do it on my own site, and I’d talk any client out of it.

The loss is real even if you can work around it

I can work inside this system. I’ve spent considerable time getting my own site legible to it, and I’ll spend more. Being able to compete within a set of rules doesn’t make the rules fair, and it doesn’t make the loss smaller.

What went away was a web where discovery didn’t require permission. A page could exist because one person cared, and another person who cared could find it, and no intermediary had to agree that either of them counted. That property was strange and valuable and mostly accidental, and it’s gone.

Something else went with it. The old system, for all its gaming, at least aimed at a question with a real answer: is this page useful. The new one aims at a question with a purchasable answer: does anyone vouch for this page. Those look similar from a distance. They select for completely different things, and only one of them rewards doing the work well.

In 2000 I thought the enemy was a corporation that didn’t know it was in a fight, and I was wrong about that too. The corporation was the visible part. What ended web rings was that the road moved, and after it moved there was no reason to drive the old one.

The road has moved again. Before you spend money trying to get onto the new one, read what the AI visibility industry is selling and what the evidence says about it, because most of it doesn’t work. Then start with the AI writing hub for the practical side, and look at what I do with AI and content if you’d rather not work it out alone.

Quality alone doesn’t get you seen anymore, and pretending it does won’t help. I hate that.

The web I grew up on rewarded whoever wrote the best page, and this one rewards whoever paid to be mentioned. Write the best thing you can anyway, get it mentioned where the machines look, and keep a site you own as the home base, because the platforms in between will change their terms again.

From the author

The Day Your Website Died
Small sites now compete for AI visibility on uneven ground. In The Day Your Website Died I cover optimizing for AI search from the ground up.

Frequently Asked Questions