TL;DR: AI handles research surprisingly well for one kind of work and dangerously badly for another, and the difference between safe use and a published embarrassment comes down to knowing which is which. Background reading, terminology, conceptual orientation, and overview synthesis are jobs AI does competently when verified. Specific citations, named quotes, dates, statistics, and any claim you’d put in print are jobs AI invents confidently and you must check by hand every time.
AI research scares me more than AI drafting does. A flat paragraph makes a book dull, but an invented quote pinned on a real person can get an author sued, and the machine will hand you one in the same pleasant tone it uses for everything else. I think it’s a terrific orientation tool and a lousy source, and an author who’ll treat its output as a source shouldn’t use it for research at all.
The research promise is real
An author working on a serious book spends real hours on background: the history of an industry, the current state of a debate, the terminology a specialist would use, the major arguments on each side.
That reading is necessary and slow, and most authors never finish it. So the background gets skipped, and the book comes out shallower than the author intended. I think that’s a bigger quality problem in business books than most people in publishing will admit.
AI changes the timeline here. Background reading that would take a week can become useful working notes in an afternoon. The author checks the summary, then follows up on the sources that matter most. Reading time drops from weeks to days.
The depth of the book goes up, because the author finally does the background work at all. That gain is real, and it’s one of the strongest cases for AI on a book project.
Watch: Don't Be Fooled By AI-Hallucinated Citations: Be Cautious When Literature Searching!, nglreference
The hallucination reality is also real
The same model that gives you the useful summary will invent a citation, fabricate a quote, misattribute a statistic, or get a date wrong by two decades, and it reports all of it in the same calm tone of authority.
The machine has no internal sense of which facts are true and which it just generated. That blindness is structural, it exists in every current model, and it shows up more than once in a long document. It’s the part of this technology I trust least.
The published failures are well documented. The lawyer fired for filing a brief with invented case citations. The newspaper that ran a fake quote attributed to a real person. The book pulled from a platform after readers found made-up references. In every case the human didn’t verify. The hallucination survival guide walks through the verification habits at length. The central point is simple. The machine states things confidently that aren’t true, and the only protection is checking every specific claim by hand before it ships.
Which AI research failures catch authors?
Five categories of output deserve constant suspicion. The machine routinely fabricates direct quotes from real people, sometimes with the right tone of voice for the person quoted. That makes them feel real. It invents citations to academic papers and books, with plausible authors and titles that don’t exist.
Dates and timelines drift, especially on events from before the model’s training cutoff. It fabricates statistics wholesale or attributes them to studies that don’t say what the machine claims they say.
Names of secondary figures get scrambled, with the wrong person credited for the right work or the right person credited for work they didn’t do.
Each of those failure modes has put authors in the news, and each one is preventable by verification. The catch is that all five feel exactly the same as correct output. The machine doesn’t flag the fabricated quote and the real one differently. They both arrive in the same calm voice with the same surface confidence. Verify each one however plausible it seems. Plausibility is what the machine optimizes for. That makes it exactly the wrong signal to trust.
A growing number of clients reach me after trying to write their book with AI and getting a bad result.
If a draft like that leaned on AI research, I’d treat every quote, date and statistic in it as unverified until somebody checks it against a real source. A flat paragraph can be rewritten in an afternoon. A quote attributed to a living person who never said it is a legal problem.
Where is AI research safe to use?
The safe zone for AI research is broader than skeptics admit, but its boundaries matter. Background overview of a topic you already know enough to spot-check. Terminology orientation, where the machine tells you what a specialist would call something so you can search more effectively from there. Conceptual scaffolding, where the machine outlines the major positions in a debate so you know where to look next. Summary of public documents the machine couldn’t plausibly have hallucinated, like a famous speech or a well-known report, where you can quickly verify the summary against the source.
In every safe case, the author is using the machine as an accelerant on work they could verify by hand and is verifying the parts that matter. The machine isn’t generating new claims you’ll quote in print.
The machine is helping you orient faster in territory you’ll then explore yourself. That use is valuable, and it’s where most of the research savings on a book project come from. A longer piece on what AI is good at covers more of these safe applications.
Where AI research never goes solo
The unsafe zone is everything that ends up in print without verification.
A direct quote you plan to attribute to a real person must come from a verified source. Statistics you plan to cite come from the actual study, read by you or your researcher. Never from a machine’s claim about what the study said. Historical dates you plan to put in the book must come from a verified source.
Claims about a specific person’s actions, statements, or positions must come from documented evidence. Machine inference doesn’t count.
This is the rule I care about most. When a client wants to make a claim about a real person, I brief them on the legal risk the way a trusted advisor would and show them how to change the details so the story still works. A machine will cheerfully invent that person’s motives for you and never mention that you’re the one who’ll answer for them.
The pattern is simple. Anything specific enough to be wrong is specific enough to need verification. The machine is a research accelerant for the orientation phase. The machine is a hazard for the citation phase. That line is where the verification habit starts. Cross it without checking and you’re gambling your publication credibility on a machine structurally incapable of knowing what’s true.
How do you verify AI research?
From the podcast
Martin Ricketts, who has spent thirty years predicting technology trends, keeps a hard line between what he trusts a chatbot for and what he doesn’t, on Leaders and Their Stories:
There’s things, especially factual things, Google is still way better at getting than any of the AI tools.Treating an answer engine as a replacement for a search engine is where most research trouble starts.
A machine will cheerfully invent that person’s motives for you and never mention that you’re the one who’ll answer for them.
Three habits keep this honest. The first is to mark every machine output in your working notes with the source the machine claims. Then verify each claim against a real primary source before any of it reaches the manuscript. Use a tag or a marker so unverified material is visually distinct from verified material in your working files. Don’t let anything cross from working notes into the book without the tag being cleared.
The second is to keep a citation log separate from your prose drafts, with each specific claim mapped to its verified source. I also keep my research outside the chat tool. When I was researching how to set up backups, I finished by having Claude create an HTML document I could download, and I put it in a little folder, so that research lives outside Claude.
A book where every specific claim has a verified source is a book that survives scrutiny. One where the claims are floating in the prose without source tracking is a book waiting to embarrass somebody. A piece on how AI distorts even your own input data covers the related failure mode where the machine subtly corrupts material you fed it. That’s harder to catch than outright fabrication.
The third habit is to read your finished chapters one more time looking for citations and factual claims, and to spot-check ten percent of them at random against your verified sources. If any of those spot checks fail, the failure is a signal to verify everything in that chapter.
I don’t have much sympathy for authors who get caught this way. The tool’s weakness is public knowledge by now, and “the AI told me” is an excuse that insults every reader who trusted the page.
My take
Use AI for background reading and terminology, and never trust it with a citation. Every claim specific enough to be wrong needs a primary source behind it. Tag every unverified claim in your notes and keep it out of the manuscript until the tag is cleared. Keep a citation log. Spot check your finished chapters. I don’t have much sympathy for authors who get caught, because “the AI told me” is no excuse.
What this means for the project plan
Build verification time into your schedule from the start. A serious nonfiction book that draws on AI research adds verification hours equal to roughly thirty percent of the research time itself. That sounds like a lot until you compare it to the time you saved on the original reading, and it sounds like nothing compared to the cost of finding out about an invented citation after publication. The verification time is the price of using the tool safely, and the price is small relative to the value the tool produces when used correctly.
The authors who get burned by AI research underestimated how confidently the machine fabricates and overestimated their ability to catch it by gut feel. Build the verification step in, keep it when the deadline gets tight, and treat every specific claim as guilty until you’ve checked it. Skip that and you’re letting a program with no idea what’s true publish under your name, and I think that’s a reckless way to treat a reputation it took you a whole career to build.
Research is where unchecked AI does the most damage. The Death of Thinking is my book on how handing over judgment slowly costs you the ability to catch the errors.
