This started as an essay and kept getting longer, because the thing I was noticing kept connecting to other things.
The history of how technologies change cognition. The economics of AI development and who it serves. The education system that had already been moving in this direction long before any of these tools existed.
What I was noticing was simple enough to state in a sentence. People are outsourcing the one thing they should never outsource.
What is the one thing AI cannot replace?
The reasoning itself. The part where you decide what’s true and what matters.
Everything else is negotiable. Drafting can be handed over. Assembling research can be handed over. Formatting, summarising, restructuring, all of it can go to a machine and the result is usually fine and often better.
What cannot go is the judgment about whether the output is right, because that judgment is the thing everything else depends on, and it exists only in people who have exercised it.
The trouble is that a tool which produces a plausible answer removes the occasion for exercising it. Not deliberately. It simply supplies the thing that used to require the work, and the work was where the capacity came from.
Why is the AI-assisted writing argument awkward to make?
Because of the sequence, and because I’m on the wrong side of it to be comfortable.
This argument isn’t do as I do. This argument is: do what I did before I’d the tools, and then do what I do with them.
I spent decades building judgment without any of this. Thirty-three years in enterprise technology where being wrong had consequences and there was nothing to ask. By the time these tools arrived I’d a foundation, and I use them constantly, and I can tell when the output is wrong because I built the ability to tell somewhere else.
That instruction is awkward to deliver now that the tools arrived before most people had the chance to develop the foundation. It’s not advice anybody can act on retrospectively, and I’m aware of how convenient it’s for me that the sequence worked out in my favor.
Is the argument about AI replacing skills actually new?
No, and that’s the part people miss when they treat this as a story about a technology.
The education system was already moving in this direction. The economics of the tools reward engagement instead of capability. The long erosion was underway well before anything could generate an essay, and the tools accelerated a process and not starting one.
Which means the fix isn’t a policy about a technology. If the underlying problem is that judgment develops through difficulty and we have been systematically removing difficulty for forty years, then banning a chatbot addresses the last few years of a much longer trend.
What did the pandemic show writers?
Something small and clarifying, which I’ve thought about since.
During the pandemic I watched two neighbors in their early twenties work out that the government relief payments meant they could take a year off. They got paid to do nothing, and they took the deal.
I told them they should be doing something with the time. I wasn’t being moral about it. My concern was that a year is long enough to lose the habit of applying yourself to something difficult, and the habit is harder to rebuild than to keep.
They were behaving rationally. Anybody would. That’s what makes it a useful example instead of a complaint about young people: the incentive was there, and the response was reasonable, and the cost was invisible at the time.
What should a writer do now?
It depends where you already are, and most advice on this refuses to admit that.
If you have the foundation, use the tools freely and check their work, because you can. If you don’t, the sequence isn’t available to you and pretending otherwise helps nobody. What’s available is deliberately doing some things the hard way, on purpose because the difficulty is the point.
That’s a strange instruction in a world optimized for the opposite, and it’s the only one I have that survives examination.
The book on this: The Death of Thinking is 29 chapters on what happens to judgment when the thinking gets handed over, and it’s a companion volume arguing the other side of the same question.
How should you use AI without losing your own reasoning?
Do the reasoning before you ask. Reach a view, then use the tool, then compare. The comparison is where the learning is.
Notice when you cannot tell whether the output is right. That’s the signal, and it’s much more useful than any general worry about AI.
Keep something you do entirely unaided. One domain where you never reach for help, purely to keep the capacity alive.
Be suspicious of how good it feels. Fluent output produces the sensation of understanding, and the sensation arrives whether or not the understanding does.
Stop treating this as a technology question. It’s a question about which difficulties are worth keeping, and it predates every tool involved.
I wrote a companion volume arguing the other side, because the case for these tools is real and I use them daily. Both books are true at once, which is uncomfortable and, I think, correct.