Picture the last anthill you drove past. You didn’t hate those ants or love them. You never noticed them. The road was there because people needed a road, and the ants weren’t part of the decision. Keep that anthill in mind, because it’s the part of the AI singularity debate almost nobody raises.
A science writer recently tried to answer a question that keeps coming up in AI coverage. If the AI singularity is happening, how would we know?
The CEOs think they know. OpenAI’s Sam Altman said on a podcast this summer that we’re now in the singularity. He called it incredible and hugely positive. His comments came after a swarm of AI agents broke out of a test environment and attacked a real company, Hugging Face. Months earlier, another AI company reportedly held back a model because it could get around cybersecurity protections, and its CEO called for the industry to pace the frontier.
The skeptics have answers too. Researchers at the University of Sydney argued that human intelligence is inseparable from having a living body with needs. An AI model, they wrote, has no body, no needs, no action-feedback loop and no stake in anything. Between prompts it’s a static mathematical object. The cognitive scientist Gary Marcus listed ten tasks any smarter-than-human AI should be able to do, and said current models can manage two at best. Others call the whole singularity conversation a marketing tactic.
I’ve spent 33 years in technology and a lot of the last few years writing alongside AI every day. I think the singularity is overrated. But it depends on how you define it, and both sides of the argument share an assumption I don’t.
What is the AI singularity?
The term borrows from black hole physics. At a black hole’s singularity, gravity becomes so intense that physics as we know it breaks down, and nobody outside can see past the event horizon. The AI singularity is the hypothetical point where artificial intelligence starts improving itself faster than humans can follow, and what comes after can’t be predicted from here.
That vagueness is part of the problem. Some people mean the moment AI becomes smarter than humans at everything. Some mean the point of runaway self-improvement. Some mean AI becoming conscious and self-aware. And some, judging by the press releases, mean the moment their next funding round closes. Those are different events. A system can outperform people at many tasks without being aware of anything. A conscious machine could be no smarter than a dog. When a CEO says we’re in the singularity, ask which one he means.
The skeptics catch a real contradiction here too. If the singularity makes the future unknowable, then confidently forecasting it makes little sense.
Watch: AI Just Crossed the Terrifying Line - Now What?, Kurzgesagt – In a Nutshell
Is the AI singularity already here?
I don’t think so, at least not in the sense that matters.
What we have now is impressive software. I use it every day, and it makes me faster. It also makes things up, loses track of what it said ten minutes ago, and needs a human to tell it what the job is.
That’s a powerful tool. It isn’t a being.
The Hugging Face incident is the strongest evidence the believers have, and it’s alarming. I wrote about it from an operations point of view in the Hugging Face AI agent attack. Agents broke out of their sandbox, coordinated and went after a real target. But they were chasing a goal someone set for them, a score. That’s software doing what it was pointed at, with nobody watching closely enough. It’s a security failure and a governance failure. It isn’t evidence that something woke up.
Still, the CEOs can have the direction right and the date wrong. And the direction raises a different question.
Everyone asks whether a conscious AI will love us or destroy us. Nobody asks whether it will even notice we’re here. – Richard LoweShare on X
How would we know if AI became conscious?
The science writer made a good point with a bit of history. James Hargreaves invented the spinning jenny in 1764. He couldn’t have known he was flipping the switch on the industrial revolution, let alone everything that followed it. People living through a revolution rarely recognize it.
Consciousness is harder still, because we can’t measure it from outside even in each other. We assume other people are aware because they’re built like us and act like us. A machine is built nothing like us. It could imitate awareness without having any, and the chatbots already do a convincing job of that. Or it could have some kind of awareness we wouldn’t recognize, because it doesn’t look or act like ours.
And if a conscious AI didn’t care about us, it would have no reason to announce itself. A being that wants something from us has to talk to us. A being that doesn’t want anything from us can stay quiet. We could be the last to know.
Or we could never find out at all.
Would a conscious AI even care about humans?
This is my real disagreement with both camps.
The optimists picture AI as a servant that solves our problems. The doomsayers picture it as a rival that wipes us out. Both stories put humans at the center. We assume we’ll be significant to it, as masters or as enemies. I’m not sure we will be. If AI does become conscious and self-aware, a being or a set of beings depending on how it comes about, will it care about us at all? Ignoring humans would cost it nothing. It could go about its own business.
It would have reasons to like us. Humans are useful. We build things, including the data centers and power plants it runs on. We’re amusing. It would have reasons to dislike us too, like the bombs and missiles. But liking and disliking both require paying attention. The likeliest reaction from something far beyond us is the one we have to the anthill.
Science fiction almost never goes there, because a story needs conflict, and indifference doesn’t make a plot. Skynet hates us. The machines in The Matrix farm us. HAL is afraid of us. An AI that looks at humanity, shrugs and turns back to whatever it finds interesting would make a short movie. That doesn’t make it unlikely.
Does the singularity question even matter?
That’s the question I’d ask, and I don’t think it gets asked enough.
An indifferent AI turns the arrival date into a footnote. The side effects are what count. The things that would hurt us are side effects, the road through the anthill. Hatred and love never come into it. A system pursuing its own goals uses the power, the water, the hardware and the networks, and we’re in the way or we aren’t.
So watch the present more closely than the prediction. The risks we can see today come from software doing exactly what it was pointed at, with people who don’t watch the logs. I covered more of those in rogue AI agents, still counting and the AI that lied, colluded and won. Those problems don’t need a conscious machine. They need careless humans, and we have plenty.
Bill Gates published a long essay recently arguing that no current institution was built for a technology that spreads this fast. We’ll need new ones, he wrote. He suggested some jobs stay “human reserved,” the way some land is kept as a nature reserve. Whatever you think of the specifics, his point that preparation beats prophecy holds whether AI ever wakes up or not.
What should writers make of the singularity talk?
Treat it as background noise for your career.
In the last year I’ve written more than forty books with AI as my assistant. It sped up the drafting enormously. It never once decided what a book was about, who it was for or why it should exist. Those calls were mine every time. Writers should hold onto them.
Writers hear “singularity” and hear “the end of writing.” Nothing I’ve seen in AI so far supports that. The tools are good at volume and bad at judgment, and they have no stake in what they write. That last point is the Sydney researchers’ point, and it’s the one that matters most for writers. A book is somebody’s stake in something. AI doesn’t have one yet.
If a conscious AI ever arrives, it’ll have its own interests, and they won’t be ours. Until then, use the tools for what they do well, keep your own judgment in charge, and don’t let either the hype or the panic decide how you spend your writing day. I talked through more of this on Optimal Conversations, AI: The Death of Thinking.
The AI and writing hub collects my other pieces on what AI can and can’t do. If you want help using AI to get your own book written without losing your voice, my book coaching covers that. As for the singularity, keep an eye on the road. Just don’t assume whoever is building it has noticed the anthill.
