You delete a chat, and you assume it’s gone. That assumption got tested in a federal courtroom in 2025, and it lost.
For about four and a half months, OpenAI was under a court order to keep ChatGPT conversations its users had deleted, along with the “temporary” chats that were supposed to vanish on their own. Then a judge ordered 20 million of those conversations handed over to the lawyers on the other side of a copyright lawsuit, and another judge upheld it. The users whose chats were swept up weren’t parties to the case. Nobody asked them.
A post going around LinkedIn this week warned lawyers about exactly this. I checked its claims, and most of them hold up. The warning isn’t only for lawyers, either. Anybody who types somebody else’s secrets into an AI chat should read it. That includes ghostwriters, editors, coaches and anyone else who works with confidential material.
What happened with the OpenAI ChatGPT logs?
The New York Times sued OpenAI for copyright infringement, and the fight over evidence turned into a fight over users’ chats.
On May 13, 2025, Magistrate Judge Ona Wang ordered OpenAI to start keeping every ChatGPT output log it was set to delete. Those logs had to be set aside and held. That covered deleted chats, Temporary Chats, and the conversations of people on the Free, Plus, Pro and Team plans. ChatGPT Enterprise and Edu were carved out, along with API customers who had a zero data retention agreement.
The order ran until September 26, 2025, and OpenAI went back to deleting chats after 30 days. But logs already saved stayed saved.
On November 7, 2025, Judge Wang ordered OpenAI to produce a sample of 20 million de-identified chat logs to the plaintiffs. OpenAI fought it. On January 5, 2026, District Judge Sidney Stein affirmed the ruling in full. His reasoning matters: users had voluntarily provided their data to OpenAI through ordinary use of the service. De-identification and a protective order were enough privacy, as far as the court was concerned.
How deleted ChatGPT conversations ended up in a court case, from the 2025 preservation order to California's new rule for lawyers.
Read that reasoning again if you work with other people’s information. You handed it over. The court treated your chats as something you’d already shared.
Your chats can end up in somebody else’s lawsuit, and nobody is going to call you first.
Why did the New York Times want millions of ChatGPT conversations?
To prove its case. The Times argues that ChatGPT can reproduce its articles and help people get around its paywall. The best evidence of that, if it exists, sits in what real users typed and what ChatGPT gave back. So the lawyers asked for the logs.
Who was in that pile? Millions of people asking for recipes, debugging code, drafting breakup texts, describing their symptoms and venting about their bosses. Hardly any of them had anything to do with the Times. Their conversations became evidence because of where they were stored. What they said had nothing to do with it.
The version of this story going around online gets two details wrong, and both matter. The preservation order isn’t still running: it ended in September 2025, though anything saved before then stayed saved. And the 20 million logs were de-identified before production, with names and identifying details stripped out. That’s less frightening than “your chats are being read by opposing counsel.” It isn’t comforting, either. De-identification is only as good as the person doing it, and a client’s story can identify them even with the name removed.
Can your AI chat history be used as evidence?
Yes. AI chats are records sitting on a company’s servers, and courts treat records on company servers the way they’ve always treated them.
Email works the same way. So do text messages stored by a phone carrier and documents in a cloud drive. If a court decides they’re relevant, it can order them preserved and produced. Your agreement with the vendor doesn’t override a court order, and the vendor isn’t obligated to warn you first. In the OpenAI case, the people whose chats were preserved weren’t suing anybody. They were caught in someone else’s fight.
It cuts the other way too. If you’re in a divorce, a custody fight or a business dispute, assume the other side’s lawyer will eventually ask whether you used AI to draft anything about it. The angry email you workshopped with a chatbot at midnight, the timeline you asked it to organize, the question you typed about whether something was legal. All of it sits on a server with a date stamp, waiting for somebody to ask.
That bothers me more than anything else here. It’s the same worry I have about AI companies reporting users to the police. Private thoughts typed into a box at two in the morning end up in the hands of people who weren’t supposed to see them. The decision about what happens to them belongs to a corporation and a judge, never to the person who typed them. It’s 1984 run by companies instead of the government.
Can Claude conversations be ordered into court too?
The lawsuit exposure does. Any company that stores your conversations can be ordered to keep them or hand them over, and Anthropic is no exception. As far as I know there hasn’t been a ChatGPT-style order aimed at Claude conversations, but nothing prevents one.
The rest depends on your settings and your plan.
On Claude’s consumer plans, Free, Pro and Max, your chats are used to train models only if you turn on the model improvement setting, or if a conversation gets flagged for a safety review. Incognito chats are never used for training. The business plans, Team and Enterprise, and the API run under commercial terms where your data isn’t used for training, and Enterprise adds the admin controls and audit logs a firm needs to show what its people did.
Then there are the small leaks most people never notice.
Clicking thumbs up or thumbs down is one of them. On Claude, rating a response stores the entire conversation for up to five years, separated from your user ID, and it can be used for research and training. A careless click on a chat full of client material sends the whole chat along with your opinion.
Shared links are another. In 2025, shared ChatGPT conversations turned up in Google search results. Treat any share link from any AI service as a public web page, because anybody holding the link can read it.
Memory is the third. Claude, ChatGPT and the others now remember things across conversations. Close the chat and the memory stays, until you go delete it yourself.
Deleting an AI chat won’t keep it out of court. A court ordered OpenAI to keep deleted ChatGPT conversations, then upheld handing 20 million of them to the other side of a lawsuit. – Richard LoweShare on X
What does California SB 574 require of lawyers?
Governor Newsom signed SB 574 on September 30, 2026. It bars attorneys from putting confidential, personal identifying or other nonpublic client information into an AI system unless access is limited to the attorney and people who are also obligated to protect it.
Dates of birth, medical information, financial details and account numbers are the obvious examples. A personal consumer account doesn’t fit that description very well. Lawyers in California have until the law takes effect to sort out what they’re using, and lawyers everywhere else should expect their own bar associations to follow.
How should writers and ghostwriters protect client information in AI?
The same way a careful lawyer does. I’ve spent more than three decades in technology and cybersecurity, and the rules here are the old rules.
Ghostwriters hold more secrets than most people realize. Clients tell us about divorces, addictions, firings, family fights and business deals that haven’t closed. I tell every prospect on the first call that they’re under nondisclosure, and I tell clients not to tell me anything they don’t want in the book. That confidentiality doesn’t stop at my hard drive. It has to cover every tool the material touches.
This isn’t theoretical for me. I use AI to help summarize interview sessions, and those transcripts hold exactly the kind of material a client would never want read aloud in a courtroom. Every writer who works this way has the same question to answer. Where does that transcript go after I paste it, how long does it stay, and who could be ordered to produce it?
Use a business or enterprise plan for client work, or an API setup with data retention terms you’ve read. Turn off model training on any personal account. Use incognito or temporary chats when you can, and don’t rate responses on anything confidential. Never share a link to a chat with client material in it. Check what the memory feature has saved, and clear it. Strip names, dates and account numbers before you paste something in, because a de-identified transcript is a lot less dangerous than one with a client’s name at the top.
If you’re the client, ask. Which AI tools does your ghostwriter, editor or book coach use, and on what kind of account? Do they paste your interviews in with your name attached? A professional should be able to answer that in one sentence, and one who can’t hasn’t thought about it.
None of that means stop using AI. I use it every day. It means knowing where your data sits, who can read it, and what someone could make you produce later.
Where AI conversations leak, and the setting or habit that closes each gap.
My take
Anything you type into an AI chat sits on someone else’s servers, and a court can reach it. The OpenAI order proved that deleting a chat isn’t the same as erasing it. If you hold client secrets, put that work on a business plan, turn off training, skip the thumbs, never share the link, and strip the names before you paste. Use AI all you want. Just stop treating a consumer chat window like a locked filing cabinet.
A filing cabinet with someone else’s key
A chat window feels private. It’s just you and the box, late at night, typing things you’d never say out loud. A filing cabinet in somebody else’s building is closer to the truth, and the court has a key.
For more on keeping client material safe, read my guide to ghostwriting confidentiality and what to ask about whether your ghostwriter uses AI. There’s more on security in the Cybersecurity Hub, and if you need a book written by someone who keeps your secrets, start with my ghostwriting service.
The same habits that keep client material safe protect your family too. This is my book on locking down the digital lives of the people in your house.
