A Manhattan lawyer named Steven Schwartz asked ChatGPT to find case law for a routine personal-injury appeal. It handed him six perfect-looking citations. He filed them. Every one was invented. On June 22, 2023, Judge P. Kevin Castel sanctioned Schwartz, his co-counsel, and their firm $5,000 and ordered them to write apology letters to the real judges whose names ChatGPT had stuck on fake opinions.
That was the warning shot. Two years later the tool is better, cheaper, and sitting open on half the desks in your office. Used one way it saves you an hour a day. Used another way it puts your name in a published opinion for the wrong reasons. Here is the line between the two, drawn for a working PI firm.
Where ChatGPT earns its keep
None of what follows involves the model telling you what the law is. That distinction is the whole ballgame, and the safe uses all sit on the same side of it.
First drafts of routine correspondence. Feed it the facts and ask for a draft demand letter, a client update, or a status email to opposing counsel. You get a serviceable skeleton in seconds instead of staring at a blank page. Caution: it is a first draft, not a filing. You edit every line, you check every number against the file, and you never let it invent a fact you did not give it.
Summarizing documents you paste in. Drop in a deposition transcript, a medical narrative, or a long insurer email and ask for the key points, the timeline, or the open questions. This is genuinely useful because you are handing it the source and asking it to compress, not to remember. Caution: read the summary against the original before you act on it. It will occasionally smooth over a detail that turns out to be the case.
Brainstorming and pressure-testing. Ask it for ten angles on a settlement negotiation, the counterarguments a defense adjuster will raise, or the questions you are forgetting to ask a new client. It is a fast, tireless sounding board. Caution: treat every idea as a prompt for your own judgment, not an answer. It has no stake in your client and no sense of your jurisdiction.
Marketing copy. Blog outlines, website section drafts, social captions, a rewrite of a page that reads like a brochure from 2009. Marketing is where ChatGPT is safest, because there is no court and no client secret on the line. Caution: bar advertising rules still apply. No guarantees of outcome, no "best," no misleading claims. The AI does not know your state's rules of professional conduct and will happily write a line that violates them.
Translating client communications. A lot of PI work is bilingual, and a fast, plain-language translation of an intake summary or a client text can keep a Spanish-speaking claimant in the loop between calls. Caution: this is a convenience, not a certified translation. Anything that goes in a filing or a signed document needs a human translator.
Checklists and process drafts. Ask it to build an intake checklist, a case-onboarding sequence, or an SOP for how your paralegal handles a new MVA file. It is good at structure. Caution: it will produce a generic list. The value comes from you cutting what does not fit and adding what your firm actually does.
The pattern across all six: ChatGPT is a drafting and thinking tool that works on material you provide, and a human reviews everything before it leaves the building. Keep it in that lane and it is one of the cheapest productivity gains a small firm can buy. Step outside that lane and you meet the part that gets people fined.
Where it gets lawyers sanctioned
There is one use that keeps landing lawyers in front of angry judges, and it is the one that feels most natural: asking ChatGPT for legal research or citations.
The model does not look anything up. It predicts text that sounds like a real citation, which means it will produce cases, quotes, and reporter numbers that do not exist. The industry word is hallucination. In court it reads as a fabricated citation, and judges have stopped being amused.
Schwartz and Mata v. Avianca opened the era. It has only gotten worse. In September 2025 a California appeals court fined Los Angeles attorney Amir Mostafavi $10,000, the largest such penalty in the state, after finding that 21 of the 23 case quotes in his opening brief were fabricated. He told the court he had run his own draft through ChatGPT to improve it and did not realize it would insert fake citations, and did not read the output before filing. The Second District Court of Appeal called it a frivolous filing and put a line in the published opinion that every PI owner should tape to the wall: no court filing should contain a citation the attorney has not personally verified.
These are not two freak stories. By late 2025 legal researchers were tracking more than 600 filings nationwide with AI-fabricated citations, and the pace had climbed from a couple a week to a couple a day. The sanctions run from public naming to five-figure fines to referrals to the bar.
The rule is simple and it has no exceptions. ChatGPT is not a legal research tool. Never cite anything it gives you without pulling the actual case in Westlaw, Lexis, or a free reporter and reading it yourself. If it names a case, assume the case is fake until you have the real opinion in front of you.
The confidentiality trap nobody warns you about
The sanctions get the headlines. This one gets less attention and is arguably a bigger exposure for a PI firm.
When you paste text into the free, consumer version of ChatGPT, you are handing it to a third party, and by default that content can be used to train the model. Your intake notes, a client's medical history, a settlement figure, the other side's confidential offer: none of that belongs in a consumer chat box. That is client confidential information, and shipping it to OpenAI's servers is a problem under your duty of confidentiality before any judge ever sees it.
Two practical moves. Strip identifying details before you paste anything, so the model gets the facts without the name, the account number, or the medical record number. And if you want to use AI on real client material, move to a business or enterprise tier with a data-processing agreement and training turned off, or a tool built for legal work with those protections baked in. The convenience of the free version is not worth a bar complaint.
For a wider look at how AI is changing PI intake, marketing, and case handling beyond one chat tool, see our guide to AI for personal injury law firms.
The one habit that keeps you safe
Use ChatGPT for drafts, summaries, ideas, marketing, and translation. Keep client secrets out of the consumer version. And treat anything that looks like law, a case, a citation, a legal conclusion, as unverified until you have checked the source yourself.
That is the whole discipline. The lawyers who got fined did not break some obscure rule. They trusted a confident machine on the one job it cannot do, and skipped the read. Do not skip the read.
Frequently asked questions
Can lawyers use ChatGPT? Yes, and many do. It is well suited to first-draft correspondence, summarizing documents you paste in, brainstorming, marketing copy, and quick translations. The trouble starts when a lawyer uses it for legal research or citations, or pastes confidential client information into the consumer version. Keep it to drafting and thinking tasks with a human reviewing the output and you stay on safe ground.
Have lawyers really been sanctioned for using ChatGPT? Yes. In Mata v. Avianca (2023) a New York judge fined two lawyers and their firm $5,000 for filing six fabricated citations. In September 2025 a California appeals court fined attorney Amir Mostafavi $10,000 after finding 21 of 23 quotes in his brief were made up. Researchers have logged more than 600 filings nationwide with AI-invented citations.
Is it safe to put client information into ChatGPT? Not in the free consumer version. Content pasted there can be used to train the model, which conflicts with your duty of confidentiality. Strip identifying details first, or use a business or enterprise tier with a data-processing agreement and training disabled before you touch real client material.
We break down how AI is actually changing personal-injury practice, what saves you time and what gets you in trouble, in plain English for firm owners. Subscribe and get the next one in your inbox.