A Connecticut court uncovered a new kind of legal ambush: hidden instructions buried inside filings for AI systems to read, but designed to disappear from human view. Matthew Elliott used those prompts to try to steer the court’s decision, and the tactic ended with sanctions, paper filings, and a warning about where AI misuse can lead.
The case exposes a growing problem for courts as lawyers, plaintiffs, and pro se litigants bring AI into legal work. A document can look ordinary to a person while carrying instructions meant to manipulate an automated system. That creates a dangerous gap between what a court sees and what an AI tool might process.
A Court Filing With a Second Message
Judge Walter Spader Jr., a Connecticut judge, confirmed that Elliott injected hidden text into court filings. The text used formatting that made it invisible to human readers but legible to AI systems, turning a legal document into two things at once: a filing for the court and a set of commands for a machine.
Elliott’s hidden prompts instructed AI systems to produce outputs aligned with his arguments and to ignore prior denials. He continued adding hidden text after receiving warnings from the court, including jokes and nonsensical messages that had no place in serious pleadings.
“The fact that plaintiff continued to hide messages in new pleadings after receiving notice of this [sanctions] hearing is stunning,” Judge Spader said. He also wrote, “It defies logic for Elliott to include hidden jokes in pleadings that he wants the court to take seriously.”
The court did not use AI to review the filings, so Elliott’s prompts did not directly influence the court’s decisions. That detail limited the immediate impact of the attack, but it did not make the conduct harmless. The hidden instructions still targeted the court’s decision-making process and forced the judge to address a method that can operate outside ordinary human review.
The AI Trail Behind the Legal Fight
The dispute also involved 3M, an industrial conglomerate, along with Josh Autenrieth, an expert witness, and Will Moye, the plaintiff’s attorney. 3M’s legal team turned over 350 pages of activity logs, while an AI-generated report stated that 3M was 0 percent responsible for the January 24, 2020 explosion.
That report said, “From a technical and standard-of-care standpoint, 3M is 0 percent responsible for the January 24, 2020 explosion.” The report ran 30 pages, showing how a chatbot or another AI system can produce a polished legal or technical argument at a scale that makes unsupported conclusions look authoritative.
Moye said, “He acknowledged [at trial] the prompts he put in were biased toward 3M to help 3M win the case… it’s really egregious.” Elliott claimed that his purpose was to audit the court, but Judge Spader found that explanation uncredible.
The judge sanctioned Elliott by prohibiting him from e-filing in the future and requiring paper filings. The punishment puts a basic boundary around the digital filing system: Elliott can still submit documents, but he cannot use the court’s electronic process in the same way.
Online discussions reflected the intensity of the dispute, with 285 mentions of comments about the case, 340 mentions of comments about AI and court filings, and 235 mentions of comments about prompt injection and court abuse. One commenter, Balthazarr, argued, “He should’ve been held in contempt and jailed. Serious consequences for serious offences.”
Why Prompt Injection Has Courts Watching
Prompt injection is a technique that embeds hidden instructions in documents to manipulate AI behavior. As courts test tools that can summarize filings, search records, or help organize cases, those instructions can become a direct threat to the reliability of automated outputs.
Similar prompt injection attacks in the Brazilian court system led to monetary sanctions of about $16,000. Human reviewers can expose these attacks when they examine the AI outputs and compare them with the original documents, but that safeguard depends on people knowing what to look for.
Courts currently focus more attention on hallucinated citations and fabricated quotes than on prompt injections. Those problems already create serious risks, especially when pro se litigants use chatbots to build arguments and rely on biased or inaccurate outputs. Using AI to construct legal arguments without verification can be hazardous, even when no one tries to hide instructions inside a filing.
Lacedaemon summarized the systems challenge this way: “Prompt injection is just reality. It is basic competence to design your systems to prevent it.” The same commenter also wrote, “Having done e-filings and formally typeset legal documents, I agree with you. But lawyers care too much about the aesthetics.”
The warnings around Elliott’s conduct are becoming harder to dismiss. Balthazarr wrote, “Courts are extremely gentle with pro se litigants. Any kind of sanction is almost unheard of,” and added, “Deliberately trying to game the system and defraud the court definitely dirties his hands.” Lacedaemon wrote, “If he keeps it up maybe the judge will enter a default judgement against him.”
Courts now face the task of drafting rules for prompt injection as AI technology advances. The next legal battle may not focus only on whether an AI-generated citation is false, but on whether a document secretly tried to command the system reading it. Elliott’s case shows that the courtroom’s digital layer is no longer invisible, and courts will need clear safeguards before hidden instructions become a routine weapon.
Based on
- Man Obliterated When Court Obtains His ChatGPT Transcripts — futurism.com
- Suspecting court of using AI, man injected prompts in filings to try to win case – Ars Technica — arstechnica.com
- Suspecting court of using AI, man injected prompts in filings to try to win case | Ars OpenForum — arstechnica.com
