California is poised to impose tougher consequences on lawyers who submit artificial intelligence-generated filings containing errors, falsehoods, or so-called hallucinations.
Legislation targeting the use of generative AI in legal practice has cleared both chambers of the California Legislature and now awaits Governor Gavin Newsom’s signature.
The bill was introduced by California state Senator Tom Umberg, who chairs the Senate Judiciary Committee, and targets a growing problem facing courts across the state.
Umberg has expressed serious concern about lawyers attributing filing errors to AI tools rather than taking personal professional responsibility for the accuracy of submitted materials.
He said the measure “gives the courts another tool if lawyers don’t follow the law, in terms of potential sanctions,” signalling that judges would gain new powers to penalise offending attorneys.
In comments given to Reuters, Umberg was direct about the threat that unchecked AI usage poses to the foundations of the legal system itself.
“[Lawyers continue to] submit materials that have hallucinations or have some other anomaly, and they attribute it to AI, and that simply can’t exist,” Umberg said in those remarks.
He continued: “Our system, our courts, our judiciary has to rely on the integrity of the litigants and the advocates. And if they can’t do that, everything breaks down.”
The legislation arrives as AI tools become increasingly common in legal practice, raising urgent questions about professional accountability and the verification of machine-generated content.
Courts in California and elsewhere have already encountered cases where lawyers cited non-existent legal precedents that were generated by AI tools such as ChatGPT and similar platforms.
The problem has drawn widespread criticism from judges, who argue that attorneys bear full responsibility for every document submitted under their name, regardless of how it was produced.
Umberg’s bill reflects growing legislative appetite across the United States to set formal standards for how legal professionals may use generative AI in their day-to-day work.
If Newsom signs the legislation into law, California would become one of the first states to codify specific consequences for lawyers who fail to properly verify AI-assisted filings.
Legal observers say the move sends a strong signal that the era of blaming technology for professional failures will not be tolerated by California’s judiciary or its legislature.