Top Law Firm Admits to AI ‘Hallucinations’ in Bankruptcy Filing Tied to Alleged Scam Network

Law firm Sullivan & Cromwell has admitted to a U.S. bankruptcy court that a recent filing in a high-profile case contained errors generated by artificial intelligence, including fabricated citations.

“We deeply regret that this has occurred,” Andrew Dietderich, the firm’s restructuring head, wrote to Judge Martin Glenn, saying the document included AI “hallucinations” that produced fictitious authorities and distorted existing ones.

The disclosure came in a letter to the U.S. Bankruptcy Court for the Southern District of New York, where the firm represents court-appointed liquidators from the British Virgin Islands. The mistakes appeared in an April 9 motion and the firm said its rules on AI use were not followed during preparation.

The case involves efforts by those liquidators to pursue claims tied to Prince Group and its owner, Chen Zhi. Prosecutors allege Chen directed scam compounds that targeted victims worldwide and have sought to recover billions of dollars in cryptocurrency they say is linked to the activity. He was detained earlier this year in Cambodia and later repatriated to China.

Through Chapter 15 proceedings in the U.S., the liquidators are seeking recognition of their authority to act on behalf of creditors and alleged victims. Prince Group, incorporated in the British Virgin Islands, has been linked by U.S. authorities to large-scale fraud operations in Southeast Asia and sanctioned by the UK and U.S. governments.

According to a corrected submission, the April filing misstated case law in multiple places and included citations that did not support the propositions attributed to them, while some appeared to have no basis at all. The firm withdrew the original motion and has filed a revised version.

Lawyers for Prince Group and Chen at Boies Schiller Flexner initially identified the errors. They said language attributed to the U.S. Bankruptcy Code could not be found and that several authorities were mischaracterized or misidentified. In one instance, they said, a cited case referred to a different decision in another circuit.

In a separate filing, defendants said at least 28 citations were erroneous, including quotations attributed to the court that do not exist. They argued the timing of the correction was prejudicial because the revised filing came after they had submitted objections, and asked the court to adjourn a scheduled hearing and hold a status conference.

Sullivan & Cromwell said its policies require lawyers to complete training before using AI tools and to independently verify all output.

“Before any Firm lawyer is granted access to generative AI tools, the lawyer must complete two required training modules, completion of which is tracked and verified. The training repeatedly emphasizes the risk of AI ‘hallucinations,’ including the fabrication of case citations, misinterpretation of authorities, and inaccurate quotations,” it said.

“It instructs lawyers to ‘trust nothing and verify everything’ and makes clear that failure to independently verify AI-generated output constitutes a violation of Firm policy.”

The firm said a broader review found additional minor drafting issues in other filings, which it attributed to human error rather than AI. It did not identify the lawyers who prepared the original motion.

AI in the dock

The incident adds to a growing list of AI-related missteps in legal practice as firms test tools designed to speed research and drafting. Courts have recently sanctioned or criticized lawyers for submitting filings with fabricated or inaccurate references produced by AI. In Australia, one lawyer was stripped of their ability to practise as a principal lawyer due to AI use last year.

Law schools are beginning to require instruction on the technology, while senior judges have warned that misuse could affect the integrity of proceedings.

Recent rulings have also addressed how AI fits within existing legal frameworks, including whether interactions with such tools are protected by privilege. At the same time, some courts are piloting AI systems to help manage heavy caseloads.