- within Accounting and Audit, Insurance, Government and Public Sector topic(s)
- with Senior Company Executives and HR
Artificial intelligence ("AI") is rapidly reshaping legal practice. Lawyers are increasingly using AI tools for drafting, research assistance, document review, summarisation and litigation preparation. As adoption grows, South African courts appear to be consistently confronting the issue where legal counsel is relying on AI without any human oversight and are routinely confronting the risks associated with the technology, particularly where AI-generated content is used without proper verification or human oversight.
With the advent of AI, courts have increasingly found themselves chastising practitioners for the negligent use of AI tools in legal proceedings. In Roux v Van Greunen and Others, the court was approached multiple times on an urgent basis by a practitioner who admitted to relying on AI as an aid while preparing his legal submissions. Notably, his submissions were littered with fabricated authorities, incorrect citations and legally incoherent arguments generated by AI.
The court did not take issue with the use of AI itself. In fact, the judgment expressly acknowledged that AI can be a useful tool when used responsibly. The primary issue was Mr Roux’s reliance on material that the court found to be legally unsound and, in some instances, entirely fictitious.
Mr Roux’s submissions contained irrelevant material, ineffective legal analysis and arguments based on principles not established in South African law. In support of his case, Mr Roux relied on Matshoba and Another v Acting Master of the High Court, Johannesburg and Others as authority for bringing the application. In this matter, the citation of the case relied on, belonged to an entirely different case, and the alleged reported judgment did not exist. It quickly became clear to the court that this was a case of AI hallucinations. Mr Roux attempted to explain the discrepancy by suggesting a possible clerical error in the court’s electronic registry. However, this explanation was rightfully rejected by the court. In addition, the court emphasised the need for practitioners to exercise caution and to verify any legal advice and sources especially where AI is used.
The risks of AI hallucination
An AI hallucination occurs where an artificial intelligence system generates false or inaccurate information presented in a convincing manner. In legal practice, this may include:
- fabricated cases;
- incorrect citations;
- invented quotations;
- mischaracterised precedent; or
- legally incoherent principles.
AI is a tool, not a substitute for legal reasoning
A common mistake in discussions about AI is treating the technology as though it can replace professional legal reasoning. Large language Models are predictive systems, they generate responses based on statistical patterns in data, not independent legal analysis or factual verification. While they are capable of producing sophisticated and persuasive outputs, they can also produce confident inaccuracies. This is precisely why human oversight remains critical. The recent judgments by South African courts reinforce the responsibility that legal practitioners have for legal submissions, regardless of whether AI was used in preparing material as such AI cannot be blamed for a practitioner’s failure to verify authorities or validate submissions. In many respects, this is simply an extension of existing professional obligations. Practitioners have always been expected to confirm the accuracy of authorities, ensure procedural compliance and exercise independent legal judgment. AI does not remove those duties. If anything, it increases the need for them.
Recently, a United States Massachusetts court refused to admit a Morgan & Morgan attorney to a Harvard-related lawsuit after the attorney had previously been sanctioned for filing motions containing AI-generated, fabricated case citations. In the earlier matter, eight of the cited authorities did not exist and were produced by the firm’s internal AI system. The court found that the prior incident raised serious concerns about the practitioner's compliance with basic professional obligations, particularly the duty to verify authorities before signing and filing pleadings. This case should serve as clear indication that AI misuse can affect a legal practitioner's reputation overtime.
AI legal privilege
Practitioners who use AI in their legal work remain bound by the rules of legal professional privilege. Using AI without a client’s knowledge may, in certain circumstances, constitute a breach of that privilege. Where AI is used as an aid in preparing or reviewing legal documents, it must be employed responsibly, as its use may compromise client confidentiality and potentially result in a waiver of privilege. In United States v Heppner, No.), a court in New York held that a person who uses AI to generate legal documents may not have a reasonable expectation of confidentiality, particularly where the AI platform’s terms of use expressly provide that user inputs and outputs may be used to train the underlying AI model.
Why the legal industry Is still moving toward AI
Despite the risks highlighted by cases of AI hallucinations, the legal profession is unlikely to move away from AI adoption. The commercial and operational benefits are simply too significant. Used properly, AI can substantially improve efficiency across legal practice. This shift is also driving the growth of specialised legal AI providers such as Harvey, Legora and Wordsmith, which unlike general purpose have source-linked outputs, audit trails and controlled legal databases. The goal is not to replace lawyers, but to augment legal work while maintaining professional oversight and accountability.
A balanced approach moving forward
The case of Roux v Van Greunen should not create fear around the use of AI in legal practice. Instead, it should encourage more responsible and informed adoption. This mirrors previous technological transitions within the legal sector such as e-discovery, cloud storage and digital contracting. All of which were approached cautiously before becoming standard features of legal practice. AI is following a similar trajectory, but at a far faster pace.
Legal practitioners and legal teams that understand the strengths and limitations of AI are likely to gain significant advantages in efficiency and service delivery. Those who rely on AI uncritically, however, risk serious professional and procedural consequences.
Ultimately, the lesson from the courts is straightforward: AI can be a powerful legal tool, but professional judgment, verification and accountability remain non-negotiable.
The content of this article is intended to provide a general guide to the subject matter. Specialist advice should be sought about your specific circumstances.
[View Source]
