The use of artificial intelligence (AI) programs such as ChatGPT has significantly increased over recent years.
These programs are known to pull information from array of different databases (what is widely available on the internet) and generate predictive answers based on the limited information provided.
Hallucinations
We have noted that many clients prior to instructing a solicitor tend to seek advice from AI programmes, such as ChatGPT.
One of the primary concerns associated with the use of AI programs is that they can generate inaccurate or fabricated information. AI systems do not have a full understanding of the factual context of a legal matter and may produce responses that may appear legally sound but do not properly apply to the specific circumstances of your case. As AI programs respond to the information entered by the user, these programs often aim to provide a helpful or affirmative response rather than identifying potential weaknesses in a claim.
Further, legal principles, legislation, and case law are subject to frequent change. AI-generated responses may therefore rely on outdated or incomplete information and may not reflect the current state of the law. AI systems may also generate content that appears authoritative, including citations to legislation or case law that may not exist or may be inaccurately described. This phenomenon is commonly referred to as an “AI hallucination”.
This issue was highlighted in the Queensland case of LJY v Occupational Therapy Board of Australia, where the judge noted that the applicant relied on fabricated cases that had been generated by AI in support of their submissions. Had the applicant sought legal advice prior to filing their pleadings, this issue may have been identified and avoided.
Courts expect all parties, including self-represented litigants, to ensure that any authorities cited in their submissions are accurate and properly verified. Reliance on non-existent or inaccurate authorities may undermine the credibility of a party’s case and may have adverse consequences.
While AI tools may assist with general research, they are not a substitute for advice from a qualified legal practitioner who can properly assess the merits of a claim and provide advice tailored to the specific circumstances of your claim. It is therefore important that you seek comprehensive legal advice before pursuing a claim, particularly if you are considering acting as a self-represented litigant.
Our solicitors at Brand can assist by assessing the strength of your claim, advising you of any risks involved, and ensuring that you are properly informed in relation to all relevant matters before commencing proceedings.
Confidentiality & waiving privilege
AI programs give you the option to upload files and often require a high volume of data to assist in generating answers to your queries.
Uploading case specific documents and information to these tools, may inadvertently waive legal professional privilege. As such, information, which is confidential, case sensitive or privileged cannot safely be entered into these programs.
A recent judicial decision in Helmold v Mariya (No 2), has formed the view that ‘the input of documents arising out of the proceedings into a generative AI program which stores, collates and replicates data may waive privilege or fall foul of the requirements that certain matters be treated as commercial in confidence’.
It is important for all users to be aware of the apparent risks before uploading information when entering questions regarding their case into these databases as it will likely result in a loss of confidentiality and ultimately a loss of legal professional privilege.
There is an obligation for your legal representation to consider the risks associated with uploading information to these AI program and consider whether there are appropriate security measures in place.
Based on the current views and understandings, it is recommended not to upload case specific information or documents into these AI programs and to consult your lawyer for any case related questions.
Final Thoughts
Utilising AI programs for the purpose of claim advice and doubting the advice of your experienced lawyer, may result in developing unrealistic claim expectations and will likely lead to the incurrence of further legal fees.
It is important to use these tools responsibly and not take the information providedby them on face value. We recommend that you consult a lawyer for any legal advice pertaining to your claim and trust the advice provided.
At Brand Barristers & Solicitors, our legal team is dedicated to assisting you with navigating your claim through its complexities and case specific issues.
LJY v Occupational Therapy Board of Australia [2025] QCAT 96
Helmold v Mariya (No 2) [2025] FedCFamC1A 163
