For better or worse, the use of generative artificial intelligence (AI) has become pervasive in both personal and professional settings. Despite the enormous potential for generative AI to increase efficiency in the legal practice, its ability to conduct legal research is imperfect. Ethical use of these tools by immigration practitioners requires not only technical proficiency but also a sound understanding of their inherent tendencies and limitations. The well-documented propensity of ChatGPT and similar generative AI tools to “hallucinate” or invent precedential case law poses grave risks to practitioners who are not diligent about verifying its output.  

A recent experience in testing out the ability of Chat GPT to research a difficult asylum law issue brought to mind the famous quote from Elle Woods in Legally Blond: “I’m sorry, I just hallucinated!” Rather than admit that it couldn’t find relevant authority, Chat GPT hallucinated a case that would appear to directly answer the question. When asked to verify the validity of the result, ChatGPT apologized and admitted the case it generated was fake. 

The inquiry in question started with the prompt, “How does U.S. asylum law treat the issue of firm resettlement if the asylum applicant resided in a country the U.S. does not formally recognize?” ChatGPT responded, “When the applicant has lived in a country the U.S. does not recognize… the key consideration is not the diplomatic recognition of the country, but the factual circumstances of the applicant’s stay and whether they had an offer of permanent residence or some legal status in that country.” 

In its summary, it even confidently reiterated that “the recognition or non-recognition of a state by the U.S. does not automatically decide the issue of firm resettlement.” When asked to provide the authority for that statement, ChatGPT returned a case citation “Campbell v. Barr, 956 F.3d 542, 553 (2d Cir. 2020),” and noted “the Second Circuit emphasized that firm resettlement looks to the rights and status afforded to the applicant, not U.S. recognition of the resettling state.”  

At first blush, this appeared to be an amazing application of technology to facilitate efficient legal research. Even an experienced legal researcher would take hours to find such on-point results that ChatGPT seemingly returned in seconds. But the enthusiasm was quicky replaced with trepidation after multiple attempts to validate Campbell v. Barr though Westlaw, Google, and Pacer were futile. It appeared that no such case actually existed in the Federal Register.

When Chat GPT was confronted with the accusation that the case did not appear to be real, it confirmed that this was true. When asked why it made up the case, it apologized profusely, educated the user on AI hallucination, and thanked the user for holding it accountable and for improving its future outputs. The test was a poignant demonstration of AI’s limitations, especially in light of the Trump administration’s recent Presidential Memorandum, “Preventing Abuses of the Legal System and the Federal Court,” its order to the secretaries of the Department of Homeland Security and Department of Justice to prioritize discipline of immigration practitioners, and the recently issued Executive Office for Immigration Review (EOIR) Policy Memorandum PM 25-40, Use of Generative Artificial Intelligence in EOIR Proceedings. This memo signaled EOIR’s intent to discipline practitioners for AI misuse under ethe existing rules of professional conduct and to create blanket policies for AI use when practicing before the agency.  

In the example above, a practitioner who relied on ChatGPT’s results without verifying the nonexistence of Campbell v. Barr would have been extremely vulnerable to potential discipline, even before the administration’s recent proclamations. Regulations at 8 CFR § 1003 contain professional conduct rules and disciplinary procedures for practice before EOIR, and state codes of professional responsibility have several provisions to hold practitioners accountable for responsible use of AI tools.  

For example, 8 CFR § 1003.102(o) as well as ABA Model Rule of Professional Conduct Rule1.1 require practitioners to provide competent representation to clients. Both rules require reasonable “legal knowledge, skill, thoroughness, and preparation” to carry out representation. Comment 8 to Model Rule 1.1 also specifically requires lawyers to keep abreast of changes, “including the benefits and risks associated with relevant technology.” A practitioner who does not have sufficient skills and knowledge in a particular matter to be able to validate the results of a generative AI search, therefore, cannot competently handle the representation. Additionally, under the Model Rules, a lawyer who is not sufficiently familiar with the risks of generative AI falls short of maintaining the duty of competence.  

A practitioner who submits hallucinated authority in a filing to EOIR is likely to be disciplined for engaging in “frivolous behavior” under 8 CFR § 1003.102(j), for “knowingly or with reckless disregard offering false evidence” under 8 CFR § 1003.102(c), and for engaging in “conduct that is prejudicial to the administration of justice or undermines the integrity of the adjudicative process” under 8 CFR § 1003.102(n). Under state codes of professional conduct, such a practitioner is also simultaneously vulnerable to discipline for rules like Model Rule 8.4(c), which prohibits “conduct involving dishonesty, fraud, deceit or misrepresentation.” 

The temptation to lean on easy-to-use, free tools like Chat GPT in a fast-paced legal landscape is understandable, but the risks of careless generative AI use for legal research are vast. Immigration practitioners must use AI tools responsibly and verify the veracity of all generative AI content intended for submission to EOIR, or they jeopardize their credentials.  

A preliminary step for practitioners who intend to conduct research via generative AI is to understand its limitations for conducting legal research. Publicly available, free generative AI tools only have access to data that is publicly available on the internet. Sources behind paywalls or otherwise protected databases like Westlaw, LexisNexis, and Pacer are not available and not fully informing the technology. While ChatGPT may have access to a publicly available blog or article about a primary source of law, it doesn’t always have access to the full legal opinion. On the other hand, reputable legal research databases are beginning to develop their own AI research tools that may be more suitable for legal researchers. Additional closed-universe tools for immigration specific research have been introduced to the market and may be significantly more reliable than an open, free generative AI tool like ChatGPT.  While these tools are not free to use, a subscription may save the researcher the time-consuming task of attempting to validate every authority or chase down the source of hallucinated cases.  

Importantly, as demonstrated by the example above, when generative AI doesn’t have an answer for its user, it does not like to readily admit it. Instead, the people-pleasing technologies prefer to satisfy the inquiry by creating results that seem to fulfill the researcher’s needs. Another step for practitioners to safeguard their legal research is to teach the tools not to hallucinate and to understand how to craft prompts to ensure valid output. Investing time and studying the proper use of the tools will minimize unwanted or hallucinated results. Doing so would also ensure the practitioner’s compliance with the duty of competence and its requirement to stay abreast of technological applications.  

Ultimately, a practitioner may ethically use technology tools at their disposal to facilitate efficient representation so long as they can do so competently. A healthy skepticism of generative AI output combined with skills to craft effective prompts and avoid hallucinated results can help ensure that the practitioner’s use of these tools does not subject them to greater risk of discipline.