By Chira Perla
December 2023

Since launching in November of 2022, ChatGPT has prompted legal employers, career services professionals, and job seekers to reimagine legal recruitment with a nod to artificial intelligence (AI).

A Spring 2023 National Association of Colleges & Employers (NACE) survey found that 40% of responding members used AI in their own work over the last year, and almost half of their students used AI to write cover letters (49%) and resumes (42%).

NALP's Fall 2023 Pulse Survey on the Use of AI in the Legal Workplace found that 11% of responding employers were using AI in their daily work, with assessments in the recruitment process being the most cited type of AI use. (See the Executive Summary of the survey results.)

What's unique about ChatGPT is its ubiquity and accessibility: unlike most AI products that support legal recruitment, generative AI chatbots like ChatGPT (together with third-party plug-ins) and Microsoft's Bing Chat are free or have a nominal cost to use.

All of this got me thinking: How can (or should) candidates and recruiters leverage free chatbots to their advantage? And what are some of the potential pitfalls or consequences?

Chatbot Tips for Candidates

Ways that a candidate (and the career services professionals who support them) may want to use chatbots in recruitment include:

• Source: Identify employers and key information about them that is important to the candidate (e.g., "List all the law firms in Vancouver with a tax planning practice. Then list those that also have a community giving statement on their website.")

• Research: Understand more about a particular employer (e.g., "What percentage of Firm X's lawyers summered at the firm?") or obtain an overview of unfamiliar practice areas of the law that may be of interest (e.g., "Provide a basic overview of securities law in Ontario" or "Tell me the five most important developments in energy law in the U.S. in 2023.")

• First Drafts: Generate a template or first draft of a cover letter or resume. This could range from identifying transferrable skills and how to describe them given the candidate's experience or employer's job description, to creating full templates for the candidate to use.

• Tailor: Customize generic application materials to a specific employer given stated constraints, such as the job description, keywords on an employer's website, or a practice focus.

• Prepare and Practice: Identify common and expected interview questions, suggest answers to those questions, and/or run a mock interview.

• Communicate: Generate rote communication, such as thank you notes.

Risks candidates should be aware of when using chatbots in recruitment include:

• Accuracy and Completeness: A chatbot's information is only as good (accurate and current) as the data set it uses. There are many examples of chatbots providing incomplete, inaccurate, or even fabricated information. Accordingly, candidates should not rely exclusively on chatbot-generated content when identifying and applying to employers.

• Race to the Middle: A recent study from the University of Minnesota found ChatGPT helped low-performing law students to score higher on exams, but negatively impacted high-performing law students. Extending this logic, top candidates may be worse off using chatbots to prepare their application materials than if they had crafted them themselves, while less competitive candidates may be able to better leverage these tools to their advantage.

• Privacy: Information entered into chatbots may become part of the chatbot's training dataset. This means that sensitive or confidential information used in prompts may be exposed to other users or third parties. Accordingly, candidates should be careful when entering personal information about themselves or others.

• Disclosure: Candidates should be prepared to answer the question of if and how they used AI tools to prepare their applications. If they would feel uncomfortable answering that question honestly (because it would show a lack of care or true interest, or call into question their skillset), they may wish to rethink if or how they use these tools to craft their applications.

Chatbot Tips for Recruiters

Subject to their employer's rules and policies on AI use, a recruiter may wish to use chatbots to:

• Pipeline Candidates: Create Boolean strings to search for talent (e.g., "Create a Boolean search string for associate lawyers in western Canada with M&A experience.")

• Create Recruitment Materials: Generate text for a job posting, improve an existing job posting for inclusivity, or even create a careers landing webpage (e.g., "Identify Vancouver law firms with more than fifty lawyers. Review their websites for a career or hiring page. Use keywords from those pages to create a careers page for a mid-sized full-service firm in Vancouver looking to hire junior associates.")

• Generate Interview Questions: Source new or different questions (e.g., "Create a list of interview questions for a 30-minute interview with a Canadian tax law firm hiring a 2L summer student. One-third to assess fit, one-third behavioural, and one-third technical or situational.")

• Communicate: Generate rote communication templates, such as interview details, application deadlines, and thank you notes.

While exact requirements will vary by jurisdiction, recruiters should be aware of privacy, human rights, and emerging AI law risks when using chatbots in hiring. Specific to Canada:

• Privacy: Privacy laws apply to employment relationships, so recruiters must ensure that their use of chatbots comply. Canadian privacy laws require candidate consent to collect, use, or disclose personal information. This means consent is required to input a candidate's application materials into a chatbot, or to conduct an informal background check by using an AI plugin to crawl their social media accounts. Although there are exceptions to consent requirements where reasonable or necessary to establish, manage, or terminate the employment relationship, what constitutes reasonable or necessary is not readily clear, making the use of these tools absent consent risky.

• Human Rights: Human rights laws in Canada prohibit discrimination in employment based on enumerated protected grounds, including race, ethnic origin, gender identity, and age. The datasets used to train chatbots are not readily discernable, making it difficult (if not impossible) to evaluate the underlying training data for bias. Using chatbots to directly screen applications for specific keywords or experience (e.g., "Review and score out of 10 the following cover letter for demonstrated experience in human rights law and leadership skills") runs the risk of filtering folks in a discriminatory way.

• AI Laws: AI regulation and legislation is rapidly changing. In Canada, there is currently no regulatory framework specific to AI. However, there are some regulations in specific areas that apply to certain uses of AI. Moreover, the Artificial Intelligence and Data Act (AIDA) is proposed legislation that aims to set the foundation for the responsible design, development, and deployment of AI systems. Any use of chatbots in legal recruitment requires close attention to evolving law governing the use of AI.

Takeaway Thoughts

At the time of this article (Fall 2023), we are in the midst of the first summer recruit aided by generative AI chatbots in a rapidly changing regulatory landscape, which makes the full extent of their use and impact a live question.

To run each of the above examples, I used a combination of ChatGPT, plug-ins, and Bing Chat. In some cases, the generative AI tools genuinely surprised me with unique solutions or approaches, or by researching, compiling, and applying large data sets incredibly fast and in a way that mirrored or improved upon my own thought processes.

The chatbots were also great at creating first skeleton drafts. But frequently the chatbots were unable to generate a response, or the response provided was clearly incomplete (even after prompts to iteratively improve given specific information). Moreover, chatbot generated application materials were not comparable to those I would expect to see from top candidates. Regardless, if this is the first iteration of generative AI for the masses, I am excited to see what comes next, particularly as tools are developed specific to law and legal recruitment.

Bigger picture, generative AI is poised to have a profound impact on all facets of the legal profession as organizations look to incorporate generative AI into their products and work (e.g., LexisNexis' Lexis+ AI, Casetext's CoCounsel, and Dentons' fleetAI). As a result, in addition to all the ways in which candidates can use generative AI to apply for jobs, and recruiters to hire for them, the next logical question becomes how can candidates demonstrate, and legal employers test for, the efficient and skillful use of generative AI in legal work?

And yes, I posed that question to Bing Chat. Try it yourself — to be honest, the response was pretty good.

Chira Perla (cperla@ropergreyell.com) is the Director of Professional Development at Roper Greyell LLP in Vancouver, British Columbia.