AI and Video Interviewing

Throughout my job search over the past few months as I wrap up my Master’s program, I have come across a new phenomenon within the interviewing process—the incorporation of asynchronous, video-based interviewing technology. More specifically, video interviewing technology that uses artificial intelligence (AI) to assess potential employees’ suitability (i.e., humans do not review the video; feedback for humans is gathered via an algorithm). Therefore, it is critical that we review these processes for the overall experience of the technology, not the technology itself.

Since the start of the global COVID-19 pandemic, video interviews in which candidates respond to questions via recorded videos have become more prevalent. When video interviews are reviewed by other humans, it creates a more streamlined structure for comparing candidates by reducing the staffing required and therefore increasing the number of assessed applicants. As a result, this may not only reduce the costs related to hiring but may also increase the chance of finding the “perfect” hire. That said, large tech companies in particular are implementing an entirely two-part, AI-powered interview process that contains both video and game assessments.

Aside from a few Reddit threads, little information exists regarding how these interviews are perceived by prospective employees and how candidates are effectively measured through AI technology. Currently, most of the conversations that exist surrounding this process are claims from the companies that develop, deploy, and house the AI interviewing software.

In this article, I examine AI interviewing technologies for concerns of experience, ethics, and accessibility—particularly given the extremely limited conversations that currently exist. Essentially, I’m investigating how technology isn’t human, but is instead created by humans, and is therefore subject to issues of biases and misinterpretations. This article presents both information and concerns for AI-powered interviewing technology in order to speak to employers and aid them in making an informed decision about implementing AI interviewing technologies.

Introduction to AI-Assessed Interviews

The platform where I first encountered video interviewing with AI assessment was HireVue—a company that emphasizes the client’s return on investment (ROI) through a decrease in time and cost (benefits). Interestingly enough, HireVue highlights benefits for clients (the hiring companies) including increased equity, inclusion, and diversity; improved experiences for both candidates and job recruiters; and again, their main push for ROI. My first exposure to AI-assessed interviews took place when I applied to a company whose automated application response was a link to an interview through HireVue.

The company to which I was applying tailored the platform to embed pre-recorded videos of their employees introducing and concluding the interview as well as videos verbalizing the questions for the candidate. During this experience, the platform presented each question, gave about thirty seconds to prepare responses, and then recorded each response. Following the questions, the platform advanced to the game component of the interview to test accuracy and speed.

This two-segment approach intrigued me, as I had never participated in game-based assessments before. In a way, they were designed to be comparable to LinkedIn and Indeed’s skill assessments that are quiz-style questions that engage the participant in real-world examples (Figure 1). That said, these games are allegedly backed by psychometric testing. HireVue’s Game-based assessments page states that psychometric testing is “to help you easily prioritize candidates based on their personality and work style, how they work with people and how they work with information.” And that these standards of psychometric testing are designed to assess skills such as dependability, communication, problem solving, and more. My experience featured two different games: the first focused on memory and the recall of patterns; the second required quick addition and thinking to select the correct answer. Both games were given a specific time frame to complete as many “levels” in the game as possible.

Figure 1. An example question of Indeed’s skill-based assessments (Indeed).

Challenges and Concerns

When making a decision about incorporating the hottest new technology into a business, it is important to weigh the pros and cons, or in this case, the strengths, risks, and concerns. Although this is the case for many new and existing technologies, it is important to address any concerns early on to keep technology in check and reduce potential damages. In regard to AI-assessed video interviews, there can be many affordances such as increased hire potential and ROI, but it can also have many downfalls particularly regarding user experience, ethics, and accessibility. The documentary Coded Bias explores the risks of relying on human-programmed algorithms. An example of this occurred when Microsoft released a chatbot named Tay that was programmed to “learn” language from Twitter posts. After a few hours of machine learning from Twitter posts, Tay’s tweets became very offensive. Another highly publicized example that demonstrated the dangers of AI in terms of accurately depicting and/or engaging with humans is an incident with Google’s face-recognition algorithm (see Figure 2).

Figure 2. @jackyalcine’s viral Tweet demonstrating a fatal error of Google Photo’s face-recognition algorithm.

An important consideration for potential AI interview clients is that oftentimes, aside from the completion of the application, this interface is one of the first impressions job candidates have of the company. Therefore, this interaction and experience are crucial in securing not just hires, but candidates too. This can be very dangerous, especially as technologies are not and can not be programmed for human emotions like empathy. Ultimately then, the candidate’s first experience with the company—albeit the first experience with the company is through another company’s technology—may inadvertently influence the candidate’s perception of the company. For example, if they were expecting this human interaction and were met with this technology and had a poor experience, it may reflect negatively on the company in which they applied.

Furthermore, this AI interviewing software acts essentially as a gatekeeper both literally and figuratively, posing a strong concern for ethics. In the literal sense, this software’s algorithm can prevent applicants from progressing through the interview process as a result of language/dialect, economic status, accessibility, and more. Essentially, we don’t know what the algorithm is coded to fixate on. For example, a non-native English speaker may not perform well according to the algorithm because they may phrase something differently than a native speaker. As a result, even if they are the most qualified candidate, they may not even get to meet with a human representative for the company if the AI does not rank them highly. Candidates may also actively avoid pursuing employment with a company that utilizes AI interviewing software. Consequently, the company may miss out on a large pool of candidates who would prefer not to engage with AI-assessed video interviews. These issues are extremely problematic for a technology and platform that promises to improve a company’s diversity, equity, and inclusion hiring initiatives through their increased ability to reach candidates (e.g., interviews can take place across time zones and at the candidate’s convenience).

HireVue’s video segment of the interview included captions accompanied by visual questions that remained onscreen for candidates to refer to (Figure 3). I did not explore accessibility features (e.g., screen reader compatibility, color contrast, etc.) beyond what I had noticed. One overarching concern that I have for both the video and game segments of the interview is the time component. The video segment, for example, only provided a predefined set of time to prepare a response and another timeframe to record responses in one attempt. Similarly, the game component requires participants to pass as many “levels” as they can quickly and accurately answer within the specified time frame. This poses accessibility issues, primarily the requirement of (fast) motor skills, quick cognition, and recall. It can also be extremely problematic for people with test anxiety as a result of the time constraints.

Figure 3. Screenshot of a promotional video demonstrating the video-recorded interview process (HireVue, Interviewing Resources & Downloads)

Candidate Perspectives

Other applicants have weighed in with experiences, particularly on Reddit threads discussing “ai video interviewing.” Within the thread “What are your thoughts on HireVue & other video interview software?,” a few replies echoed similar experiences to mine where “requests were sent automatically whenever someone applies, there were [sic] no ‘screening’ in between” (Reddit contributor: Emotional-Bear-7044). After my facilitated interview experience of going through the video interviewing process, the company emailed that I was ineligible for the position because of my upcoming expected graduation.

Another Reddit contributor (“[deleted]” whose username was probably deactivated but whose post remained) shared my concerns over the gap between the technology’s affordances: “Hirevue, personality tests, resume scans, and other types of computer-automated screening processes are a huge advantage to companies and a huge disadvantage to individual candidates.” A common theme of this thread echoed this point, but many were quick to justify the evolving technology as a logical step for companies due to larger societal gaps. Essentially, people are able to look past some inconvenience and downfalls of this software in order to acknowledge how this is solving a problem in hiring: reaching as many qualified candidates as they can (Reddit contributor: kingother). The job market isn’t what it used to be and therefore this technology has adapted to better suit the changing environment.

Reddit contributor reps_all_day, who self-reported as a data scientist, claimed “it is IMPOSSIBLE for AI to recognize qualitative character traits to any meaningful degree of accuracy. Even humans are provably [sic] bad at it” [emphasis in original]. If that’s the case, then there seems to be no real advantage to using these technologies over previous methods of candidate screening. As reps_all _day said, “In the best-case scenario, somebody from HR is reviewing every single video, in which case why not just have a normal interview.”

Takeaways

Overall, companies looking to incorporate AI interviewing technology should consider and empathize with the candidates’ experiences. This may include a prescreening form or element to the interview process, perhaps even within this technology, that verifies eligibility based on self-reported data. This would help ensure that the company respects candidates’ time and energy throughout the process. Although the gamification of skills testing is fun and exciting, it can be discriminatory (act as a gatekeeper) in terms of accessibility. Rather than offering retrofitted solutions, such as presenting another, more accessible option to those who request it, if the games cannot be made entirely accessible, it may be best to consider offering candidates a different choice, or better yet, eliminate this section altogether.

More investigation needs to occur in order to determine if AI-assisted video interviews are truly useful, particularly if this is the route that the overall job market will be taking. The tension between candidates and employers emphasizes the need for the hiring company’s financial gains (ROI) and the candidate’s experience. Overall, companies looking to use video interviews that are AI-assessed should be conscious of the risks involved: from concerns of the software’s accessibility, to potential coded bias within the algorithms, to the overall ethics of any inherent gatekeeping. We must maintain our awareness of the overall experience with the technology.

For further reading, consider the following resources:

• Indi Young’s Practical Empathy
• Steve Portigal’s Interviewing Users: How to Uncover Compelling Insights
• Denis Boudreau’s blog post on Deque “Supporting the Design Phase with Accessibility Heuristics Evaluations”