# The science behind AI scored

Hirevue developed AI scored using a branch of computer science known as Natural Language Processing (NLP). NLP gives computers the ability to understand text and spoken words in a similar way to humans. AI scored uses NLP to replicate the way in which humans evaluate interview responses. Specifically, with AI scored, Hirevue is able to transcribe the content of what the candidate says in response to an on-demand video interview, and then evaluate it against the core competencies of the job. When evaluation is complete, AI scored can provide a non-biased recommendation to recruiters and hiring managers that they can use when making their final evaluation of the candidate.

## AI scored development

A team of trained I-O psychologists from Hirevue created AI scored by evaluating tens of thousands of candidate interview responses using a structured scoring guide called Behaviorally Anchored Rating Scales (BARS). Using BARS as a tool to evaluate candidate responses against a set of core competencies, teams of Hirevue I-O psychologists evaluated each of the interviews, first individually and then comparing notes to reach a consensus.

This rigorous method for evaluating each candidate response was used to train the AI models and develop scoring algorithms that could be applied to data the AI had not yet encountered. The algorithms were tested on the new data set to determine whether they could be used to successfully replicate human raters at an extremely precise level. When testing proved to be successful, AI scored was built into the Hirevue product and introduced as a way to provide recruiters with AI-powered benchmarks that could help them to make fair, efficient, and effective recommendations.

## Reliability of AI scored

Reliability refers to the degree to which you can rely on a model to replicate trained human raters. Leading experts in the field of AI recommend that machine-generated and human ratings be correlated at 0.60 as a standard to support use in guiding selection decisions. AI models used in AI scored average 0.77, far exceeding the standard recommended by experts. The AI models used in AI scored achieve reliabilities this high because they are trained on tens of thousands of expert ratings and 1.5 million interview responses.

Because the models used in AI scored were fine-tuned on interview question responses, this specificity allows it to detect and understand more nuances in interview responses. As a result, the scoring models used by AI scored can replicate subject matter expert ratings with much more precision than even the most powerful language models.

The science behind AI scored

For recruiters and hiring managers, use the feature guides to learn more about each area of functionality in Hirevue.

Screen Sharing in Your Interview

CodeVue FAQ

My browser is frozen 

Completing Your Game-Based Assessment (Computer)

Practice the Interview on your Computer

Update Browser (Safari)

Camera and Microphone Access Required Error

Update Browser (Firefox)

Camera Settings (Mac/Apple)

Camera Settings (Windows)

Microphone Settings (Mac/Apple)

Microphone Settings (Windows)

Audio & Speakers Settings (Mac/Apple)

Audio & Speakers Settings (Windows)

Camera in Use by Another Program Error

Update Browser (Google Chrome) 

COMPUTER INTERVIEW

User Roles and Permissions

Invitation Email

Interview questions

Job Welcome Video

Reminders

Self-Register

Add stages to a job

Create jobs

Create job templates

Jobs

Jobs page

Job templates page

Job summary

Stages

Edit or remove stages

Clone jobs

Clone job templates

Documents

Foyer

Email invitations

Send for review (forwarding)

Rating and review

Work with questions

Advancement

Groups

Resources

General

Select assessment

Assessments

AI scored

AI scored comparison

AI scored recommendations

AI scored question sets

Job competencies

How to use AI scored in a job

Add AI scored questions to a stage

AI scored question library

AI scored language support

AI scored ranked recommendations

Assessments & AI Scored

Builder

Builder page

Add Builder questions to a stage

Invite candidates to on-demand interviews

Add candidates to a stage

Candidates

Candidate details

Candidate activity

Candidates page

Import candidates to Hirevue

Add candidates to Hirevue

Archiving data

Data retention and archiving data

Data Retention & Archiving

Insights tiles

Hirevue dashboard

High priority tiles

Hirevue Dashboard

Live video virtual backgrounds

Live video experience

Live interview page

Live interview questions

Location

Live interviews

Live interview setup

Live Interviews

On-demand interview page

How to Reset an On-Demand Interview

On-demand voice interview questions

On-demand video interview questions

On-demand text interview questions

Candidate options

About on-demand interviews

OnDemand Interviews

Question Library

Question Library Import Guide

Question Library page

Question Library admin

Reviewing Interviews for Hiring Managers

Review all ratings page

Send an interview for review

Reviewer Options (On-Demand)

Review candidate interviews

Review page (on-demand interviews)

Review page (live interviews)

Review Interviews

Analytics

Analytics page

Legacy reporting

Analytics data export

Analytics export field definitions

Reporting & Analytics

Cancel or reschedule live interviews

Scheduling

Scheduling methods

Scheduling assistant

Scheduling Assistant Requests page

Schedule Interview Page

Manual scheduling

Manually schedule live interviews

Events

Events page

Create events

Configure event options Who tab

Configure event options What tab

Configure event options When tab

Configure event options Review tab

My availability

Candidate self-scheduling

Navigating and Managing Platform Features

Feature Guides

For recruiters and hiring managers, learn how to administer your Hirevue account by managing users, creating custom email templates, and setting up realistic job previews.

Application hierarchy

Add, edit, import, or deactivate user accounts

User management page

Languages

Language Hierarchy

Supported languages

Email templates

Email templates page

Edit email templates

Available Fields Reference

Email Templates

Realistic Job Previews

Realistic Job Preview Tips and Best Practices

Realistic Job Preview page

Realistic Job Preview template

Create a New Realistic Job Preview

Duplicate a Realistic Job Preview

Export and import a Realistic Job Preview

Realistic Job Preview publication status

Archive and Unarchive a Realistic Job Preview

Client Admin Guides

For recruiters and hiring managers, review Hirevue training videos to master platform fundamentals.

The science behind AI scored

Share

Share

The science behind AI scored

Share

Share