Communicating Criterion-Related Validity Using Expectancy Charts: A New Approach

Imagine that you are a practitioner and have just conducted a criterion-related validity study on a new personnel selection test.Your organization's top leadership asks for a quick summary of the findings.How would you convey the predictive ability of the test on a single slide in a presentation without having to delve into statistical terminology?In the academic literature, criterion-related validity coefficients are often used as a measure of predictive ability of a test.However, practitioners may find it challenging to translate the meaning of a criterion-related validity coefficient to a nontechnically-savvy audience in a manner that would allow the audience to quickly make informed decisions.We have found expectancy charts to be useful for these purposes.These charts can convey the predictive ability of a test in a single slide with little explanation required.To help practitioners translate the results of their criterion-related validity studies, we set out to meet the following goals: (a) to describe how expectancy charts can assist personnel selection practitioners in translating complex, technical concepts such as criterion-related validity to nontechnically-savvy audiences and (b) present a new methodology for constructing expectancy charts.We also provide R syntax that practitioners can use to compute more accurate expectancy values.We begin our paper by reviewing existing approaches to creating expectancy charts and highlighting some potential issues with these approaches.We then present our new approach and explain how to implement it in a step-by-step format.We also discuss the inherent assumptions and limitations of our approach and cover some special situations in which it may be used (e.g., multiple-hurdle selection systems). Expectancy ChartsExpectancy charts are bar charts that illustrate the relationship between a range of predictor scores, such as personality, and a range of criterion scores, such as job performance.Figure 1 illustrates an expectancy chart for the relationship between ranges of test scores and the percentage of special agents who were rated as superior by their supervisors.For example, of those special agents with test scores of 90 or higher, 43.4% were rated as superior, compared to 3.1% for those with test scores of 69 or lower.