Using Latent Profile Analysis to Describe and Understand Medical Student Paths to Communication Skills Expertise

Purpose: Despite the importance of communication skills, the skills of graduating medical students vary widely. Identification of patterns of communication skills development among medical students could guide coaching of all students toward mastery. For the past 13 years at the NYU Grossman School of Medicine, students have been required to pass a Comprehensive Clinical Skills Exam (CCSE) administered at the end of required clinical rotations. Consistent case content and a behaviorally anchored checklist allows for examination of patterns of communication skills exhibited by successive cohorts of students. In this study, we use latent profile analysis (LPA) to identify and describe distinct profiles of communication styles among clinically experienced medical students. Method: Data from 1,135 third-year medical students at NYU Grossman SOM who completed the 8 case CCSE from 2011 to 2019 and who consented to use of their educational data (through our Medical education registry) was analyzed. 1 Communication is measured by a 17-item behaviorally anchored Clinical Communication Skills Assessment Tool rated by SPs as not done, partially done, or done well. 2 This includes 4 domains: information gathering (6 items) relationship development (4 items), patient education (3 items), and organization/time management (3 items). An LPA was conducted to identify learners exhibiting unique patterns (profiles) of communication performance on the CCSE across all cases. LPA clusters students by item response patterns and enables us to describe subsets of learners with similar strengths and weaknesses. One-way analysis of variance was performed with profile as the between-subject factor for each item on the communication checklist to determine if significant differences by profile existed for checklist items. Results: Six profiles were identified with adequate model fit estimations. These profiles clustered into 3 groups: 2 high-performing profiles (HP1 and HP2), 2 average-performing profiles (AP1 and AP2), and 2 lower-performing profiles (LP1 and LP2). Within each group, 2 differing patterns of performance were seen. The profiles are generally distributed over the 9 years, suggesting that the profiles are stable over time. Some items provided more ability to significantly differentiate learner’s skills from others. “Asked questions to see what you (the patient) understood” differentiated between HP1 and HP2, and between AP1 and AP2. “Allowed you (the patient) to talk without interrupting” and “nonverbal behavior enriched communication” differentiated between AP1 and AP2, and between LP1 and LP2. Some items, particularly patient education items, were challenging across all 6 profiles, with learners scoring lower on those items relative to their average performance. “Asked questions to see what you understood,” had the lowest mean average score across all communication items, with “collaborated with you in identifying possible next steps” only slightly higher. Discussion: In this study, we describe performance patterns which provide a rich understanding of individual learners’ abilities and guidance for tailoring the curriculum for these critical clinical communication skills. Six profiles emerged from communication skills during a high-stakes exam. Even our highest-performing students (HP1) had relatively poor performance on patient education tasks. While the average (AP2) profile group performed solidly, they struggled with a subset of skills that may interfere with patient relationship development. The low-performing groups—many who would not have “failed” the CCSE—demonstrated a need for focused work on the broad swath of communication skills. Significance: Knowledge of unique performance patterns can provide information about not only who may need additional help but also what areas are appropriate to target. Rather than focusing on communication as a unitary domain, describing and classifying across a combination of their performance is critical in understanding how to support students both during remediation and developmentally through medical school. Acknowledgments: The authors would like to thank the standardized patients, faculty, and trainees for their engagement with the assessments.