Evaluating Medical Spanish Proficiency: A Comparison of Physician Assistant Student Self-Assessment to Standardized Patient and Expert Faculty Member Ratings.

PURPOSE:Whether physician assistant (PA) students' self-assessment or standardized patient (SP) evaluations of students' medical Spanish proficiency accurately reflect their language proficiency is unclear. This study compares PA student and SP ratings with an expert faculty member's rating to determine whether student or SP ratings can be used to evaluate language proficiency. METHODS:Fifty-eight students participated in a single-station Spanish Objective Structured Clinical Examination (OSCE) at the midpoint of a medical Spanish curriculum. Using the Interagency Language Roundtable (ILR)-a 6-point, single-item language proficiency scale previously validated among physicians-PA students and SPs evaluated students' medical Spanish proficiency. Their scores were then compared with the scores derived by an expert faculty rater who had viewed a video of each student-SP encounter. The faculty's score was considered the gold standard. Correlation between scores was calculated using Spearman's rank correlation coefficient. RESULTS:Mean student scores were highest when rated by SPs (M = 3.8, SD = 0.9), followed by self (M = 3.0, SD = 0.9), and then faculty (M = 2.5, SD = 1.2). Spearman's rank correlation coefficient showed a strong positive correlation between students and the expert faculty rater (rs = 0.67, P < .001) and between SPs and the expert faculty rater (rs = 0.72, P < .001). The correlation was stronger for high- than for low-proficiency students. Students' self-rated scores showed significant improvement from baseline to the OSCE. CONCLUSIONS:PA students participating in a medical Spanish curriculum and SPs show good correlation with an expert faculty rater in assessing Spanish proficiency during an OSCE. Standardized patients demonstrate scoring leniency. The ILR has potential for tracking aggregate student progress and curriculum effectiveness. With training, student self-rating could be used for interval assessment of medical Spanish communication.