Rating of Physiotherapy Student Clinical Performance in A Paediatric Setting: Is It Possible to Gain Assessor Consistency?

Abstract Background: During workplace based clinical placements, best practice in assessment dictates that students should expect consistency between assessors rating their performance. To assist clinical educators (CEs) to provide a consistent assessment of physiotherapy student performance, nine paediatric vignettes depicting various standards of student performance, as assessed by the Assessment of Physiotherapy Practice (APP), were developed. The project aimed to evaluate the consistency of physiotherapy educators assessing student competence in a paediatric setting using video vignettes alongside the APP. Methods: Thirty-six CEs, with minimum 3-years clinical experience and had supervised a student within the past year, were sent three videos at four-week intervals. Videos depicted the same clinical scenario, however student performance varied with each video. Consistency among raters was assessed using percentage agreement to establish reliability. Results: The vignettes were assessed a combined total of 60 times. Across scenarios, percentage agreement at the not adequate level was 100%, and combining adequate or better, percentage agreement was >86%. The study demonstrated strong consensus when comparing not adequate to adequate or better student performance. Importantly, no student performance scripted as not adequate was passed by any assessor. Conclusions: Experienced educators demonstrate consistency in identifying a not adequate from adequate or better performance when assessing a one-off student performance using the APP. These validated video vignettes will be a valuable training tool to improve educator consistency when assessing student performance in paediatric physiotherapy.