What Does an OSCE Checklist Measure?

In this issue of Family Medicine, Drs Turner and Dankoski1 raise questions about the reliability and validity of objective structured clinical examinations (OSCEs). They propose that many institu- tions may be using an OSCE that does not have adequate reliability, and their interpretation of the litera- ture is that the validity of OSCEs has not been established. I agree with their conclusions. The reader, however, may be left with the impression that the reliability and validity of OSCEs, and specifically of OSCEs using standardized patients, can be im- proved to acceptable levels in their own institutions simply with added resources and attention to details. I believe this is possible when stan- dardized patients are used to test a narrow range of communication skills. I am less convinced about their use—even with the best of resources—as a comprehensive test of clinical competency. I think of clinical competency as a concept that can be displayed in a Venn diagram, with overlapping circles of knowledge, clinical skills, and clinical reasoning. Knowledge can be measured with a well- written multiple-choice test, and it is thought that skills and reasoning can be measured with OSCEs. But, many institutions use standardized patient OSCEs as an overall test of clinical competency and score those OSCEs with a checklist. In my opinion, standardized patient OSCEs that are graded with a checklist probably do not effec- tively measure knowledge, clinical skill, or reasoning. We should keep standardized patients but abandon the checklist.