Comparing Traditional Versus Retrospective Pre-/Post-assessment in an Interdisciplinary Leadership Training Program

Objectives As the U.S. healthcare system shifts toward collaboration, demand for leaders with interdisciplinary skills increases. Leadership competencies guide interdisciplinary training programs; however, identifying cost-effective methods for evaluating leadership competencies is challenging, particularly when interdisciplinary trainees have different areas of expertise and professional goals. Traditional pre-/post-testing, a common method for evaluating leadership competencies, is subject to response-shift bias, which can occur when participants’ understanding of a construct changes between pre- and post-test. As a result, participants may rate their knowledge of the construct lower at post-test. Retrospective pre-tests are one method thought to reduce response-shift bias in pre-/post-tests. The current study explores the use of a retrospective pre-test to control for response-shift bias in an interdisciplinary training program. Methods Over three cohort years, thirty-four trainees from an interdisciplinary leadership program completed a self-assessment aligned with MCH leadership competencies. The traditional pre-test self-assessment was completed at the beginning of the training program. The retrospective pre-/post-test self-assessment was completed at the end of the training program. Results Retrospective pre/post-test scores indicate significant self-reported increases in all 24 leadership areas (p ≤ .001). Furthermore, participants’ self-ratings were significantly higher on the traditional pre-test for all 24 areas than on the retrospective pre-test (p ≤ .001). Conclusions for Practice Retrospective pre-tests appeared to control for response-shift bias and may be a cost-effective way to evaluate trainee change within an interdisciplinary leadership program. These findings suggest the methodology’s usefulness in interdisciplinary training and its potential use in the broader world of community-based MCH training initiatives.