Choosing a simulation assessment tool:

BACKGROUND To explore whether clinical, teamwork, and interpersonal skills of nurses could be monitored inexpensively at scale, we compared two simulation measurement tools at 160 primary health care facilities in Bihar, India. The aim was to determine if the less expensive, less resource-intensive real-time assessment tool (RTAT) could be used in place of the more expensive, more resource-intensive video coding tool. METHODS Embedded in a large-scale ongoing nurse mentoring program “AMANAT”, we conducted a cross-sectional study comparing the measurement tools to assess clinical performance in normal and emergency obstetrical and neonatal simulations. The RTAT consisted of a series of seven global questions related to clinical identification, management, use of evidence-based practices, communication and teamwork. We also analysed the simulations using a video coding tool, which included 80 clinical, teamwork and communication indicators. We calculated the means and standard errors for each indicator using both tools. We calculated the Cohen’s kappa coefficient to determine the level of agreement between the two tools. RESULTS We analysed paired data from the two tools for 222 simulations in three types of scenarios: normal spontaneous vaginal deliveries, neonatal resuscitations and post-partum haemorrhages. The mean scores on the RTAT were generally higher than those on the video coding tool. The kappa coefficients for all indicators indicated no agreement between the two tools. CONCLUSION RTAT performed poorly against the video coding tool to measure the clinical skills of nurses in simulated normal and complicated births at primary health care facilities in Bihar.