Evaluating Nontechnical Skills in US Emergency Departments Using Simulation Validating and Contextualizing a UK Assessment Tool

Introduction Nontechnical skills (NTS) in medicine are the "cognitive, social, and personal resource skills that complement technical skills contributing to safe and efficient care." We aimed to (1) evaluate the validity and reliability of a 12-element United Kingdom emergency medicine (EM) NTS assessment tool in the context of United States (US) EM practice and (2) identify behaviors unique to US clinical practice. Methods This was a mixed methods study conducted in 2 phases, following Kane's validity framework. The intended use of the NTS tool is to provide formative assessment of US EM physicians (EPs) from a video of simulated clinical encounters. In phase I, a focus group assessed the appropriateness of each aspect of the tool in the context of US EM practice by reviewing and identifying the NTS of an EP in a simulated clinical scenario. In phase II, EPs (N = 208) attending a national EM conference evaluated an EP's behaviors in 1 of 2 video simulations. Reliability in the form of internal consistency was calculated using Cronbach alpha. All participants suggested exemplar behaviors for the 12 elements in the context of their own clinical practice and generated new assessment elements. Results Internal consistency was acceptable (alpha > 0.7) for all categories, except teamwork and cooperation. Participants proposed 4 novel behavioral elements and suggested US exemplar behaviors for all 12 original elements. Conclusions This tool can be used to assess US EP's NTS for the purpose of formative assessment. Refinement of exemplar behaviors and inclusion of novel US-specific elements may optimize usability.