Inter-rater reliability of the Abbreviated Injury Scale scores in patients with severe head injury shows good inter-rater agreement but variability between countries. An inter-country comparison study

Introduction Substantial difference in mortality following severe traumatic brain injury (TBI) across international trauma centers has previously been demonstrated. This could be partly attributed to variability in the severity coding of the injuries. This study evaluated the inter-rater and intra-rater reliability of Abbreviated Injury Scale (AIS) scores of patients with severe TBI across three international level I trauma centers. Methods A total 150 patients (50 per center) were randomly selected from each respective trauma registry: University Medical Center Utrecht (UMCU), the Netherlands; John Hunter Hospital (JHH), Australia; and Harborview Medical Center (HMC), the United States. Reliability between coders and trauma centers was measured with the intraclass correlation coefficient (ICC). Results The reliability between the coders and the original trauma registry scores was 0.50, 0.50, and 0.41 in, respectively, UMCU, JHH, and HMC. The AIS coders at UMCU scored the most AIS codes of ≥ 4. Reliability within the trauma centers was substantial in UMCU (ICC = 0.62) and HMC (ICC = 0.78) and almost perfect in JHH (ICC = 0.85). Reliability between trauma centers was 0.70 between UMCU and JHH, 0.70 between JHH and HMC, and 0.59 between UMCU and HMC. Conclusion The results of this study demonstrated a substantial and almost perfect reliability of the AIS coders within the same trauma center, but variability across trauma centers. This indicates a need to improve inter-rater reliability in AIS coders and quality assessments of trauma registry data, specifically for patients with head injuries. Future research should study the effect of differences in AIS scoring on outcome predictions.