Detecting adverse events in clinical care using natural language processing

Abstract Background Current methods for retrospective review of medical records require both time- and cost-wise a substantial effort. Therefore, we wanted to find the best method (based on natural language processing (NLP)) to select cases out of the medical records for further investigation in search for a (potentially preventable) adverse event (AE) to the decrease this effort. Methods The basic dataset consisted of 2987 medical records of patients who died during their hospitalization. To gain insight into the signal to noise ratio of the various resources, several subsets of our basic dataset were tested. Thereafter, we tested the scalability. After the best subset was chosen, several NLP algorithms were tested to select the best performing algorithm for the detecting of AEs. In the last experiment we tested the performance of the computer algorithms to predict potentially preventable AEs. The results of the NLP were compared with the outcome of the original retrospective medical record review. Results The dataset which contained he last three letters of the medical record showed the biggest potential. The scalability experiment showed that more data leads to a better performance of the algorithm. The best performing algorithm in the third test was the one based on support vector machine (SVM), with a precision of 79%, a negative predictive value (NPV) of 95% and a specificity of 85%. The results of the preventability experiment showed that the performance of the algorithms was almost equal to the results of the AEs. Conclusions In this study, we have shown that the SVM algorithm generates the most accurate results for the selection of cases for further investigation in the search for a (potentially preventable) AE. The sensitivity of the algorithms was around 75%. However, the SVM algorithm selected fewer cases to be examined for AEs compared to the original method. Consequently, this would lead to a lower workload for the committee. At the same time, there are a substantial number of cases, with potentially preventable AEs, not detected by machine learning.