Clinical features-based machine learning models to separate sexually transmitted infections from other skin diagnoses

Introduction Many sexual health services are overwhelmed and cannot cater for all the individuals who present with sexually transmitted infections (STIs). Digital health software that separates STIs from non-STIs could improve the efficiency of clinical services. We developed and evaluated a machine learning model that predicts whether patients have an STI based on their clinical features. Methods We manually extracted 25 demographic features and clinical features from 1,315 clinical records in the electronic health record system at Melbourne Sexual Health Centre. We examined 16 machine learning models to predict a binary outcome of an STI or a non-STI diagnosis. We evaluated the models’ performance with the area under the ROC curve (AUC), accuracy and F1-scores. Results Our study included 1,315 consultations, of which 36.8% (484/1315) were diagnosed with STIs and 63.2% (831/1315) had non-STI conditions. The study population predominantly consisted of heterosexual men (49.5%, 651/1315), followed by gay, bisexual and other men who have sex with men (GBMSM) (25.7%), women (21.6%) and unknown gender (3.2%). The median age was 31 years (intra-quartile range (IQR) 26-39). The top 5 performing models were CatBoost (AUC 0.912), Random Forest (AUC 0.917), LightGBM (AUC 0.907), Gradient Boosting (AUC 0.905) and XGBoost (AUC 0.900). The best model, CatBoost, achieved an accuracy of 0.837, sensitivity of 0.776, specificity of 0.831, precision of 0.782 and F1-score of 0.778. The key important features were lesion duration, type of skin lesions, age, gender, history of skin disorders, number of lesions, dysuria duration, anorectal pain and itchiness. Conclusions Our best model demonstrates a reasonable performance in distinguishing STIs from non-STIs. However, to be clinically useful, more detailed information such as clinical images, may be required to reach sufficient accuracy.