Development of the Key Performance Indicators for Digital Health Interventions: A Scoping Review (Preprint)

BACKGROUND Development of the Key Performance Indicators for Digital Health Interventions: A Scoping Review OBJECTIVE Digital health interventions (DHIs) offer new methods for delivering healthcare, with the potential to innovate healthcare services. Key performance indicators (KPIs) play a role in evaluation, measurement, and improvement in healthcare quality and service performance. The scoping review question was developed following an initial search to identify literature to assist in the development of KPIs for an ongoing DHI initiative. During the initial search, it became clear that there was limited literature on how to develop specific and measurable KPIs that evaluate DHIs. The aim of this scoping review was to identify current knowledge and evidence surrounding the development of KPIs for DHIs. METHODS A rigorous literature search was conducted across ten key databases: AMED - The Allied and Complementary Medicine Database, CINAHL Complete, Health Source: Nursing/Academic Edition, MEDLINE, APA PsycINFO, EMBASE, EBM Reviews - Cochrane Database of Systematic Reviews, EBM Reviews - Database of Abstracts of Reviews of Effects, EBM Reviews - Health Technology Assessment, and IEEE Xplore. A descriptive summary of the literature was performed, and thematic analysis identified important or reoccurring themes. RESULTS Five references (representing four unique publications) were eligible for the review. Of the four included publications, two were articles on original research studies of a specific DHI, and two were overviews of methods for developing DHIs (not specific to a single DHI). All the included reports discussed the involvement of stakeholders in developing KPIs for DHIs. The step of identifying and defining the KPIs was completed using various methodologies, but all centered on a form of stakeholder involvement. Potential options for stakeholder involvement for KPI identification include the use of an elicitation framework, a factorial survey approach, or a Delphi study. Most of the included articles recognised the lack of literature relating to KPI development for DHIs, compared to the breath of literature available on the development of KPIs in other fields like health or informatics CONCLUSIONS Few articles were identified, highlighting a significant gap in the evidence-based knowledge in this domain. All the included articles discussed the involvement of stakeholders in developing KPIs for DHIs, which was performed using various methodologies. The articles acknowledged a lack of literature related to KPI development for DHIs. To allow comparability between KPI initiatives and facilitate work in the field, further research would be beneficial to develop a common methodology for KPI development for DHIs.