A realist approach to implementation fidelity in a mixed-method evaluation of electronic decision support systems to improve the quality of antenatal care in Nepal

Background Understanding implementation fidelity, or adherence to the intervention-as-intended, is essential to interpreting the results of evaluations. In this paper, we propose a longitudinal, explanatory approach to implementation fidelity through a realist evaluation lens. We apply this approach to a mixed-method assessment of implementation fidelity to an electronic decision support system intervention to improve the quality of antenatal care in Nepal. Methods The tablet-based electronic decision support system was implemented in 19 primary care facilities in Nepal. As part of the project’s process evaluation, we used four data sources – monitoring visit checklists and fieldnotes, software backend data, and longitudinal case studies in four facilities – to examine three components of fidelity: use at the point of care, use for all antenatal visits, and quality of data entry. Quantitative data were analysed descriptively. Qualitative data were analysed thematically using template analysis to examine descriptive findings across the three fidelity components and later to develop and reflect on the causal mechanisms. Findings were synthesised, drawing on Normalization Process Theory, to understand the processes driving the different patterns of fidelity observed. Results Fidelity to point-of-care use declined over time with healthcare providers often entering data after antenatal visits had ended because providers understood the intervention as primarily about recordkeeping rather than decision support. Even in facilities with higher fidelity to point-of-care use, software decision-support prompts were largely ignored. Low antenatal client caseloads and the suggestion by fieldworkers to practice back-entering data from previous antenatal visits undermined understanding of the intervention’s purpose for decision support. Conclusions Our assessment explains how and why patterns of implementation fidelity occurred, yielding more nuanced understanding of the project evaluation’s null result that moves beyond intervention vs implementation failure. Our findings demonstrate the importance of discussing intervention theory in terms fieldworkers and participants understand so as not to undermine fidelity. ### Competing Interest Statement The authors have declared no competing interest. ### Clinical Trial NCT03700034 ### Funding Statement The mIRA project, of which this study was part, was funded by the United Kingdom Medical Research Council (MRC) Newton Fund (MR/R022127/1). Work to develop the mIRA EDSS was funded by the Department of Biotechnology (DBT), India (BT/IN/DBT-MRC/DIFD/DP/14/2018-19). ### Author Declarations I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained. Yes The details of the IRB/oversight body that provided approval or exemption for the research described are given below: Kathmandu University School of Medical Sciences' Institutional Review Committee (ref: IRC-KUSMS 25/22), Nepal Health Research Council's Ethical Review Board (ref: 2695) and the London School of Hygiene & Tropical Medicine's Interventions Research Ethics Committee (ref: 25094-1) gave ethical approval for this work, as part of the mIRA project. I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals. Yes I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance). Yes I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable. Yes All fully anonymised, quantitative data produced in the present study are available upon reasonable request to the authors. The qualitative data produced in the present study involved small numbers of participants in potentially identifiable settings that cannot be fully anonymised, so this data is not available for sharing.