1599 Usability-focused development of a neonatal feedback dashboard for a low-resource neonatal unit, Malawi

BackgroundNeonatal mortality remains high in low-income countries; 22/1000 in Malawi. The World Health Organisation has called for actionable information systems to monitor and impact and outcomes. Electronic Audit and Feedback dashboards are an increasingly used healthcare quality improvement strategy which summarise clinical practice over a specified time period and feed information back to clinicians, via graphs and data visualisations. Audit and feedback dashboards have had limited previous use in low-resource hospitals. ‘NeoTree’ is a digital newborn quality improvement platform currently under development, that provides electronic audit and feedback. This study aimed to apply a usability-focused approach to co-developing the dashboard component of the NeoTree system in Kamuzu Central Hospital, Malawi.ObjectivesTo develop a first working prototype of the NeoTree dashboard and then gather usability feedback regarding this prototype to develop a beta version referred to as a Minimum Viable Product (MVP-1).MethodsOver a seven-month period Microsoft Excel and then Power BI were used to co-develop and user-test a first prototype dashboard, visualising data collected by nurses on the NeoTree app on bedside tablet devices at Kamuzu Central Hospital Neonatal Unit. Data were exported from the app to a cloud database in Amazon Web Service via a WIFI network. The first prototype was co-designed with neonatal health professionals, during scoping meetings with front-line nurses (micro-level), senior department managers (meso-level) and Ministry of Health experts (Macro-level stakeholders). Theory and evidence from current behaviour change and implementation science were mapped onto the dashboard. The first prototype was then user-tested with frontline neonatal nursing staff during one-to-one, video-recorded, usability sessions following think-aloud interview methods. System usability scores were collected from the same nurses. Rapid insights and inductive themes from thematic analysis of usability session transcripts informed dashboard changes. Iterative changes were also made to the dashboard while it was used at six morbidity and mortality (M&M) meetings and was played live on the ward screen for one-month.ResultsTwenty micro-level, nine meso-level and two macro-level participants attended scoping meetings. Twenty-three evidence-based feedback characteristics and six behaviour change techniques from Control Theory were mapped onto the dashboard. Ten frontline neonatal staff attended usability sessions, and fifty staff used the dashboard in M&M meetings and live on the ward. Eleven rapid insights included; data visualisations should reflect local understanding of colours and use locally appropriate language. Eight inductively generated themes included; difficulty interpreting complex charts, data alone are not useful, tendency to focus on extremes of graphs and positive interpretation of data. Nurses were motivated to change when data were accompanied by specific feedback characteristics such as take-home messages, standards, goals and recommendations. Mean system usability score of the NeoTree system including the dashboard, was high (89.3/100).ConclusionsElectronic Audit and Feedback dashboards for neonatal nurses can be highly usable in low-income countries. Usability of dashboards can be enhanced by clear messaging of how to change practice in response to feedback. This study substantiates previous electronic audit and feedback evidence from high-income countries, spearheads usability-focused approaches to dashboard development and could support dashboard-driven quality improvement in similar settings.