1526 Usability focused development of the neotree-beta-app for newborn care in a low resource neonatal unit, Malawi

BackgroundUsability is the quality of a user’s experience when interacting with an intervention, encompassing the effectiveness, efficiency and overall satisfaction of the user. It is crucial for user engagement and therefore success of digital health interventions. Agile, user-centred approaches are being applied more commonly to the development of healthcare apps but have scarcely been applied to developing digital health interventions for neonatal healthcare professionals (HCPs) in low-resourced hospitals, where usability may be particularly important.ObjectivesTo evaluate usability and usage of NeoTree, an app for Neonatal HCPs; and to conduct usability-focused development of the NeoTree user interface in a Malawian neonatal unit.MethodsUsability of the NeoTree app was evaluated using think-aloud usability sessions, System Usability Scores (SUS) and iterative feedback during six months of ward use. Usability sessions involved guiding end-users through the app while encouraging them to voice their thoughts aloud. Feedback notes by a facilitator were analysed within an agile ‘product backlog’, where they were converted into user-stories, grouped into larger ‘Epics’, and subjected to thematic analysis. Iterative feedback notes were similarly analysed in the same backlog. Resulting adjustments were made to the app either by the researcher via a bespoke web-editor platform (without requiring coding expertise) or coded by developers where necessary. For usage, the number and cadre of HCPs using the app, the proportion of admissions/outcomes captured digitally (coverage) and median time taken to complete forms (completion time) were calculated.ResultsSix neonatal HCPs attended usability sessions in which 20 themes, under 12 overarching themes, generated 57 app adjustments (stories). Forty-three stories (75%) were executed via the web-editor with only 14 (25%) requiring coding by the developers. Themes relating to usability as an electronic medical record included exhaustiveness of data schema, prevention of errors to support data integrity, ease of progression through the app, efficiency of data entry using shortcuts, navigation of user interface and relevancy of content. Themes relating to usability as a clinical decision support system included, confidentiality of identifiable information, cohesion with ward process, embedded education and decision support, locally coherent language, adaptability of user-interface to resource availability and printout design to facilitate handover. Eight HCPs completed SUS. Mean SUS was above average at 88.1 and 89.4. After suggested features were implemented in the app, it was used by 93 different HCPs to record 1323 admissions and 1197 outcomes, over a six-month period. Coverage slightly exceeded usual paper-based data collection, with NeoTree capturing 25 more admissions and 17 more outcomes than the ward clerk. Median completion times were 16 and 8 minutes for an admission and outcome respectively.ConclusionsThis study demonstrates how a digital health app can be optimised according to think-aloud usability insights to produce a highly usable intervention that is ultimately used and taken up by newborn HCPs in a low-resource neonatal unit. This study spearheads translational approaches to digital health intervention development by combining agile, user-centred design with traditional qualitative methods to expedite timely and pragmatic development of NeoTree. These findings could inform optimisation and successful uptake of similar apps in other low-resource settings.