Unpacking the "Black Box": How an SMS-Based Continuing Medical Education Intervention Improved Medical Knowledge Among HIV Clinicians in Vietnam.

Background: A mobile-based continuing medical education (mCME) intervention implemented over 6 months between 2016 and 2017, consisting of daily SMS multiple choice quizzes and access to online daily readings and CME courses, was shown to be effective in increasing medical knowledge among HIV providers in Vietnam. We hypothesized this improvement was a result of "lateral learning," a process in which the daily SMS quizzes acted as a stimulus for interacting with other study materials. Methods: We explored how study materials directly provided by the intervention - the daily readings and the online CME courses - and independent study behaviors, such as using medical textbooks and reviewing national guidelines, contributed to medical knowledge as measured by baseline and endline exams. At baseline, there were 53 participants each in the intervention and control groups (N=106). Using linear regression models, we estimated the association between intervention-prompted and independent study behaviors and endline test scores. We also conducted a series of interaction analyses to test the extent to which the effect of daily quiz performance on endline test scores depended on use of the intervention-prompted or independent study materials. Finally, we estimated the proportion of variance in endline test scores explained by each of the intervention-prompted behaviors. Results: The average medical knowledge test score among all participants was 46% at baseline and 54% at endline. Among the intervention group, 82% of the daily quizzes were answered, although only about half were answered correctly. Responding to the daily quizzes (beta=0.24; P=.05), quiz performance (beta=0.42; P<.001), and accessing daily readings (beta=0.22; P=.06) were statistically significantly associated with higher endline test scores. While accessing the online CME courses and some of the independent study behaviors, such as use of medical textbooks, had positive associations with endline test scores, none reached statistical significance. Quiz performance explained 51% of the variation in endline test scores. Interaction analysis found that quiz performance had a stronger, but not statistically significant, association with endline test scores when both daily readings (beta=0.87; P=.08) and online CME courses (beta=0.25; P=.09) were accessed more frequently. Conclusion: In mCME interventions, daily SMS quizzes can effectively act as a stimulus for uptake of study behaviors when paired with access to relevant readings and online courses. While further investigation is needed to more fully understand the role of outside study materials, we believe this model has the potential for further use in Vietnam and other low-resource settings.