Classification of lapses in smokers attempting to stop: A supervised machine learning approach using data from a popular smoking cessation smartphone app

Temporary smoking episodes after the quit date (‘lapses’) often lead to full relapse. Real-time support delivered via technology-mediated just-in-time adaptive interventions (JITAIs) offer a promising means of preventing lapses. To inform JITAI development, we used observational data from a popular smoking cessation app (‘Smoke Free’) to develop supervised machine learning algorithms to distinguish lapse from non-lapse reports at the group- and individual-level. We used data from app users with ≥20 unprompted craving feature entries within 3 months from the date of app registration, which included information about craving severity, mood, activity, social context, and lapse incidence. A series of group-level supervised machine learning algorithms (e.g., Random Forest, XGBoost) were trained and tested. Their ability to classify lapses for out-of-sample i) observations and ii) individuals were evaluated. Finally, a series of individual-level and hybrid (i.e., group- and individual-level) algorithms were trained and tested. A total of 791 participants were included, who provided 37,002 craving feature entries (7.6% lapses). The best-performing group-level algorithm had an area under the receiver operating characteristic curve (AUC) of 0.969 (95% CI = 0.961-0.978). Its ability to classify lapses for out-of-sample individuals ranged from poor to excellent (AUC = 0.482-1.000). Individual-level algorithms could be constructed for 39/791 participants with sufficient data, with a median AUC of 0.938 (range: 0.518 to 1.000). For 31/39 participants, the individual-level algorithm led to improved performance compared with the group-level algorithm. Hybrid algorithms could be constructed for 184/791 participants and had a median AUC of 0.825 (range: 0.375 to 1.000). Using unprompted app data appeared feasible for constructing a high-performing group-level lapse classification algorithm but its performance was variable when applied to unseen individuals. Separate algorithms trained and tested on each individual’s dataset led to improved performance but could only be constructed for a minority of participants. Triangulation of results with those from a prompted study design is recommended prior to developing and evaluating a JITAI.