Natural Language Processing to Classify Caregiver Strategies for Participation-Focused Pediatric Rehabilitation

Research Objectives There is limited use of artificial intelligence for undertaking the complex task of participation-focused goal setting in pediatric rehabilitation. We therefore explored using natural language processing (NLP) to automate the classification of participation-focused caregiver strategies generated by the Young Children's Participation and Environment Measure (YC-PEM) into key predictors of participation (i.e., the child's environment/context, activity competencies, preferences, and sense of self). Design Cross-sectional study design. Setting Home and community setting. Participants 125 caregivers of young children receiving pediatric rehabilitation services in an early intervention (EI) service context. Most caregivers had pursued graduate education (56%). Their children were mostly male (56%), aged between 2 and 3 years (56%), and received services through EI because of developmental delay (68%). Interventions N/A. Main Outcome Measures Caregivers completed the YC-PEM with open-ended items asking caregivers to report on their strategies to support their child's participation in valued activities. This resulted in a total of 780 caregiver strategies that were manually and independently annotated by two research staff according to five classes (i.e., four key predictors of participation and a non-strategy class for irrelevant or indeterminate instances). Multinomial classification of these caregiver strategies was pursued across these five classes, experimenting with various classical machine learning and Transformer-based classification models commonly used in NLP (i.e., logistic regression, naïve Bayes, Bidirectional Encoder Representation from Transformers (BERT), and Bio-ClinicalBERT). The top-performing model was then used for subsequent experiments, including pipelined approaches classifying strategies into the five classes through stepwise classifications tasks that increase in their specificity. Results BERT achieved highest performance overall, with accuracy=64.47%, precision=0.73, recall=0.53 and F1=0.56. This Transformer-based model was, therefore, used for subsequent analyses testing classification tasks within pipelined approaches, which reached accuracy=58.06-88.15%, precision=0.56-0.82, recall=0.53-0.76 and F1=0.51-0.79. Conclusions Findings provide preliminary evidence for establishing an automated classification of caregiver strategies within participation-focused goal setting interventions under development that employ data from a Participation and Environment Measure. Author(s) Disclosures YC-PEM is licensed for distribution through CanChild Centre for Childhood Disability Research. Revenue from YC-PEM sales supports research and development activities.