Criterion validation of nutrient profiling systems: a systematic review and meta-analysis

Background: Nutrient profiling systems (NPSs) use algorithms to evaluate the nutritional quality of foods and beverages. Criterion validation, which assesses the relationship between consuming foods rated as healthier by the NPS and objective measures of health, is essential to ensure the accuracy of NPSs. Objective: We examined and compared NPSs that have undergone criterion validity testing in relation to diet-related disease risk and risk markers. Methods: Academic databases were searched for prospective cohort and cross-sectional studies published before November, 2022. NPSs were eligible if they incorporated multiple nutrients or food components using an algorithm to determine an overall summary indicator (e.g., a score or rank) for individual foods. Studies were included if they assessed the criterion validity of an eligible NPS. Validation evidence was first summarized in narrative form by NPS, with random effects meta-analysis where >= 2 prospective cohort studies assessed the same NPS and outcomes. Results: Of 4519 publications identified, 29 describing 9 NPSs were included in the review. The Nutri-Score NPS was assessed as having substantial criterion validation evidence. Highest compared with lowest diet quality as defined by the Nutri-Score was associated with significantly lower risk of cardiovascular disease (hazard ratio [HR]: 0.74; 95% confidence interval [CI]: 0.59, 0.93; n = 6), cancer (HR: 0.75; 95% CI: 0.59, 0.94; n = 5), all-cause mortality (HR: 0.74; 95% CI; 0.59, 0.91; n = 4) and change in body mass index (HR: 0.68; 95% CI: 0.50, 0.92; n = 3). The Food Standards Agency NPS, Health Star Rating, Nutrient Profiling Scoring Criterion, Food Compass, Overall Nutrition Quality Index, and the Nutrient-Rich Food Index were determined as having intermediate criterion validation evidence. Two other NPSs were determined as having limited criterion validation evidence. Conclusions: We found limited criterion validation studies compared with the number of NPSs estimated to exist. Greater emphasis on conducting and reporting on criterion validation studies across varied contexts may improve the confidence in existing NPSs.