A randomized evaluation of on-site monitoring nested in a multinational randomized trial

Background: Evidence from prospectively designed studies to guide on-site monitoring practices for randomized trials is limited. A cluster randomized study, nested within the Strategic Timing of AntiRetroviral Treatment (START) trial, was conducted to evaluate on-site monitoring. Methods: Sites were randomized to either annual on-site monitoring or no on-site monitoring. All sites were centrally monitored, and local monitoring was carried out twice each year. Randomization was stratified by country and projected enrollment in START. The primary outcome was a participant-level composite outcome including components for eligibility errors, consent violations, use of antiretroviral treatment not recommended by protocol, late reporting of START primary and secondary clinical endpoints (defined as the event being reported more than 6 months from occurrence), and data alteration and fraud. Logistic regression fixed effect hierarchical models were used to compare on-site versus no on-site monitoring for the primary composite outcome and its components. Odds ratios and 95% confidence intervals comparing on-site monitoring versus no on-site monitoring are cited. Results: In total, 99 sites (2107 participants) were randomized to receive annual on-site monitoring and 97 sites (2264 participants) were randomized to be monitored only centrally and locally. The two monitoring groups were well balanced at entry. In the on-site monitoring group, 469 annual on-site monitoring visits were conducted, and 134 participants (6.4%) in 56 of 99 sites (57%) had a primary monitoring outcome. In the no on-site monitoring group, 85 participants (3.8%) in 34 of 97 sites (35%) had a primary monitoring outcome (odds ratio = 1.7; 95% confidence interval: 1.1-2.7; p = 0.03). Informed consent violations accounted for most outcomes in each group (56 vs 41 participants). The largest odds ratio was for eligibility violations (odds ratio = 12.2; 95% confidence interval: 1.8-85.2; p = 0.01). The number of participants with a late START primary endpoint was similar for each monitoring group (23 vs 16 participants). Late START grade 4 and unscheduled hospitalization events were found for 34 participants in the on-site monitoring group and 19 participants in the no on-site monitoring group (odds ratio = 2.0; 95% confidence interval: 1.1-3.7; p = 0.02). There were no cases of data alteration or fraud. Based on the travel budget for on-site monitoring and the hours spent conducting on-site monitoring, the estimated cost of on-site monitoring was over US$2 million. Conclusion: On-site monitoring led to the identification of more eligibility and consent violations and START clinical events being reported more than 6 months from occurrence as compared to no on-site monitoring. Considering the nature of the excess monitoring outcomes identified at sites receiving on-site monitoring, as well as the cost of on-site monitoring, the value to the START study was limited.