Validity of data collection methods for time spent, professional involvement and treatment volume for the purpose of cost-effectiveness studies in dentistry

Objectives Economic evaluations can support provision of adequate and affordable oral care, requiring valid information on costs. The aim was to assess the validity of (a) patients' self-report (PS) and routine electronic patient records (EPR) regarding time spent per visit and (b) PS regarding types of treatment and type of dental professionals involved. Methods Data were collected in four dental clinics regarding time spent using PS and EPR, on types of treatment and dental professionals involved using PS. As reference standard for time spent, independent research assistants (RA) collected data on time per visit using stopwatches. As reference standard for types of treatment and of dental professionals involved, we used the dental clinic's Electronic Patient Files (DEPF). The two one-sided tests (TOST) equivalence procedure for the difference between paired means for time and kappa statistics for treatment and professional were used to assess agreement of data collection methods with the reference standards. Results Equivalence and agreement was good between (a) PS and RA registration concerning waiting time, appointment time and total time spent and (b) EPR and DEPF concerning appointment time. Agreement between PS and DEPF concerning types of treatment was moderate to fair (kappa values between 0.49 and 0.56 for preventive consultation, restoration, radiographs and extractions and between 0.15 and 0.26 for fluoride applications and sealants). Agreement between PS and DEPF for dental professional involved was fair (kappa = 0.41). Conclusions Data collection regarding time using PS and EPR was valid. Data collection via PS on treatment and professionals involved were not sufficiently valid and should occur via DEPF.