Inter-Rater Agreement In The Assessment Of Video Recordings Of Eye Drop Instillation By Glaucoma Patients

PurposeTo create a standardized method for evaluating the video recordings of patients self-instilling eye drops and to determine the level of agreement of eye drop instillation efficacy, safety and efficiency ratings by three masked graders.DesignProspective cross-sectional study.Participants78 patients with open-angle glaucoma or ocular hypertension who had at least 6 months of experience with the use of eye drop medications.MethodsParticipants were video recorded while self-instilling artificial tears sequentially to both eyes. Three masked observers graded these video recordings on three criteria: efficacy (the determination of whether an eye drop was instilled on the ocular surface), safety (assessment of whether the tip of the medication bottle made contact with the ocular surface or eyelids), and efficiency (the number of eye drops expressed from the bottle).Main Outcome MeasuresAfter grading the video recordings based on efficacy, safety, and efficiency, kappa statistics were used to estimate inter-rater agreement.ResultsThe mean kappa level of agreement for efficacy, safety, and efficiency was 0.64 (95% confidence interval (CI), 0.42-0.87), 0.73 (95% CI, 0.58-0.88), and 0.62 (95% CI, 0.42-0.81), respectively.ConclusionsWe demonstrated good inter-rater reproducibility of the masked analysis of video recordings of patients self-instilling eye drops based on three criteria: efficiency, safety, and efficacy.