Analysis of quality control outcomes of grass pollen identification and enumeration: experience matters

Pollen identification and enumeration is subject to human errors, and hence, it is crucial to evaluate the proficiency of pollen counters. Many networks still depend on manual pollen monitoring, and those adopting automation use manual counting data as a reference. A quality control exercise was undertaken across the AusPollen Aerobiology Collaboration Network to compare data analysis methods, gauge factors associated with accuracy, and improvements in counting proficiency. Counters were instructed to count grass and other pollen of the same two slides. Reported pollen concentrations were compared to an approximation of the true concentration values applying the published benchmark approach and alternative approach using bootstrapping technique. Participants were asked about their experience, training and usual practice via an online questionnaire. The majority (92% of 72) of reported values fell within acceptable ranges of variation from approximated true values. Outcomes were similar regardless of analysis approach, but bootstrapping did not require detection of outliers, and worked well with a small sample size with non-normal distribution. Counter reported pollen data were significantly shifted towards better outcomes compared to an initial exercise, and five of eight counters who were tested two times improved. Counting performance seemed not to be associated with amount of training received but was significantly related to counter experience. For future quality control exercises, particularly for small and skewed datasets, confidence limits of true pollen concentrations may be analysed by bootstrapping. Implementation of quality control exercises with harmonised analysis would enhance delivery of reliable pollen information to community, clinicians and government agencies for forecasting and environmental health management.