Reproducibility assessment and uncertainty quantification in subjective dust source mapping

Accurate dust-source characterizations are critical for effectively modeling dust storms. A previous study developed an approach to manually map dust plume-head point sources in a geographic information system (GIS) framework using Moderate Resolution Imaging Spectroradiometer (MODIS) imagery processed through dust-enhancement algorithms. With this technique, the location of a dust source is digitized and recorded if an analyst observes an unobscured plume head in the imagery. Because airborne dust must be sufficiently elevated for overland dust-enhancement algorithms to work, this technique may include up to 10 km in digitized dust-source location error due to downwind advection. However, the potential for error in this method due to analyst subjectivity has never been formally quantified. In this study, we evaluate a version of the methodology adapted to better enable reproducibility assessments amongst multiple analysts to determine the role of analyst subjectivity on recorded dust source location error. Four analysts individually mapped dust plumes in Southwest Asia and Northwest Africa using five years of MODIS imagery collected from 15 May to 31 August. A plume-source location is considered reproducible if the maximum distance between the analyst point-source markers for a single plume is ≤10 km. Results suggest analyst marker placement is reproducible; however, additional analyst subjectivity-induced error (7 km determined in this study) should be considered to fully characterize locational uncertainty. Additionally, most of the identified plume heads (> 90%) were not marked by all participating analysts, which indicates dust source maps generated using this technique may differ substantially between users.