Active cross-query learning: A reliable labeling mechanism via crowdsourcing for smart surveillance

Crowdsourcing is an effective way to collect plenty of labeled data. Rather than just relying on feedback from the crowd, active learning can intentionally request informative instances to be annotated in surveillance applications. Previous work that combines crowdsourcing with active learning focuses on the scenario with the expert being responsible for the most matching task in common communication surveillance. Compared with similar methods, we propose an innovative approach based on the active cross-query learning scheme, allowing each ordinary worker instead of domain experts to label part of the selected query samples, especially in the networks of smart surveillance. By using the balanced incomplete block design (BIBD), each labeling task is repeated several times to complete the cross-query learning. The generated consensus labels are iteratively added to the existing labeled datasets for training the classifier. Experiments conducted on three real-world datasets with our algorithms demonstrate that our method ensures model accuracy and label quality in terms of text classification compared with the several state-of-the art algorithms.