Cascaded foreign object detection in manufacturing processes using convolutional neural networks and synthetic data generation methodology

Foreign object detection in manufacturing processes based on machine vision remains a challenge. The vastly different foreign objects and the complex background, as well as the scarcity of images with foreign objects constrain the application of traditional and deep learning methods, respectively. This paper discusses a novel method for intelligent foreign object detection and automatic data generation. A cascaded convolutional neural network to detect foreign objects on the surface of the tobacco pack is proposed. The cascaded network transforms the inspection into a two-stage YOLO based object detection, consisting of the tobacco pack localization and the foreign object detection. To address the scarcity of images with foreign objects, several data augmentation methods are introduced to avoid overfitting. Furthermore, a data generation methodology based on homography transformation and image fusion is developed to generate synthetic images with foreign objects. Models trained using synthetic images generated by this method show superior performance, which presents a viable approach to detecting newly introduced foreign objects. Extensive experimental results and comparisons on the tobacco pack foreign object dataset with several state-of-the-art methods demonstrate the effectiveness and superiority of the proposed method. The proposed cascaded foreign object detection network and synthetic data generation methodology have the potential for widespread applications.