Application of deep-learning based techniques for automatic metrology on scanning and transmission electron microscopy images

Scanning or transmission electron microscopy (SEM/TEM) are standard techniques used during Research and Development (R & D) phases to study the structure and morphology of microscopic materials. Variety in object shapes and sizes are observed in such images to ensure robust micro- and nanomaterials critical dimension analysis. This way, precision and accuracy can be guaranteed during materials manufacturing processes. Such diversity and complexity in the data make it challenging to automatically extract the desired measurements of these microscopic structures. Existing tools in metrology often require many manual interactions, therefore being slow and prone to user errors. Proposed semiautomatic and automatic tools in the state-of-the-art are also limited and not designed to handle large variations across the images. Thus, the application of advanced machine or deep learning techniques could bring great efficiency in SEM/TEM image analysis and measurements for microscopic scale R & D processes. In this paper, we demonstrate the feasibility of deep-learning based object detection and instance segmentation models to perform automatic and accurate metrology on microscopic images with high object variability. We also show that custom object detection models prepared using pretrained weights, finetuned on very limited custom data, can outperform detection models built using traditional methods. This is particularly useful in metrology for the semiconductor industry, where data scarcity is common. When the data are available, we observe that it can be useful to be able to generate a large number of quality annotations to use instance segmentation. This could allow the training of more complex deep learning models for particle recognition and analysis. Therefore, we propose a semiautomatic tool to help produce annotations and demonstrate its application in an instance segmentation task.