Using Machine Learning Classification Algorithms in Official Statistics

In most surveys, the occupation and job-industry related questions are asked through open-ended questions, and the coding of this information into thousands of categories is done manually. This is very time consuming and costly. Given the requirement of modernizing the statistical system of countries, it is necessary to use statistical learning methods in official statistics for primary and secondary data analysis. Statistical learning classification methods are also useful in the process of producing official statistics. The purpose of this article is to code some statistical processes using statistical learning methods and familiarize executive managers about the possibility of using statistical learning methods in the production of official statistics. Two applications of classification statistical learning methods, including automatic coding of economic activities and open-ended coding of statistical centers questionnaires using four iterative methods, are investigated. The studied methods include duplication, support vector machine (SVM) with multi-level aggregation methods, a combination of the duplication method and SVM, and the nearest neighbor method.