Requirements-Driven Method to Determine Quality Characteristics and Measurements for Machine Learning Software and Its Evaluation

As the applications of machine learning algorithms in various fields are widely demanded, the development of machine learning software systems (MLS) is rapidly increasing. The quality of MLS is different from that of conventional software systems, in the sense that it depends on the amount and distribution of training data in a model learning and input data during operation. This is a major challenge in quality assurance of MLS development for the enterprise. In this paper, we propose a requirements-driven method to determine the quality characteristics of the MLS. Major contributions of this paper include: (1) Extending the quality characteristics of ISO 25010, which defines the conventional software quality, to those unique to MLS; this paper also defines its measuring method. (2) A method to identify requirements, i.e., issues to be determined in the requirements definition, in order to derive the quality characteristics and measurement methods for MLS, since the quality characteristics and the measurement method depend on the goals of the system under development. In order to evaluate the proposed method, we carried out an empirical study of the quality characteristics and measurement methods related to functional correctness and the maturity of the MLS for the enterprise. Based on the study, we compare the quality characteristics and measurement methods derived by the proposed method with those suggested by developers, and demonstrate the effectiveness of the proposed method.