Using Segmentation to Improve Machine Learning Performance in Human-in-the-Loop Systems

The expectations of Machine Learning systems are becoming increasingly demanding, namely in what concerns the diversity of applications, the expected accuracy, and the pressure for results. However, there are cases in which Human experts are needed to label the data, which may have a significant cost in terms of human resources and time. In these cases, it is often best to learn on-the-fly, without expecting for the whole data to be labeled. Often, it is desirable to guide the Human annotators into focusing on the more relevant instances: this constitutes the so-called active learning. In this paper we propose an approach in which a clustering algorithm is used to find groups of similar instances. Then, the procedure is guided with the objective of favoring the annotation of the groups that are under-represented in the labeled dataset. Results show that this approach leads to models that are, over time, more accurate and reliable.