Improving Multi-Domain Learning by Balancing Batches With Domain Information

Collections of data obtained or generated under similar conditions are referred to as domains or data sources. The distinct conditions of data acquisition or generation are often neglected, but understanding them is vital to address any phenomena emerging from these differences that might hinder model generalization. Multi-domain learning seeks to find the best way to train a model so that it performs adequately in all domains used during training. This paper explores multi-domain learning techniques that use explicit information about the domain of an example, in addition to its class. This study evaluates a general approach (Stew) by mixing all available data and also proposes two novel batch domain-regularization methods: Balanced Domains and Loss Sum. We train machine learning models with the listed approaches using datasets with multiple sources of data for image and audio classification tasks. The results suggest training a model using the Loss Sum method improves the results of models otherwise trained in a mix of all available data.