Transferring variational autoencoders with coarse-and-fine alignment for open set broad classification

Transfer learning aims to help target learners with a different but related source domain. Open set recognition extends the settings of transfer learning for identifying whether an instance belongs to an unseen category. However, it is often pragmatic and valuable to further classify the unseen categories in target domain. We present a new setting called open set broad classification (OSBC) to classify unseen target categories which are open within the broad classes of source domain. Aiming at adapting to the challenging domain shift between unseen categories and seen categories, we propose a variational autoencoders model with coarse-and-fine alignment (CFVA) to leverage the structural information in the OSBC setting. First, two-stream decoders are employed and coarsely aligned by a relaxed parameters regularizer, which can absorb domain shift on features to facilitate fine alignment. Then fine alignment at encoding level enhances discriminative power of the latent representation by mixing the distributional structure hinted by source domain. Experimental results demonstrate the effectiveness of our CFVA approach in improving the accuracies in both unsupervised and semi-supervised cases.