Continual Learning of 3D Point Cloud Generators.

Most continual learning evaluations to date have focused on fully supervised image classification problems. This work for the first time extends such an analysis to the domain of 3D point cloud generation, showing that 3D object generators are prone to catastrophic forgetting along the same vein as image classifiers. Classic mitigation techniques, such as regularization and replay, are only partially effective in alleviating this issue. We show that due to the specifics of generative tasks, it is possible to maintain most of the generative diversity with a simple technique of uniformly sampling from different columns of a progressive neural network. While such an approach performs well on a typical synthetic class-incremental setup, more realistic scenarios might hinder strong concept separation by shifting task boundaries and introducing class overlap between tasks. Therefore, we propose an autonomous branch construction (ABC) method. This learning adaptation relevant to parameter-isolation methods employs the reconstruction loss to map new training examples to proper branches of the model. Internal routing of training data allows for a more effective and robust continual learning and generation of separate concepts in overlapping task setups.