Adaptive multi-task learning using lagrange multiplier for automatic art analysis

Numerous computer vision applications, such as image classification, have benefited from multi-task learning techniques. However, the relative weighting between each task’s loss is hard to be tuned by hand, causing multi-task learning prohibitive in real applications. In this paper, we present a novel and principled adaptive multi-task learning method that weights multiple loss functions based on lagrange multiplier strategy. Our method starts from the standard multi-task learning model. Based on Gaussian likelihood and lagrange multiplier, we then design an adaptive multi-task learning model to learn suitable weightings of each task and boost performance. In order to validate the feasibility of proposed method, we conduct automatic art analysis tests, including art classification and cross-modal art retrieval. Experimental results demonstrate that our method outperforms several state-of-the-art techniques, showing that performance is improved by up to 4.2% in art classification and 8.7% in cross-modal art retrieval when compared with the latest automatic loss weights learning method.