Paying Deep Attention to Both Neighbors and Multiple Tasks.

In this paper, we propose a novel deep attention neural network. This network is designed for multi-task learning. The network is composed of multiple dual-attention layers of attention over neighbors and tasks. The neighbor attention layer represents each data point by paying attention over its neighboring data points, i.e., the output of this layer of a data point is the weighted average of its neighbors’ inputs, and the weighting scores are calculated according to the similarity between the data point and its neighbors. The task attention layer takes the output of the neighbor attention layer as input, and transfer it to multiple task-specific representations for an input data point, and uses attention mechanism to calculate the outputs for different tasks. The output of the input data points for a task is calculated by a weighted average over all the task-specific representations, and the weighting scores are based on the similarity between the target task and the other tasks. The outputs of the neighbor attention layer and task attention layer are concatenated as the output of one dual-attention. To train the parameters of the network, we minimize the classification losses and encourage the correlation among different tasks. The experiments over the benchmark of multi-task learning show the advantage of the proposed method.