LAB-Net: Lidar and aerial image-based building segmentation using U-Nets

We describe our approach in the 2022 NORA MapAI competition. The objective of the competition was to construct methods that were able to detect and segment buildings from aerial imaging and laser data. There were two subtasks: (1) building segmentation from aerial imaging; (2) building segmentation from lidar data, optionally combined with aerial images. We trained multiple dynamic U-Net models with self-attention layers. For Task 1, we used a ResNet34-based encoder pre-trained on the ImageNet challenge dataset and further pre-trained the U-Net on another similar aerial image dataset. For Task 2, we adapted the dynamic U-Net to deal with multispectral data. Our ensembles placed us in second place, with the top score on Task 1. The complete source code for reproducing our results is available at https://github.com/HVL-ML/LAB-Net.