Hierarchical Building Use Classification from Multiple Modalities with a Multi-Label Multimodal Transformer Network
International Journal of Applied Earth Observations and Geoinformation(2024)
摘要
Building use information is important for urban planning, city digital twins, and informed policy formulation. Prior research has predominantly focused on mapping building use in broad categories, offering general insight into their actual use. Our study investigates the extraction of hierarchical building categories, encompassing both broad and detailed classifications while accounting for mixed-use. To achieve this, we explore the fusion of building function information from satellite images, digital surface models (DSM), street view images, and point of interest (POI) data. We propose a novel multi-label multimodal transformer-based feature fusion network, which is capable of simultaneously predicting four broad categories and 13 detailed categories. Experimental results demonstrate the efficacy of our method, as it maps most of the building use categories, with the weighted average F1 score for four broad categories and 13 detailed categories of 91% and 77%, respectively. Our experiments underscore the critical role of satellite images in building use classification, with the inclusion of DSM data and POI significantly enhancing the classification accuracy. By considering detailed use categories and accounting for mixed-use, our method provides more detailed insights into land use patterns, thereby contributing to urban planning and management.
更多查看译文
关键词
Building hierarchical use classification,Multimodal integration,Mixed-use,Multi-label classification
AI 理解论文
溯源树
样例
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要