GeoBeam: A distributed computing framework for spatial data

Artificial intelligence and big data technology are important technical means to improve quantitative understanding of natural phenomena in Earth sciences. Large-scale spatial data provides a basic geospatial background for geoscience research. An effective and efficient distributed computing frame for spatial data is an indispensable infrastructure. It is still a challenge for disk-based distributed computing framework to analyze and process large-scale data efficiently. Such a challenge has driven the rapid development of various memory-based distributed computing platforms such as Spark, Flink, Apex, and more. Now, it is urgent to develop an efficient platform-independent distributed computing framework with a unique focus on large-scale spatial data. This paper provides a memory-based distributed computing framework named GeoBeam. It abstracts all the operations of spatial data into spatial pipeline, collection and transforms. Finally, they are encapsulated as feature dataset and feature store interfaces in GIS to shield the details of the underlying distributed operations. Experimental results show that GeoBeam can support efficient range query and processing of large-scale spatial data on Spark cluster and Flink cluster. GeoBeam provides an effective cross-platform distributed computing solution for fast processing of large-scale spatial data.