GeoBeam: A distributed computing framework for spatial data
Computers & Geosciences(2019)
Abstract
Artificial intelligence and big data technology are important technical means to improve quantitative understanding of natural phenomena in Earth sciences. Large-scale spatial data provides a basic geospatial background for geoscience research. An effective and efficient distributed computing frame for spatial data is an indispensable infrastructure. It is still a challenge for disk-based distributed computing framework to analyze and process large-scale data efficiently. Such a challenge has driven the rapid development of various memory-based distributed computing platforms such as Spark, Flink, Apex, and more. Now, it is urgent to develop an efficient platform-independent distributed computing framework with a unique focus on large-scale spatial data. This paper provides a memory-based distributed computing framework named GeoBeam. It abstracts all the operations of spatial data into spatial pipeline, collection and transforms. Finally, they are encapsulated as feature dataset and feature store interfaces in GIS to shield the details of the underlying distributed operations. Experimental results show that GeoBeam can support efficient range query and processing of large-scale spatial data on Spark cluster and Flink cluster. GeoBeam provides an effective cross-platform distributed computing solution for fast processing of large-scale spatial data.
MoreTranslated text
Key words
GeoBeam,Spatial data,Distributed computing,Beam,Spark,Flink
AI Read Science
Must-Reading Tree
Example
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
Generate MRT to find the research sequence of this paper
Chat Paper
Summary is being generated by the instructions you defined