Trends in Large-Scale Graph Processing

Graphs represent an excellent data-structure for representing objects and relations among them. With the emergence of social networks, efficient mining of large graphs has turned these structures into an important analytics problem. The irregular structure of graphs make them hard to parallelize and solving this problem has become an interesting systems research area. The different approaches to this problem cover in-memory processing in both distributed as well as single machine systems and out-of-core systems storing the graphs on secondary storage. In this work we provide an overview of the state of the art in each of these categories. Galois is a generic parallel processing system with a recent modification allowing for efficient shared-memory graph processing. Being a general programming model it allows greater flexibility and adaptability to certain problems then restricted DSL systems. PowerGraph represents the state of the art for in-memory distributed graph processing systems introducing a different approach to graph partitioning leading to reduced communication and memory overheads. Unlike the previous systems, X-Stream adapts its programming model to maximize the achieved bandwidth to secondary storage using the fact that sequential access to any type of storage always outperforms random access. Finally we briefly present our own work and future directions in this area.