Automatic Data Partitioning by Hierarchical Genetic Search

Automatic Data partitioning is one of the most crucial issues in the parallelization of programs for distributed memory message passing parallel machines. Several aspects of this problem are known to be NP-compfete and other researchers have proposed heuristic based solutions to solve this problem. In this paper, we propose a novel approach based on genetic algorithm as a powerful alternative. Our algorithm is interesting not only because it applies a randomized search technique to solve this problem, but also because it is simple and efficient. Moreover, our algorithm can afford to look at large search spaces of possible partitioning schemes to quickly arrive at the right data partition. We present some of the results from our prototype implementation of our algorithm on the IBM SP2 cluster.