MAPREDUCE CHALLENGES ON PERVASIVE GRIDS

This study presents the advances on designing and i mplementing scalable techniques to support the development and execution of MapReduce application in pervasive distributed computing infrastructures, in the context of the PER-MARE project. A pervasive fr amework for MapReduce applications is very useful i n practice, especially in those scientific, enterpris es and educational centers which have many unused o r underused computing resources, which can be fully e xploited to solve relevant problems that demand lar ge computing power, such as scientific computing appli cations, big data processing, etc. In this study, w e propose the study of multiple techniques to support vo latility and heterogeneity on MapReduce, by applyin g two complementary approaches: Improving the Apache Hado op middleware by including context-awareness and fault-tolerance features; and providing an alternat ive pervasive grid implementation, fully adapted to dynamic environments. The main design and implementation de cisions for both alternatives are described and val i ated through experiments, demonstrating that our approac hes provide high reliability when executing on perv asi e environments. The analysis of the experiments also leads to several insights on the requirements and constraints from dynamic and volatile systems, rein forcing the importance of context-aware information and advanced fault-tolerance features to provide effici nt and reliable MapReduce services on pervasive gr ds.