Performance testing in lexical analysis on latest Twitter trends for enterprise network using PIG

Twitter usage is outgrowing each day and so is the ever changing trend. The data flow keeps on increasing each day through tweets and thus analysis on the latest trends are done by using most widely used framework for processing large datasets-Apache Hadoop. This paper presents an overview of Twitter Trend analysis using Apache PIG in Hadoop framework and its performance will be tested using lexical analysis. Hadoop is preferred over other frameworks due to its robustness, scalability, cost effectiveness and high speed which makes parallel processing over distributed network easy. The paper also shows results obtained through lexical analysis done on the Hadoop cluster to show the efficiency of the program.