AnsweredAssumed Answered

benchmarking mapred with sort in mapr

Question asked by cindy on Aug 30, 2011
Latest reply on Sep 10, 2011 by andrew_wells
Hi,
I've just testing mapred performance in mapr and I'm very surprised that it takes long time for the shuffle-phase.  Our cluster has 7 nodes . The total Memory is 27 GB (4*7GB) and has 1,4 TB disk capacity (each has 200 GB). I run the standard random-writer|sort in the hadoop-example.jar with the command for 
 - hadoop jar $HADOOP_INSTALL/hadoop-*-examples.jar randomwriter random-data
 - hadoop jar $HADOOP_INSTALL/hadoop-*-examples.jar sort random-data sorted-data
I receive following results:
 -Time taken for random-data generation : 22min, 9sec
 -Time taken for random-data Sort : 6hrs, 24mins, 20sec

Do you think why it takes me a long time for sorting out the data in mapr?

Outcomes