AnsweredAssumed Answered

Terasort reduce tasks failed  :out of memory ( java heap size)

Question asked by sriharsha on Jun 5, 2013
Hi

i was trying to test my culster performance and when i did a teragen for 10Gb i went through and when i tried doing a terasort on that map phase works fne(100%) and when reduce phase stars i get the bellow error

13/06/04 17:21:22 INFO mapred.JobClient:  map 95% reduce 0%
13/06/04 17:21:23 INFO mapred.JobClient:  map 96% reduce 0%
13/06/04 17:21:25 INFO mapred.JobClient:  map 97% reduce 0%
13/06/04 17:21:26 INFO mapred.JobClient:  map 98% reduce 0%
13/06/04 17:21:27 INFO mapred.JobClient:  map 99% reduce 0%
13/06/04 17:21:30 INFO mapred.JobClient:  map 100% reduce 0%
13/06/04 17:21:42 INFO mapred.JobClient: Task Id : attempt_201305201759_1088_r_000000_0, Status : FAILED on node hostname.com
Error: java.lang.OutOfMemoryError: Java heap space
        at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$MapOutputCopier.shuffleInMemory(ReduceTask.java:1774)
        at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$MapOutputCopier.getMapOutputFromFile(ReduceTask.java:1487)
        at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$MapOutputCopier.copyOutput(ReduceTask.java:1361)
        at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$MapOutputCopier.run(ReduceTask.java:1278)


is there a way where i can tweek these things

Outcomes