AnsweredAssumed Answered

Out of memory at shuffle phase

Question asked by kogut on Jun 10, 2013
Latest reply on Oct 7, 2013 by jerdavis

I got following error during a job execution:

2013-06-07 10:50:16,046 INFO org.apache.hadoop.mapred.TaskInProgress: Error from attempt_201306071023_0014_r_000000_3:
Error: java.lang.OutOfMemoryError: Java heap space
at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$MapOutputCopier.shuffleInMemory(
at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$MapOutputCopier.getMapOutputFromFile(
at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$MapOutputCopier.copyOutput(
at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$

Does anybody know any workarround or solution for that? It happens from time to time, which only makes things harder to solve.

I am using 2.1.0 version.