AnsweredAssumed Answered

Out of memory at shuffle phase

Question asked by kogut on Jun 10, 2013
Latest reply on Oct 7, 2013 by jerdavis
Hi,

I got following error during a job execution:

<pre>
2013-06-07 10:50:16,046 INFO org.apache.hadoop.mapred.TaskInProgress: Error from attempt_201306071023_0014_r_000000_3:
Error: java.lang.OutOfMemoryError: Java heap space
at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$MapOutputCopier.shuffleInMemory(ReduceTask.java:1750)
at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$MapOutputCopier.getMapOutputFromFile(ReduceTask.java:1475)
at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$MapOutputCopier.copyOutput(ReduceTask.java:1361)
at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$MapOutputCopier.run(ReduceTask.java:1278)

</pre>
Does anybody know any workarround or solution for that? It happens from time to time, which only makes things harder to solve.

I am using 2.1.0 version.

Thanks,

Outcomes