Author: Najmuddin Chirammal
Original Publication Date: December 2, 2014
TaskTracker (or any java process) uses a garbage collector based on certain hardware and operating system configurations. There are occasions when this may need to be changed. In this article we go through the steps required to configure the TaskTracker to use a specific GC instead of the default method chosen by the tasktracker jvm when it comes up.
- Specify the GC option in hadoop-env.sh
In the environment file for for hadoop variables, add the option,
This should go into the file /opt/mapr/hadoop/hadoop-0.20.2/conf/hadoop-env.sh (On the tasktracker node) to use 'ConcMarkSweepGC' collector for Garbage collection instead of the default. This affects all jobs that are launched on the node after the change.
- Specify the option during job launch
This is particularly useful if you only want the garbage collector for one particular job and not all the jobs launched on a node. To achieve this, specify the following option while launching the job:
hadoop jar <jar file> <app> -Dmapred.child.java.opts="-XX:+UseConcMarkSweepGC"