How to configure tasktracker to use a specific Garbage Collector than default?

Document created by najmuddin_chirammal Employee on Feb 7, 2016Last modified by najmuddin_chirammal Employee on Feb 7, 2016
Version 2Show Document
  • View in full screen mode

Author: Najmuddin Chirammal

 

Original Publication Date: December 2, 2014

 

TaskTracker (or any java process) uses a garbage collector based on certain hardware and operating system configurations. There are occasions when this may need to be changed. In this article we go through the steps required to configure the TaskTracker to use a specific GC instead of the default method chosen by the tasktracker jvm when it comes up.

 

Solution:

 

- Specify the GC option in hadoop-env.sh

In the environment file for for hadoop variables, add the option,

"-XX:+UseConcMarkSweepGC" 

This should go into the file /opt/mapr/hadoop/hadoop-0.20.2/conf/hadoop-env.sh (On the tasktracker node) to use 'ConcMarkSweepGC' collector for Garbage collection instead of the default. This affects all jobs that are launched on the node after the change.

 

- Specify the option during job launch

This is particularly useful if you only want the garbage collector for one particular job and not all the jobs launched on a node. To achieve this, specify the following option while launching the job:

hadoop jar <jar file> <app> -Dmapred.child.java.opts="-XX:+UseConcMarkSweepGC"

 

 

Attachments

    Outcomes