Author: Nabeel Moidu, last modified by Jonathan Bubier on December 4, 2014
Original Publication Date: November 11, 2014
To enable debug level logging for a particular class used by the TaskTracker, use the maprcli setloglevel command. Ex:
$ maprcli setloglevel tasktracker -classname org.apache.hadoop.mapred.LinuxTaskController -loglevel DEBUG -node `hostname -f` -port 50030
Alternatively perform the same change via the log4j configuration by adding the following to /opt/mapr/hadoop/hadoop-0.20.2/conf/log4j.properties:
Note that this change requires a restart of the TaskTracker to take effect.
The following are other examples of using maprcli setloglevel to enable debug level logging for other TaskTracker classes.
$ maprcli setloglevel tasktracker -node <ip/host of TT> -port 50060 -classname org.apache.hadoop.ipc.Server -loglevel WARN
$ maprcli setloglevel tasktracker -node <ip/host of TT> -port 50060 -classname org.apache.hadoop.ipc.TaskTracker -loglevel ERROR
Reverting the logging back to default setting :
To revert the logging level back to the default setting, replace DEBUG/WARN to INFO in /opt/mapr/hadoop/hadoop-0.20.2/conf/log4j.properties. Ex:
To revert via maprcli use the following syntax with the previously used class name:
$ maprcli setloglevel tasktracker -classname org.apache.hadoop.mapred.LinuxTaskController -loglevel INFO -node `hostname -f` -port 50030
Viewing the debug output:
The debug output can be viewed in the main Tasktracker log in the Hadoop log directory. Typically this is /opt/mapr/hadoop/hadoop-0.20.2/logs/hadoop-<username>-tasktracker-<hostname>.log. Depending on the class for which debug logging is enabled the resulting output may be in the map-reduce task attempt logs. These logs are typically found under /opt/mapr/hadoop/hadoop-0.20.2/logs/userlogs/<job id>/<attempt id>/.
What class to enable debug for:
To determine the class which requires debug logging look at where task failures occur during a job run, and add debug for those classes and the immediately preceding ones. Monitor the logs after debug logging is enabled to see if it can throw further light on what happened between those log entries around the failure event.