Author: Jitendra Yadav, last modified by Hao Zhu on May 8, 2015
Original Publication Date: May 1, 2015
MapReduce job fails with the following error:
Failed to create directory /tmp/mapr-hadoop/mapred/local/taskTracker/user - No such file or directory failed to initialize user directory
However following directory does exist: /tmp/mapr-hadoop/mapred/local/ttprivate/taskTracker/user.
TT Private is used by Task Tracker for its own usage(Permission 0700).Job's files are downloaded in private cache firstly and then job is initialized in public cache (/tmp/mapr-hadoop/mapred/local/taskTracker/).
The distributed log splitting feature is known to behave in a faulty manner at times.
In this case either the /tmp/mapr-hadoop/mapred/local/taskTracker/ does not have the correct permission or it actually dose not exist.
One of the major culprit of this issue is linux tmpwatch package which clears the /tmp directory and removes old unused directory files.
TT private is created automatically when Task Tracker starts therefore it is recommended to restart the Task Tracker on affected nodes so directories can be created with proper permissions.If tmpwatch is causing the issue, please disbale tmpwatch or configure it to ignore /tmp/mapr-hadoop path while deleting. For example:
/usr/sbin/tmpwatch "$flags" -x /tmp/.X11-unix -x /tmp/.XIM-unix \
-x /tmp/.font-unix -x /tmp/.ICE-unix -x /tmp/.Test-unix \
-X '/tmp/hadoop-*' -X '/tmp/hsperfdata_*' 10d /tmp