AnsweredAssumed Answered

New Task Trackers not getting work

Question asked by thealy on Feb 6, 2013
Latest reply on Feb 7, 2013 by thealy
Running M3 / V2.0.0 on 21 nodes. Just replaced one bad node, and installed a new one. Web shows both OK. My balancer is set very low so a fair amount of data was migrated to each node as it was added. But when a daily M/R job is run, they are given no map jobs. A look at the TT log and they stop at "Successfully clear space alarm" (More trace below)

Strangely, neither node runs map tasks, but both are given reduce attempts, which work fine.
<code>
[time] INFO org.mortbay.log: Started SelectChannelConnector@0.0.0.0:50060
[time] INFO org.apache.hadoop.mapred.UserLogCleaner: Adding job_201301151503_0073 for user-log deletion with retainTimeStamp:1360267047626
[time] INFO org.apache.hadoop.mapred.UserLogCleaner: Adding job_201301151503_0070 for user-log deletion with retainTimeStamp:1360267047626
[time] INFO org.apache.hadoop.mapred.UserLogCleaner: Adding job_201301151503_0072 for user-log deletion with retainTimeStamp:1360267047626
[time] INFO com.mapr.fs.JobTrackerWatcher: Current running JobTracker is: hd-jt.xxx.yyy.zzz/11.22.33.22:9001
[time] INFO org.apache.hadoop.mapred.TaskTracker: clear space alarm.  Command invoked is : /opt/mapr//bin/maprcli alarm clear -alarm NODE_ALARM_TT_LOCALDIR_FULL -entity hdxx.yyy.zzz
[time] INFO org.apache.hadoop.mapred.TaskTracker: Sucessfully clear space alarm
</code>

Outcomes