AnsweredAssumed Answered

TaskTracker crashes

Question asked by nneubauer on May 10, 2012
Latest reply on May 11, 2012 by nabeel
Hi, I noticed that TaskTracker crashes (seemingly randomly) on some nodes. I just asked another question regarding nodes registering as localhost.localdomain on the NameNode (might be related: [localhost.localdomain question][1]).

Logs shows:

        2012-05-11 11:24:48,699 ERROR org.apache.hadoop.mapred.TaskTracker: Failed to create and mount local mapreduce volume at /var/mapr/local/localhost.localdomain/mapred/. Please see logs at /opt/mapr//logs/createTTVolume.log
        2012-05-11 11:24:48,699 ERROR org.apache.hadoop.mapred.TaskTracker: Command ran /opt/mapr//server/createTTVolume.sh localhost.localdomain /var/mapr/local/localhost.localdomain/mapred/ /var/mapr/local/localhost.localdomain/mapred/taskTracker/
        2012-05-11 11:24:48,699 ERROR org.apache.hadoop.mapred.TaskTracker: Command output
        2012-05-11 11:24:48,700 ERROR org.apache.hadoop.mapred.TaskTracker: Can not start TaskTracker because org.apache.hadoop.util.Shell$ExitCodeException: Failed to create local mapred volume.
        Command used: /opt/mapr/bin/maprcli volume create -name "mapr.localhost.localdomain.local.mapred" -localvolumehost localhost.localdomain -localvolumeport 5660 -shufflevolume true -rereplicationtimeoutsec 300 -replication 1

        at org.apache.hadoop.util.Shell.runCommand(Shell.java:322)
        at org.apache.hadoop.util.Shell.run(Shell.java:249)
        at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:442)
        at org.apache.hadoop.mapred.TaskTracker.createTTVolume(TaskTracker.java:1889)
        at org.apache.hadoop.mapred.TaskTracker.initialize(TaskTracker.java:976)
        at org.apache.hadoop.mapred.TaskTracker.<init>(TaskTracker.java:2155)
        at org.apache.hadoop.mapred.TaskTracker.main(TaskTracker.java:5216)


/opt/mapr//logs/createTTVolume.log shows (a lot):

        ERROR (22) -  Unable to map host: localhost.localdomain to non-local ipaddress while creating volume mapr.localhost.localdomain.local.mapred

Any ideas?

  [1]: http://answers.mapr.com/questions/2339/nodes-register-as-localhostlocaldomain

Outcomes