AnsweredAssumed Answered

Issue completing jobs

Question asked by robinus2 on Sep 11, 2012
Latest reply on Jun 16, 2014 by nabeel
Hi there,

Today we started deploying Mapr M3 into production. However we're having problems completing jobs. During a typical job the job return this:


    12/09/11 16:33:20 INFO mapred.JobClient: Task Id : attempt_201209111629_0002_r_000001_2, Status : FAILED on node cl004.flxviz.com
    java.lang.Throwable: Child Error
            at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:267)
    Caused by: java.io.IOException: Task process exit with nonzero status of 7.
            at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:254)
    12/09/11 16:33:20 WARN mapred.JobClient: Error reading task output http://cl004.flxviz.com:50060/tasklog?plaintext=true&attemptid=attempt_201209111629_0002_r_000001_2&filter=stdout
    12/09/11 16:33:20 WARN mapred.JobClient: Error reading task output http://cl004.flxviz.com:50060/tasklog?plaintext=true&attemptid=attempt_201209111629_0002_r_000001_2&filter=stderr*




When I get the logs of the tasktracker, I see things like:


    2012-09-11 16:32:43,204 INFO org.apache.hadoop.mapred.TaskInProgress: Error from attempt_201209111629_0002_r_000002_1: java.lang.Throwable: Child Error
            at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:267)
    Caused by: java.io.IOException: Task process exit with nonzero status of 7.
            at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:254) on tasktracker tracker_cl004.flxviz.com:localhost/127.0.0.1:53126
    2012-09-11 16:32:46,234 INFO org.apache.hadoop.mapred.JobTracker: Removing task 'attempt_201209111629_0002_r_000002_1'
    2012-09-11 16:32:46,512 INFO org.apache.hadoop.mapred.JobTracker: Adding task (JOB_SETUP) 'attempt_201209111629_0002_m_000011_2' to tip task_201209111629_0002_m_000011, for tracker 'tracker_cl003.flxviz.com:localhost/127.0.0.1:42339'
    2012-09-11 16:32:48,027 INFO org.apache.hadoop.mapred.TaskInProgress: Error from attempt_201209111629_0002_m_000011_2: java.lang.Throwable: Child Error
            at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:267)
    Caused by: java.io.IOException: Task process exit with nonzero status of 7.
            at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:254) on tasktracker tracker_cl003.flxviz.com:localhost/127.0.0.1:42339
    2012-09-11 16:32:51,055 INFO org.apache.hadoop.mapred.JobTracker: Adding task (JOB_SETUP) 'attempt_201209111629_0002_r_000002_2' to tip task_201209111629_0002_r_000002, for tracker 'tracker_cl003.flxviz.com:localhost/127.0.0.1:42339'
    2012-09-11 16:32:51,056 INFO org.apache.hadoop.mapred.JobTracker: Removing task 'attempt_201209111629_0002_m_000011_2'
    2012-09-11 16:32:51,359 INFO org.apache.hadoop.mapred.TaskInProgress: Error from attempt_201209111629_0002_r_000002_2: java.lang.Throwable: Child Error*

Does anyone have a clue where to start?

Outcomes