AnsweredAssumed Answered

Unable to start NodeManager service.

Question asked by danielsobrado on Mar 21, 2017
Latest reply on Mar 23, 2017 by mufeed
Branched from an earlier discussion

A similar issue here when installing a new node manually:

 

2017-03-22 10:12:36,435 ERROR com.mapr.hadoop.mapred.LocalVolumeAuxService: Failed to create and mount local mapreduce volume at /var/mapr/local/devmapr-n3/mapred. Please see logs at /opt/mapr/logs/createNMVolume.log
2017-03-22 10:12:36,435 ERROR com.mapr.hadoop.mapred.LocalVolumeAuxService: Command ran /opt/mapr/server/createTTVolume.sh devmapr-n3 /var/mapr/local/devmapr-n3/mapred /var/mapr/local/devmapr-n3/mapred/nodeManager yarn
2017-03-22 10:12:36,435 ERROR com.mapr.hadoop.mapred.LocalVolumeAuxService: Command output

 

2017-03-22 10:12:36,441 INFO org.apache.hadoop.service.AbstractService: Service org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl failed in state STARTED; cause: org.apache.hadoop.service.ServiceStateException: ExitCodeException exitCode=208:
org.apache.hadoop.service.ServiceStateException: ExitCodeException exitCode=208:
at org.apache.hadoop.service.ServiceStateException.convert(ServiceStateException.java:59)
at org.apache.hadoop.service.AbstractService.start(AbstractService.java:204)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices.serviceStart(AuxServices.java:173)
at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
at org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.serviceStart(ContainerManagerImpl.java:468)
at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
at org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120)
at org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceStart(NodeManager.java:267)
at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
at org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:477)
at org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:524)
Caused by: ExitCodeException exitCode=208:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:545)
at org.apache.hadoop.util.Shell.run(Shell.java:456)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:722)
at com.mapr.hadoop.mapred.LocalVolumeAuxService.initVolume(LocalVolumeAuxService.java:260)
at com.mapr.hadoop.mapred.LocalVolumeAuxService.serviceStart(LocalVolumeAuxService.java:229)
at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
... 10 more
2017-03-22 10:12:36,442 INFO org.apache.hadoop.ipc.Server: Stopping server on 33439
2017-03-22 10:12:36,444 INFO org.apache.hadoop.ipc.Server: Stopping IPC Server listener on 33439
2017-03-22 10:12:36,444 INFO org.apache.hadoop.ipc.Server: Stopping IPC Server Responder
2017-03-22 10:12:36,445 INFO org.apache.hadoop.ipc.Server: Stopping server on 8040

The log is not available: /opt/mapr/logs/createNMVolume.log

 

I tried manually to run:

 

createTTVolume.sh devmapr-n3 /var/mapr/local/devmapr-n3/mapred /var/mapr/local/devmapr-n3/mapred/nodeManager yarn

 

No logs, no errors, I tried to start nodemanager, same issue.

 

There is no /var/mapr folder, but I couldn't find it in the other nodes that don't have issues.

Outcomes