AnsweredAssumed Answered

Host services are not getting up while adding new host on cluster

Question asked by vikashardia2010 on Oct 5, 2015
Latest reply on Dec 3, 2015 by mufeed
I have a one node cluster of MapR and now i am adding another node in the cluster. I am able to start warder after installing the rpm on new host by following this link

http://doc.mapr.com/display/MapR/Adding+Nodes+to+a+Cluster

But the host is not listing in the Web UI so i looked at the logs and below is the error i found in warden logs:

    2015-10-06 00:05:26,779 INFO  com.mapr.warden.service.baseservice.Service [Timer-1]: -------------Service is starting for: nodemanager
    2015-10-06 00:05:26,780 INFO  com.mapr.warden.service.baseservice.Service$ServiceMonitorRun [Timer-1]: Command: [/opt/mapr/hadoop/hadoop-2.5.1/sbin/yarn-daemon.sh, status, nodemanager], Directory: /opt/mapr/hadoop/hadoop-2.5.1/sbin/
    2015-10-06 00:05:26,783 INFO  com.mapr.warden.service.baseservice.Service [nodemanager_monitor]: Adding pluggable alarm NODE_ALARM_SERVICE_NODEMANAGER_DOWN to service: nodemanager
    2015-10-06 00:05:28,780 ERROR com.mapr.warden.service.baseservice.Service executeSimpleSHHCommand [nodemanager_monitor]: Error while running command: [maprcli, alarm, add, -alarm, NODE_ALARM_SERVICE_NODEMANAGER_DOWN, -terse, nanmd, -service, nodemanager, -displayName, NodeManagerDown, -baseService, 1, -serviceName, NodeManager, -displayName, NodeManagerDown, -baseService, 1, -serviceName, NodeManager, -displayName, NodeManagerDown, -baseService, 1, -serviceName, NodeManager, -displayName, NodeManagerDown, -baseService, 1, -serviceName, NodeManager, -displayName, NodeManagerDown, -baseService, 1, -serviceName, NodeManager]
    2015-10-06 00:05:28,781 ERROR com.mapr.warden.service.baseservice.Service executeSimpleSHHCommand [nodemanager_monitor]: ERROR (22) -  Terse name of nanmd already exists in the system.
    ERROR (17) -  Alarm NODE_ALARM_SERVICE_NODEMANAGER_DOWN already exists in the system.
    2015-10-06 00:05:28,781 WARN  com.mapr.warden.service.baseservice.Service [nodemanager_monitor]: Unable to add alarm NODE_ALARM_SERVICE_NODEMANAGER_DOWN
    2015-10-06 00:05:28,784 INFO  com.mapr.warden.service.baseservice.Service [nodemanager_monitor]: Alarm clearing command: [/opt/mapr/bin/maprcli, alarm, clear, -alarm, NODE_ALARM_SERVICE_NODEMANAGER_DOWN, -entity, mapr1]
    2015-10-06 00:05:30,753 ERROR com.mapr.warden.service.baseservice.Service executeSimpleSHHCommand [nodemanager_monitor]: Error while running command: [/opt/mapr/bin/maprcli, alarm, clear, -alarm, NODE_ALARM_SERVICE_NODEMANAGER_DOWN, -entity, mapr1]
    2015-10-06 00:05:30,754 ERROR com.mapr.warden.service.baseservice.Service executeSimpleSHHCommand [nodemanager_monitor]: ERROR (2) -  Operation failed. Error: No such entity


Same type of error in starting other services like fileserver and hoststats

Outcomes