AnsweredAssumed Answered

ERROR (10009) -  Couldn't connect to the CLDB service

Question asked by hadoopnoob on May 9, 2013
Latest reply on Apr 6, 2018 by jgschmitz
Branched to a new discussion
Hi,

I'm trying to setup a MapR sandbox single node environment on an Ubuntu EC2 instance. I followed the instruction on the MapR documentation site but seem to have run into a problem - the only difference from the instructions is that I executed the mapr-single-node package from the repository.

I get the following error when trying to permission a user:

<code>mapr@master:/opt/mapr/server$ /opt/mapr/bin/maprcli acl edit -type cluster -user mapr:fc
ERROR (10009) -  Couldn't connect to the CLDB service
</code>

This is the error in the maprcli-root-0.log file:

<code>Header: hostName: master, Time Zone: Eastern Standard Time, processName: MapRCLI, processId: 30500, MapR Build Version: 2.1.2.18401.GA
2013-05-09 14:49:07,526 INFO  com.mapr.cliframework.driver.CLIMainDriver [main]: [node, list, -columns, hostname, -noheader]
2013-05-09 14:49:07,546 INFO  com.mapr.baseutils.cldbutils.CLDBRpcCommonUtils [main]: init
2013-05-09 14:49:07,563 ERROR com.mapr.baseutils.cldbutils.CLDBRpcCommonUtils getDataForParticularCLDB [main]: No data returned in RPC: CLDB Ips: 54.243.183.142-, Port: 7222. Continue searching for correct CLDB
2013-05-09 14:49:07,565 INFO  com.mapr.baseutils.cldbutils.CLDBRpcCommonUtils [main]: Bad CLDB credentials removed: CLDB Ips: 54.243.183.142-, Port: 7222
2013-05-09 14:49:07,565 ERROR com.mapr.baseutils.cldbutils.CLDBRpcCommonUtils getZkConnect [main]: No data is received from any cldb
2013-05-09 14:49:07,566 ERROR com.mapr.cli.ServerCommands getServicesInfo [main]: zkConnectString is null/empty. Cannot proceed further.
2013-05-09 14:49:07,579 INFO  com.mapr.baseutils.cldbutils.CLDBRpcCommonUtils [main]: init
2013-05-09 14:49:07,584 ERROR com.mapr.baseutils.cldbutils.CLDBRpcCommonUtils getDataForParticularCLDB [main]: No data returned in RPC: CLDB Ips: 54.243.183.142-, Port: 7222. Continue searching for correct CLDB
2013-05-09 14:49:07,585 INFO  com.mapr.baseutils.cldbutils.CLDBRpcCommonUtils [main]: Bad CLDB credentials removed: CLDB Ips: 54.243.183.142-, Port: 7222
2013-05-09 14:49:07,585 ERROR com.mapr.cli.ServerCommands sendRequest [main]: RPC Request to list Nodes failed. No data returned
2013-05-09 14:49:07,587 INFO  com.mapr.cliframework.driver.CLIMainDriver [main]: ERROR (10009) -  Couldn't connect to the CLDB service. Check if at least one CLDB is running.
</code>

cldb.log:

<code>Connection 90 came on 36f3b78e. But, peer ad7df6f:5660 did not advertise this ip
Connection 90 came on 36f3b78e. But, peer ad7df6f:5660 did not advertise this ip
2013-05-09 14:56:52,413 FATAL CLDB [WaitForLocalKvstore Thread]: CLDBShutdown: CLDB had master lock and was waiting for its local mfs to become Master.Waited for 7 (minutes) but mfs did not become Master. Shutting down CLDB to release master lock.
2013-05-09 14:56:52,413 INFO CLDBServer [WaitForLocalKvstore Thread]: Shutdown: Stopping CLDB
2013-05-09 14:56:52,414 INFO CLDB [Thread-12]: CLDB ShutDown Hook called
2013-05-09 14:56:52,414 INFO ZooKeeperClient [Thread-12]: Setting the clean cldbshutdown flag to true
2013-05-09 14:56:52,421 INFO ZooKeeperClient [Thread-12]: Zookeeper Client: Closing client connection:
2013-05-09 14:56:52,426 INFO CLDBServer [main-EventThread]: The CLDB received notification that a ZooKeeper event of type NodeDeleted occurred on path /datacenter/controlnodes/cldb/active/CLDBMaster
2013-05-09 14:56:52,427 INFO ZooKeeper [Thread-12]: Session: 0x13e8a917355000a closed
2013-05-09 14:56:52,427 INFO CLDB [Thread-12]: CLDB shutdown
</code>

Warden log:

<code>2013-05-09 14:58:03,680 INFO  com.mapr.job.mngmnt.hadoop.metrics.WardenRequestBuilder [cldb_monitor]: [e_SERV_FAIL, hostName, ma_host, ma_process]
2013-05-09 14:58:03,680 INFO  com.mapr.job.mngmnt.hadoop.metrics.WardenRequestBuilder [cldb_monitor]: []
2013-05-09 14:58:03,681 INFO  com.mapr.warden.service.baseservice.Service [cldb_monitor]: Need delayed alarm raising for: NODE_ALARM_SERVICE_CLDB_DOWN
2013-05-09 14:58:16,220 INFO  com.mapr.warden.centralconfig.PullCentralConfigTaskScheduler [PullCentralConfigTask]: Launching a separate process to execute /opt/mapr/server/pullcentralconfig
2013-05-09 14:58:33,678 INFO  com.mapr.warden.service.baseservice.Service [Thread-13-EventThread]: Process path: null. Event state: SyncConnected. Event type: None
2013-05-09 14:58:33,678 INFO  com.mapr.warden.service.baseservice.Service [Thread-13]: Connected to ZK: master:5181With State: State:CONNECTED Timeout:30000 sessionid:0x13e8a917355000b local:/10.215.223.111:45118 remoteserver:master/54.243.183.142:5181 lastZxid:0 xid:1 sent:1 recv:1 queuedpkts:0 pendingresp:0 queuedevents:0
2013-05-09 14:58:33,679 INFO  com.mapr.warden.service.baseservice.Service [Thread-13-EventThread]: ZK Connect state:State:CONNECTED Timeout:30000 sessionid:0x13e8a917355000b local:/10.215.223.111:45118 remoteserver:master/54.243.183.142:5181 lastZxid:0 xid:2 sent:2 recv:1 queuedpkts:0 pendingresp:1 queuedevents:0
2013-05-09 14:58:33,682 INFO  com.mapr.warden.service.baseservice.Service [Thread-13]: Node: /nodes/master/services/cldb does not exist yet
2013-05-09 14:58:45,082 INFO  com.mapr.warden.centralconfig.PullCentralConfigTaskScheduler [PullCentralConfigTask]: /opt/mapr/server/pullcentralconfig process terminated with status: 0
2013-05-09 14:59:53,340 ERROR com.mapr.job.mngmnt.hadoop.metrics.MaprRPCContext run [Thread-4]: RPC request failed with status: 1
</code>

Any help/direction would be awesome!

Thanks!

Outcomes