AnsweredAssumed Answered

cldb crashed and cannot restart

Question asked by zongjie on Jul 9, 2014
Latest reply on Jul 14, 2014 by aaron
My mapr version is M3, OS is ubuntu 12.04

cldb is installed on server 192.168.0.161,
and the cldb is crashed and can hardly restart, the latest log is like this:


----------
`2014-07-10 17:33:49.212::INFO:  Started SelectChannelConnector@0.0.0.0:7221
2014-07-10 17:33:49,394 INFO CLDBServer [Lookup-1]: Rejecting RPC 2345.5 from 192.168.0.167:1111 with status 3 as CLDB is waiting for local kvstore to become master.
2014-07-10 17:33:51,973 INFO CLDBServer [RPC-10]: FSRegister: Request  FSID: 8270064052242629523 FSNetworkLocation:  FSHost:Port: 192.168.0.161- FSHostName: slave1 StoragePools  Capacity: 0 Available: 0 Used: 0 Role: 0 isDCA: false Received registration request
2014-07-10 17:33:51,973 INFO CLDBServer [RPC-10]: Cluster uuid is -6059180010972635351-9031140024879237064
2014-07-10 17:33:51,973 WARN Topology [RPC-10]: FileSever on slave1 reported an invalid topology . Ignoring reported topology
2014-07-10 17:33:51,982 INFO CLDBServer [RPC-10]: FSRegister: Registered FileServer: 192.168.0.161- at topology /default-rack/slave1
2014-07-10 17:33:59,118 INFO Containers [RPC-1]: FileServer 192.168.0.161 did not report volume 1 as part of FCR. Requesting node to  confirm missing containers
2014-07-10 17:34:49,516 INFO CLDBServer [RPC-10]: Rejecting RPC 2345.103 from 192.168.0.161:43789 with status 3 as CLDB is waiting for local kvstore to become master.
2014-07-10 17:35:54,734 INFO CLDBServer [Lookup-1]: Rejecting RPC 2345.5 from 192.168.0.169:1111 with status 3 as CLDB is waiting for local kvstore to become master.
2014-07-10 17:36:57,294 INFO CLDBServer [RPC-10]: Rejecting RPC 2345.40 from 192.168.0.161:36286 with status 3 as CLDB is waiting for local kvstore to become master.
2014-07-10 17:37:02,652 INFO Containers [RPC-2]: FileServer 192.168.0.161 did not report volume 1 as part of FCR. Requesting node to  confirm missing containers
2014-07-10 17:38:02,213 INFO CLDBServer [Lookup-3]: Rejecting RPC 2345.5 from 192.168.0.167:1111 with status 3 as CLDB is waiting for local kvstore to become master.
2014-07-10 17:39:03,817 INFO CLDBServer [Lookup-5]: Rejecting RPC 2345.5 from 192.168.0.161:58160 with status 3 as CLDB is waiting for local kvstore to become master.
2014-07-10 17:40:03,977 INFO CLDBServer [RPC-7]: Rejecting RPC 2345.103 from 192.168.0.161:51815 with status 3 as CLDB is waiting for local kvstore to become master.
2014-07-10 17:40:06,109 INFO Containers [RPC-8]: FileServer 192.168.0.161 did not report volume 1 as part of FCR. Requesting node to  confirm missing containers`

----------

When I run maprcli dump cldbnodes -zkconnect 192.168.0.161:5181 -json , the result is:

   
{

        "timestamp":1404986606700,
"status":"OK",
"total":2,
"data":[
  {
   "valid":"192.168.0.161:5660-"
  },
  {
   "invalid":[
    "192.168.0.162:5660-",
    "192.168.0.167:5660-"
   ]
  }
]
}




----------
and with command /opt/mapr/server/mrconfig sp list, the result is

     ListSPs resp: status 0:0
    
    2014-07-10 18:07:43,0526 ERROR Global mrconfig.cc:542 x.x.0.0:0 No SP on this disk

could anybody help me?
Thank you very much

Outcomes