AnsweredAssumed Answered

Issue with CLDB after upgrade from 2.1.2 to 3.0.1

Question asked by mandoskippy on Sep 12, 2013
Latest reply on Sep 12, 2013 by mandoskippy
I am assisting a friend upgrading his cluster. I performed the EXACT same steps on my cluster and everything worked well, so I am confused on why he is having issues.   Basically after the upgrade, the CLDB is shooting out some errors and refusing to come up.  I am not sure what to make of it, but could use some assistance from the group.

<pre>
2013-09-12 10:26:26,351 INFO HttpServer [main]: Listener started on SelectChannelConnector@0.0.0.0:7221 port 7221
2013-09-12 10:26:26,351 INFO HttpServer [main]: Starting Jetty WebServer
2013-09-12 10:26:26.351:INFO::jetty-6.1.26
2013-09-12 10:26:26,443 INFO CLDBServer [RPC-2]: Rejecting RPC 2345.103 from 192.168.16.103:53306 with status 3 as CLDB is waiting for local kvstore to become master.
2013-09-12 10:26:26.725:INFO::Started SelectChannelConnector@0.0.0.0:7221
2013-09-12 10:26:27,155 INFO CLDBServer [RPC-6]: FSRegister: Request  FSID: 3671575442874589802 FSNetworkLocation:  FSHost:Port: 192.168.16.100- FSHostName: box0.hadoop StoragePools 695b377db7b43c1100521aacc20bbfae-bc624c063bcd6eb000521aacbe0cf838-d9a1a643c94fa49b00521aacc00dd334-e8b223b2599f7b9700521aacbc0c50ae- Capacity: 18270203 Available: 17476705 Used: 793498 Role: 0 isDCA: false Received registration request
2013-09-12 10:26:27,156 INFO CLDBServer [RPC-6]: Cluster uuid is -5403997500799178163--8361667013234308244
2013-09-12 10:26:27,156 WARN Topology [RPC-6]: FileSever on box0.hadoop reported an invalid topology . Ignoring reported topology
2013-09-12 10:26:27,175 INFO CLDBServer [RPC-6]: FSRegister: Registered FileServer: 192.168.16.100- at topology /default-rack/box0.hadoop
2013-09-12 10:26:27,283 INFO ZooKeeperClient [RPC-9]: Storing KvStoreContainerInfo to ZooKeeper  Container ID:1 Servers:  Inactive:  192.168.16.101-11(3756088561801188852) 192.168.16.102-11(568571831440474766) 192.168.16.100-11(3671575442874589802) Unused:  Epoch:11 SizeMB:0
2013-09-12 10:26:27,310 INFO Containers [RPC-9]: No master for kvstore, waiting for local kvstore to become master
2013-09-12 10:26:27,392 INFO ZooKeeperClient [RPC-10]: Storing KvStoreContainerInfo to ZooKeeper  Container ID:1 Master:192.168.16.100-11-BM(3671575442874589802) Servers:  192.168.16.100-11-BM(3671575442874589802) Inactive:  192.168.16.101-11(3756088561801188852) 192.168.16.102-11(568571831440474766) Unused:  Epoch:11 SizeMB:0
2013-09-12 10:26:27,582 INFO ZooKeeperClient [RPC-3]: Storing KvStoreContainerInfo to ZooKeeper  Container ID:1 Master:192.168.16.100-11(3671575442874589802) Servers:  192.168.16.100-11(3671575442874589802) Inactive:  192.168.16.101-11(3756088561801188852) 192.168.16.102-11(568571831440474766) Unused:  Epoch:11 SizeMB:0
2013-09-12 10:26:27,588 INFO CLDBServer [RPC-3]: CLDB Mode: MASTER_REGISTER_READY CLDB HA Check
2013-09-12 10:26:27,599 INFO LicenseManager [RPC-3]: using default license validator
2013-09-12 10:26:27,684 ERROR LicenseManager [RPC-3]: could not determine the unique id of the node
2013-09-12 10:26:27,688 INFO LicenseManager [RPC-3]: 0x25: unique id: 00259089480c--0003670017-0006291457-0005242882
2013-09-12 10:26:27,688 FATAL LicenseManager [RPC-3]: CLDB HA check failed: not licensed, failover denied: elapsed time since last failure=41 minutes
2013-09-12 10:26:27,688 INFO CLDBServer [RPC-3]: CLDB HA check returned: false
2013-09-12 10:26:27,688 FATAL CLDB [RPC-3]: CLDBShutdown: HA Check failed: Shutting down CLDB
2013-09-12 10:26:27,688 INFO CLDBServer [RPC-3]: Shutdown: Stopping CLDB
2013-09-12 10:26:27,689 INFO CLDB [Thread-11]: CLDB ShutDown Hook called
2013-09-12 10:26:27,690 INFO ZooKeeperClient [Thread-11]: Setting the clean cldbshutdown flag to true
2013-09-12 10:26:27,697 INFO ZooKeeperClient [Thread-11]: Zookeeper Client: Closing client connection:
2013-09-12 10:26:27,702 INFO CLDBServer [main-EventThread]: The CLDB received notification that a ZooKeeper event of type NodeDeleted occurred on path /datacenter/controlnodes/cldb/active/CLDBMaster
2013-09-12 10:26:27,703 INFO ZooKeeper [Thread-11]: Session: 0x141128d8e900003 closed
2013-09-12 10:26:27,703 INFO CLDB [Thread-11]: CLDB shutdown

Outcomes