AnsweredAssumed Answered

Currrent CLDB's KvStore Epoch is not latest

Question asked by anirudh on Feb 9, 2014
Latest reply on Jun 8, 2014 by nabeel
Hi There,

I have a fresh 6 Virtual Machine Node cluster with following config

Machines: SAW00,SAW01,SAW02,SAW03,SAW04,SAW05

IPAdress: XX.XX.2XX.80 - XX.XX.2XX.85



FileServer: ALL

Jobtracker: SAW00,SAW01

TaskTracker: ALL

License: M5

The exact steps I followed to install were:

 1. Disable SELINUX
 2. Installed MapR Components
 3. Stopped ZK and WARDEN on each node
 4. Ran -C SAW00,SAW01 -Z SAW00,SAW01,SAW02,SAW03,SAW04,SAW05 -N SAW --isvm
 5. Started ZK
 6. Started Warden
 7. Added a single disk '/dev/sdb/' using 'disksetup -F /disk' (/disk contains /dev/sdb)
 8. Monitored the cldb log file
 9. First I got the message on one of theCLDB as :
 CLDBShutdown: CLDB had master lock and was waiting for its local mfs to become Master.Waited for 7 (minutes) but mfs did not become Master. Shutting down CLDB to release master lock.
2014-02-10 01:25:56,683 INFO CLDBServer [WaitForLocalKvstore Thread]: Shutdown: Stopping CLDB

 10. After this shutdown the next message I am getting from CLDB is

2014-02-10 01:26:47,494 INFO ZooKeeperClient [ZK-Connect]: ZooKeeperClient: KvStore is of latest epoch CLDB trying to become Master
2014-02-10 01:26:47,497 INFO ZooKeeperClient [ZK-Connect]: ZooKeeperClient: CLDB is current Master
2014-02-10 01:26:47,497 INFO ZooKeeperClient [ZK-Connect]: CLDB became master. Initializing KvStoreContainer for cid: 1
2014-02-10 01:26:47,498 FATAL CLDB [ZK-Connect]: CLDBShutdown: Currrent CLDB's KvStore Epoch is not latest. KvStore epoch: 2 latest epoch: 3 Shutting down CLDB

I came across a post which mentioned about a failover for CLDB. But what I cannot make out of it is that the cluster is new, so no failover should ocuur and even after a night I still see that KvStore is still not the latest.

Is there a walkthrough which tells how to reset you cluster with zero data ?

I cannot make any sense out of it what is happening here. Is there a way to force this KvStore epoch? I have already spent a day in searching for related issues; no luck yet.

Any help is appreciated!