AnsweredAssumed Answered

Zookeeper qstatus throws "Error contacting service. It is probably not running"

Question asked by sandy99 on Aug 11, 2013
Latest reply on Aug 11, 2013 by yufeldman
Hi,
We started zk on 3 node cluster on VMs. started zk on all the three VMs. However, zookeeper qstatus command throws the following error.
**service mapr-zookeeper qstatus
JMX enabled by default
Using config: /opt/mapr/zookeeper/zookeeper-3.3.6/conf/zoo.cfg
Error contacting service. It is probably not running.**

1.verified that i m able to telnet from each server to others on port 3888
2.Zookeeper status command shows that zk is running on all 3 servers


Please find the log & also zoo.cfg. zoo.cfg is exactly same on all 3 servers.

<code>
zoo.cfg:[3-Nodes]
[root@Master conf]# cat zoo.cfg
# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial
# synchronization phase can take
initLimit=20
# The number of ticks that can pass between
# sending a request and getting an acknowledgement
syncLimit=10
# the directory where the snapshot is stored.
dataDir=/opt/mapr/zkdata
# the port at which the clients will connect
clientPort=5181
# max number of client connections
maxClientCnxns=100
server.0=Master.localdomain:2888:3888
server.1=Slave1.localdomain:2888:3888
server.2=Slave2.localdomain:2888:3888


zookeeper.log :
[root@Slave2 logs]# service mapr-zookeeper qstatus
JMX enabled by default
Using config: /opt/mapr/zookeeper/zookeeper-3.3.6/conf/zoo.cfg
Error contacting service. It is probably not running.
[root@Slave2 logs]# more zookeeper.log
2013-08-11 15:26:30,046 - INFO  [main:QuorumPeerConfig@90] - Reading configuration from: /opt/mapr/zookeeper/zookeeper-3.3.6/conf/zoo.cfg
2013-08-11 15:26:30,051 - INFO  [main:QuorumPeerConfig@310] - Defaulting to majority quorums
2013-08-11 15:26:30,069 - INFO  [main:QuorumPeerMain@119] - Starting quorum peer
2013-08-11 15:26:30,112 - INFO  [main:NIOServerCnxn$Factory@143] - binding to port 0.0.0.0/0.0.0.0:5181
2013-08-11 15:26:30,151 - INFO  [main:QuorumPeer@819] - tickTime set to 2000
2013-08-11 15:26:30,152 - INFO  [main:QuorumPeer@830] - minSessionTimeout set to -1
2013-08-11 15:26:30,153 - INFO  [main:QuorumPeer@841] - maxSessionTimeout set to -1
2013-08-11 15:26:30,154 - INFO  [main:QuorumPeer@856] - initLimit set to 20
2013-08-11 15:26:30,197 - INFO  [main:FileSnap@82] - Reading snapshot /opt/mapr/zkdata/version-2/snapshot.300000000
2013-08-11 15:26:30,213 - INFO  [Thread-1:QuorumCnxManager$Listener@473] - My election bind port: 3888
2013-08-11 15:26:30,242 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:5181:QuorumPeer@621] - LOOKING
2013-08-11 15:26:30,244 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:5181:FastLeaderElection@663] - New election. My id =  2, Proposed zxid = 12884901888
2013-08-11 15:26:30,272 - INFO  [WorkerReceiver Thread:FastLeaderElection@496] - Notification: 2 (n.leader), 12884901888 (n.zxid), 4 (n.round), LOOKING (n.state), 0 (n.
sid), LOOKING (my state)
2013-08-11 15:26:30,274 - INFO  [WorkerReceiver Thread:FastLeaderElection@496] - Notification: 2 (n.leader), 12884901888 (n.zxid), 1 (n.round), LOOKING (n.state), 2 (n.
sid), LOOKING (my state)
2013-08-11 15:26:30,277 - INFO  [WorkerReceiver Thread:FastLeaderElection@496] - Notification: 2 (n.leader), 12884901888 (n.zxid), 4 (n.round), LOOKING (n.state), 2 (n.
sid), LOOKING (my state)
2013-08-11 15:26:30,279 - INFO  [WorkerReceiver Thread:FastLeaderElection@496] - Notification: 0 (n.leader), 17179869184 (n.zxid), 5 (n.round), LOOKING (n.state), 0 (n.
sid), LOOKING (my state)
2013-08-11 15:26:30,280 - INFO  [WorkerReceiver Thread:FastLeaderElection@496] - Notification: 0 (n.leader), 17179869184 (n.zxid), 5 (n.round), LOOKING (n.state), 2 (n.
sid), LOOKING (my state)
2013-08-11 15:26:30,282 - INFO  [WorkerReceiver Thread:FastLeaderElection@496] - Notification: 1 (n.leader), 17179869184 (n.zxid), 5 (n.round), LOOKING (n.state), 0 (n.
sid), LOOKING (my state)
2013-08-11 15:26:30,283 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:5181:FastLeaderElection@721] - Updating proposal
2013-08-11 15:26:30,284 - INFO  [WorkerReceiver Thread:FastLeaderElection@496] - Notification: 1 (n.leader), 17179869184 (n.zxid), 5 (n.round), LOOKING (n.state), 2 (n.
sid), LOOKING (my state)
2013-08-11 15:26:30,291 - INFO  [WorkerReceiver Thread:FastLeaderElection@496] - Notification: 1 (n.leader), 17179869184 (n.zxid), 5 (n.round), FOLLOWING (n.state), 0 (
n.sid), LOOKING (my state)
2013-08-11 15:26:30,292 - INFO  [WorkerReceiver Thread:FastLeaderElection@496] - Notification: 1 (n.leader), 17179869184 (n.zxid), 5 (n.round), FOLLOWING (n.state), 0 (
n.sid), LOOKING (my state)
2013-08-11 15:26:30,301 - INFO  [WorkerReceiver Thread:FastLeaderElection@496] - Notification: 1 (n.leader), 17179869184 (n.zxid), 5 (n.round), FOLLOWING (n.state), 0 (
n.sid), LOOKING (my state)
2013-08-11 15:26:30,302 - INFO  [WorkerReceiver Thread:FastLeaderElection@496] - Notification: 1 (n.leader), 17179869184 (n.zxid), 5 (n.round), FOLLOWING (n.state), 0 (
n.sid), LOOKING (my state)
2013-08-11 15:26:30,304 - INFO  [WorkerReceiver Thread:FastLeaderElection@496] - Notification: 2 (n.leader), 12884901888 (n.zxid), 4 (n.round), LOOKING (n.state), 1 (n.
sid), LOOKING (my state)
2013-08-11 15:26:30,306 - INFO  [WorkerReceiver Thread:FastLeaderElection@496] - Notification: 1 (n.leader), 17179869184 (n.zxid), 5 (n.round), LOOKING (n.state), 1 (n.
sid), LOOKING (my state)
2013-08-11 15:26:30,307 - INFO  [WorkerReceiver Thread:FastLeaderElection@496] - Notification: 1 (n.leader), 17179869184 (n.zxid), 5 (n.round), LEADING (n.state), 1 (n.
sid), LOOKING (my state)
2013-08-11 15:26:30,317 - INFO  [WorkerReceiver Thread:FastLeaderElection@496] - Notification: 1 (n.leader), 17179869184 (n.zxid), 5 (n.round), LEADING (n.state), 1 (n.
sid), LOOKING (my state)
2013-08-11 15:26:30,318 - INFO  [WorkerReceiver Thread:FastLeaderElection@496] - Notification: 1 (n.leader), 17179869184 (n.zxid), 5 (n.round), LEADING (n.state), 1 (n.
sid), LOOKING (my state)
2013-08-11 15:26:30,319 - INFO  [WorkerReceiver Thread:FastLeaderElection@496] - Notification: 1 (n.leader), 17179869184 (n.zxid), 5 (n.round), LEADING (n.state), 1 (n.
sid), LOOKING (my state)
2013-08-11 15:26:30,320 - INFO  [WorkerReceiver Thread:FastLeaderElection@496] - Notification: 1 (n.leader), 17179869184 (n.zxid), 5 (n.round), LEADING (n.state), 1 (n.
sid), LOOKING (my state)
2013-08-11 15:26:30,522 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:5181:QuorumPeer@643] - FOLLOWING
2013-08-11 15:26:30,528 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:5181:Learner@80] - TCP NoDelay set to: true
2013-08-11 15:26:30,536 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:5181:Environment@97] - Server environment:zookeeper.version=3.3.6--1, built on 09/07/2012 18:16 GMT
2013-08-11 15:26:30,536 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:5181:Environment@97] - Server environment:host.name=Slave2.localdomain
2013-08-11 15:26:30,537 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:5181:Environment@97] - Server environment:java.version=1.7.0_25
2013-08-11 15:26:30,537 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:5181:Environment@97] - Server environment:java.vendor=Oracle Corporation
2013-08-11 15:26:30,538 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:5181:Environment@97] - Server environment:java.home=/usr/java/jdk1.7.0_25/jre
2013-08-11 15:26:30,538 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:5181:Environment@97] - Server environment:java.class.path=/opt/mapr/zookeeper/zookeeper-3.3.6/bin/../build/
classes:/opt/mapr/zookeeper/zookeeper-3.3.6/bin/../build/lib/*.jar:/opt/mapr/zookeeper/zookeeper-3.3.6/bin/../zookeeper-3.3.6.jar:/opt/mapr/zookeeper/zookeeper-3.3.6/bi
n/../lib/log4j-1.2.15.jar:/opt/mapr/zookeeper/zookeeper-3.3.6/bin/../lib/jline-0.9.94.jar:/opt/mapr/zookeeper/zookeeper-3.3.6/bin/../src/java/lib/*.jar:/opt/mapr/zookee
per/zookeeper-3.3.6/conf:
2013-08-11 15:26:30,544 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:5181:Environment@97] - Server environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:
/lib:/usr/lib
2013-08-11 15:26:30,544 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:5181:Environment@97] - Server environment:java.io.tmpdir=/tmp
2013-08-11 15:26:30,544 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:5181:Environment@97] - Server environment:java.compiler=<NA>
2013-08-11 15:26:30,545 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:5181:Environment@97] - Server environment:os.name=Linux
2013-08-11 15:26:30,545 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:5181:Environment@97] - Server environment:os.arch=amd64
2013-08-11 15:26:30,545 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:5181:Environment@97] - Server environment:os.version=2.6.32-358.14.1.el6.x86_64
2013-08-11 15:26:30,545 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:5181:Environment@97] - Server environment:user.name=root
2013-08-11 15:26:30,546 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:5181:Environment@97] - Server environment:user.home=/root
2013-08-11 15:26:30,546 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:5181:Environment@97] - Server environment:user.dir=/
2013-08-11 15:26:30,548 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:5181:ZooKeeperServer@154] - Created server with tickTime 2000 minSessionTimeout 4000 maxSessionTimeout 4000
0 datadir /opt/mapr/zkdata/version-2 snapdir /opt/mapr/zkdata/version-2
2013-08-11 15:26:30,560 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:5181:Learner@294] - Getting a snapshot from leader
2013-08-11 15:26:30,565 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:5181:Learner@326] - Setting leader epoch 5
2013-08-11 15:26:30,566 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:5181:FileTxnSnapLog@256] - Snapshotting: 500000000
2013-08-11 15:26:41,753 - INFO  [main:FourLetterWordMain@43] - connecting to localhost 5181
</code>

Outcomes