AnsweredAssumed Answered

Exception connecting to HBase from Hive

Question asked by samudra on May 25, 2014
Latest reply on May 25, 2014 by nabeel
Hi Experts,

I have been trying to connect to HBase from Hive to try HiveQL queries. I run the following to start hive:

<pre>

./hive --auxpath $HIVE_HOME/lib/hive-hbase-handler-0.12.0.jar,$HIVE_HOME/lib/hbase-0.94.6.1.jar,$HIVE_HOME/lib/zookeeper-3.4.5.jar,$HIVE_HOME/lib/guava-11.0.2.jar -hiveconf hbase.master=cpu03.nbl.cewit.stonybrook.edu:60010 -hiveconf hbase.rootdir=hdfs://cpu02.nbl.cewit.stonybrook.edu:54137/hbase/ -hiveconf hbase.cluster.distributed=true -hiveconf zookeeper.znode.parent=/hbase -hiveconf mapred.job.tracker=cpu02.nbl.cewit.stonybrook.edu:54030 -hiveconf hbase.zookeeper.quorum=cpu11 -hiveconf hive.zookeeper.client.port=2222

</pre>

I have a zookeeper server running on "cpu11" and hbase master on "cpu03.nbl.cewit.stonybrook.edu". I am able to use hbase with the given configurations and create tables and all. However, with hive, I get the following exception:
<pre>

2014-05-25 20:57:56,205 WARN  zookeeper.ClientCnxn (ClientCnxn.java:run(1057)) - Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:601)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:286)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1035)
2014-05-25 20:57:56,328 WARN  zookeeper.RecoverableZooKeeper (RecoverableZooKeeper.java:retryOrThrow(219)) - Possibly transient ZooKeeper exception: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid
2014-05-25 20:57:57,327 WARN  zookeeper.ClientCnxn (ClientCnxn.java:run(1057)) - Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:601)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:286)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1035)
2014-05-25 20:57:58,429 WARN  zookeeper.ClientCnxn (ClientCnxn.java:run(1057)) - Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:601)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:286)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1035)
2014-05-25 20:57:58,530 WARN  zookeeper.RecoverableZooKeeper (RecoverableZooKeeper.java:retryOrThrow(219)) - Possibly transient ZooKeeper exception: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid
2014-05-25 20:57:59,531 WARN  zookeeper.ClientCnxn (ClientCnxn.java:run(1057)) - Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:601)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:286)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1035)
2014-05-25 20:58:00,633 WARN  zookeeper.ClientCnxn (ClientCnxn.java:run(1057)) - Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:601)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:286)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1035)
2014-05-25 20:58:01,735 WARN  zookeeper.ClientCnxn (ClientCnxn.java:run(1057)) - Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:601)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:286)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1035)
2014-05-25 20:58:02,837 WARN  zookeeper.ClientCnxn (ClientCnxn.java:run(1057)) - Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:601)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:286)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1035)
2014-05-25 20:58:02,938 WARN  zookeeper.RecoverableZooKeeper (RecoverableZooKeeper.java:retryOrThrow(219)) - Possibly transient ZooKeeper exception: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid
2014-05-25 20:58:03,939 WARN  zookeeper.ClientCnxn (ClientCnxn.java:run(1057)) - Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:601)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:286)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1035)
2014-05-25 20:58:04,404 WARN  zookeeper.ZKUtil (ZKUtil.java:checkExists(457)) - hconnection Unable to set watcher on znode (/hbase/hbaseid)
java.lang.InterruptedException: sleep interrupted
at java.lang.Thread.sleep(Native Method)
at java.lang.Thread.sleep(Thread.java:321)
at java.util.concurrent.TimeUnit.sleep(TimeUnit.java:356)
.........................................................
at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:781)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:675)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:614)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:622)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
2014-05-25 20:58:04,412 WARN  zookeeper.ZKUtil (ZKUtil.java:watchAndCheckExists(433)) - hconnection Unable to set watcher on znode /hbase/master
java.lang.InterruptedException
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:502)
at org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1253)
....................................................................................
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:622)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
2014-05-25 20:58:04,416 WARN  zookeeper.ZKUtil (ZKUtil.java:watchAndCheckExists(433)) - hconnection Unable to set watcher on znode /hbase/root-region-server
java.lang.InterruptedException
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:502)
....................................................................................
at java.lang.reflect.Method.invoke(Method.java:622)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
2014-05-25 20:58:04,446 WARN  zookeeper.ZKUtil (ZKUtil.java:checkExists(457)) - hconnection Unable to set watcher on znode (/hbase)
java.lang.InterruptedException
at java.lang.Object.wait(Native Method)
......................................................................................
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:622)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
2014-05-25 20:58:04,449 ERROR client.HConnectionManager$HConnectionImplementation (HConnectionManager.java:checkIfBaseNodeAvailable(750)) - Check the value configured in 'zookeeper.znode.parent'. There could be a mismatch with the one configured in the master.

</pre>

I found older posts regarding this also, but all of them seem to suggest setting the configuration properties (quorum.master, clientPort, etc) correctly, which I think are fine.

Any idea if I am missing something else here?

Thanks,
Samudra

Outcomes