AnsweredAssumed Answered

hbase -hive integration error

Question asked by hegde on Aug 9, 2013
Latest reply on Aug 12, 2013 by hegde
I was trying the sample program of HIVE-HBASE integration (on Single Node) available @ http://www.mapr.com/doc/display/MapR/Hive but always getting HBase Master not running error though it is running.  The following are the details

I'm using MapR M3,

1) Hadoop
<code>
$ hadoop version
Hadoop 1.0.3
Source http://mapr.com -r 823b1560415225caf96cb1653a06d7e64f28f25b
Compiled by root on Tue Mar  5 00:05:52 PST 2013
From source with checksum 1b1e5028df05f73f133ee59e6c34ce88
</code>

2) Hive
<code>
$ pwd
/opt/mapr/hive
$ ls -l
total 4
drwxr-xr-x 9 mapr mapr 4096 Aug  7 12:43 hive-0.11
</code>

3) Hbase
<code>
$ pwd
/opt/mapr/hbase
$ ls -l
total 8
drwxr-xr-x 10 mapr mapr 4096 Jul 25 09:08 hbase-0.94.5
-rw-r--r--  1 mapr mapr    7 Jul 25 09:08 hbaseversion
</code>

The following are the services running.
<code>
$ sudo maprcli service list
name         state  logpath                        displayname
fileserver   2      /opt/mapr/logs/mfs.log         FileServer
nfs          4      /opt/mapr/logs/nfsserver.log   NFS Gateway
webserver    2      /opt/mapr/logs/adminuiapp.log  Webserver
cldb         2      /opt/mapr/logs/cldb.log        CLDB
tasktracker  2      /opt/mapr/hadoop/logs          TaskTracker
jobtracker   2      /opt/mapr/hadoop/logs          JobTracker
hoststats    2      /opt/mapr/logs/hoststats.log   HostStats

$ ls -l /opt/mapr/roles
total 0
-rwxr-xr-x 1 mapr mapr 0 Mar  5 08:14 cldb
-rwxr-xr-x 1 mapr mapr 0 Mar  5 08:14 fileserver
-rw-r--r-- 1 mapr mapr 0 Jul 24 16:36 hbinternal
-rwxr-xr-x 1 root root 0 Jul 24 16:36 hbmaster
-rwxr-xr-x 1 root root 0 Jul 24 16:36 hbregionserver
-rwxr-xr-x 1 mapr mapr 0 Mar  5 08:14 jobtracker
-rwxr-xr-x 1 mapr mapr 0 Mar  5 08:14 nfs
-rwxr-xr-x 1 mapr mapr 0 Jul 13 00:06 oozie
-rwxr-xr-x 1 mapr mapr 0 Mar  5 08:14 tasktracker
-rwxr-xr-x 1 mapr mapr 0 Mar  5 08:14 webserver
-rwxr-xr-x 1 mapr mapr 0 Mar  5 08:14 zookeeper

$ pwd
/tmp
$ ls -l hbase*
-rw-rw-r-- 1 mapr mapr    6 Aug 10 03:32 hbase-mapr-master.pid

hbase-mapr:
total 4
drwxrwxr-x 3 mapr mapr 4096 Aug 10 03:32 zookeeper

$ cat /tmp/hbase-mapr-master.pid
15535
$ ps -ef | grep 15535
mapr      5239 14073  0 04:16 pts/1    00:00:00 grep 15535
mapr     15535     1  0 03:32 pts/1    00:00:15 /usr/lib/jvm/java-6-openjdk-amd64/jre/bin/java -XX:OnOutOfMemoryError=kill -9 %p -Xmx1000m -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:NewRatio=16 -XX:CMSInitiatingOccupancyFraction=70 -XX:+UseCMSInitiatingOccupancyOnly -XX:MaxGCPauseMillis=100 -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/opt/cores/ -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.port=10101 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.port=10101 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.port=10101 -Dhbase.log.dir=/opt/mapr/hbase/hbase-0.94.5/bin/../logs -Dhbase.log.file=hbase-mapr-master-ip-10-220-15-88.log -Dhbase.home.dir=/opt/mapr/hbase/hbase-0.94.5/bin/.. -Dhbase.id.str=mapr -Dhbase.root.logger=INFO,DRFA -Djava.library.path=/opt/mapr/hadoop/hadoop-0.20.2/lib/native/Linux-amd64-64::/opt/mapr/hbase/hbase-0.94.5/bin/../lib/native/Linux-amd64-64 -Dhbase.security.logger=INFO,DRFAS -classpath /opt/mapr/hbase/hbase-0.94.5/bin/../conf:/usr/lib/jvm/java-6-openjdk-amd64/jre/lib/tools.jar:/opt/mapr/hbase/hbase-0.94.5/bin/..:/opt/mapr/hbase/hbase-0.94.5/bin/../hbase-0.94.5-mapr.jar:/opt/mapr/hbase/hbase-0.94.5/bin/../hbase-0.94.5-mapr-tests.jar:/opt/mapr/hbase/hbase-0.94.5/bin/../lib/activation-1.1.jar:/opt/mapr/hbase/hbase-0.94.5/bin/../lib/asm-3.1.jar:/opt/mapr/hbase/hbase-0.94.5/bin/../lib/aspectjrt-1.6.5.jar:/opt/mapr/hbase/hbase-0.94.5/bin/../lib/aspectjtools-1.6.5.jar:/opt/mapr/hbase/hbase-0.94.5/bin/../lib/avro-1.5.3.jar:/opt/mapr/hbase/hbase-0.94.5/bin/../lib/avro-ipc-1.5.3.jar:/opt/mapr/hbase/hbase-0.94.5/bin/../lib/commons-beanutils-1.7.0.jar:/opt/mapr/hbase/hbase-0.94.5/bin/../lib/commons-beanutils-core-1.8.0.jar:/opt/mapr/hbase/hbase-0.94.5/bin/../lib/commons-cli-1.2.jar:/opt/mapr/hbase/hbase-0.94.5/bin/../lib/commons-codec-1.4.jar:/opt/mapr/hbase/hbase-0.94.5/bin/../lib/commons-collections-3.2.1.jar:/opt/mapr/hbase/hbase-0.94.5/bin/../lib/commons-configuration-1.6.jar:/opt/mapr/hbase/hbase-0.94.5/bin/../lib/commons-daemon-1.0.1.jar:/opt/mapr/hbase/hbase-0.94.5/bin/../lib/commons-digester-1.8.jar:/opt/mapr/hbase/hbase-0.94.5/bin/../lib/commons-el-1.0.jar:/opt/mapr/hbase/hbase-0.94.5/bin/../lib/commons-httpclient-3.1.jar:/opt/mapr/hbase/hbase-0.94.5/bin/../lib/commons-io-2.1.jar:/opt/mapr/hbase/hbase-0.94.5/bin/../lib/commons-lang-2.5.jar:/opt/mapr/hbase/hbase-0.94.5/bin/../lib/commons-logging-1.1.1.jar:/opt/mapr/hbase/hbase-0.94.5/bin/../lib/commons-logging-api-1.0.4.jar:/opt/mapr/hbase/hbase-0.94.5/bin/../lib/commons-math-2.1.jar:/opt/mapr/hbase/hbase-0.94.5/bin/../lib/commons-net-3.1.jar:/opt/mapr/hbase/hbase-0.94.5/bin/../lib/core-3.1.1.jar:/opt/mapr/hbase/hbase-0.94.5/bin/../lib/eval-0.5.jar:/opt/mapr/hbase/hbase-0.94.5/bin/../lib/guava-11.0.2.jar:/opt/mapr/hbase/hbase-0.94.5/bin/../lib/high-scale-lib-1.1.1.jar:/opt/mapr/hbase/hbase-0.94.5/bin/../lib/httpclient-4.1.2.jar:/opt/mapr/hbase/hbase-0.94.5/bin/../lib/httpcore-4.1.3.jar:/opt/mapr/hbase/hbase-0.94.5/bin/../lib/jackson-core-asl-1.8.8.jar:/opt/mapr/hbase/hbase-0.94.5/bin/../lib/jackson-jaxrs-1.8.8.jar:/opt/mapr/hbase/hbase-0.94.5/bin/../lib/jackson-mapper-asl-1.8.8.jar:/opt/mapr/hbase/hbase-0.94.5/bin/../lib/jackson-xc-1.8.8.jar:/opt/mapr/hbase/hbase-0.94.5/bin/../lib/jamon-runtime-2.3.1.jar:/opt/mapr/hbase/hbase-0.94.5/bin/../lib/jasper-compiler-5.5.23.jar:/opt/mapr/hbase/hbase-0.94.5/bin/../lib/jasper-runtime-5.5.23.jar:/opt/mapr/hbase/hbase-0.94.5/bin/../lib/jaxb-api-2.1.jar:/opt/mapr/hbase/hbase-0.94.5/bin/../lib/jaxb-impl-2.2.3-1.jar:/opt/mapr/hbase/hbase-0.94.5/bin/../lib/jersey-core-1.8.jar:/opt/mapr/hbase/hbase-0.94.5/bin/../lib/jersey-json-1.8.jar:/opt/mapr/hbase/hbase-0.94.5/bin/../lib/jersey-server-1.8.jar:/opt/mapr/hbase/hbase-0.94.5/bin/../lib/jets3t-0.6.1.jar:/opt/mapr/hbase/hbase-0.94.5/bin/../lib/jettison-1.1.jar:/opt/mapr/hbase/hbase-0.94.5/bin/../lib/jetty-6.1.26.jar:/opt/mapr/hbase/hbase-0.94.5/bin/../lib/jetty-servlet-tester-6.1.14.jar:/o
</code>

======  Actual test --------
<code>
$ hive

Logging initialized using configuration in jar:file:/opt/mapr/hive/hive-0.11/lib/hive-common-0.11-mapr.jar!/hive-log4j.properties
Hive history file=/tmp/mapr/hive_job_log_mapr_5348@ip-10-220-15-88_201308100417_40359575.txt
hive> SHOW TABLES;
OK
pokes
web_log
Time taken: 9.514 seconds, Fetched: 2 row(s)
</code>

(loaded the contents of examples/files/kv1.txt to pokes Table)


# Attempt to create HIvE-HBASE TABLE fails here
<code>
hive> CREATE TABLE hbase_table_1(key int, value string)
> STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
    > WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf1:val")
    > TBLPROPERTIES ("hbase.table.name" = "xyz");
FAILED: Error in metadata: MetaException(message:org.apache.hadoop.hbase.MasterNotRunningException: Retried 10 times
        at org.apache.hadoop.hbase.client.HBaseAdminImpl.<init>(HBaseAdminImpl.java:128)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:532)
        at org.apache.hadoop.hbase.client.GenericHFactory.getImplementorInstance(GenericHFactory.java:49)
        at org.apache.hadoop.hbase.client.HBaseAdminFactory.create(HBaseAdminFactory.java:24)
        at org.apache.hadoop.hbase.client.HBaseAdminFactory.create(HBaseAdminFactory.java:20)
        at org.apache.hadoop.hbase.client.HBaseAdmin$1.construct(HBaseAdmin.java:99)
        at org.apache.hadoop.hbase.client.HBaseAdmin.getApacheHBaseAdmin(HBaseAdmin.java:1369)
        at org.apache.hadoop.hbase.client.HBaseAdmin.getApacheHBaseAdmin(HBaseAdmin.java:1352)
        at org.apache.hadoop.hbase.client.HBaseAdmin.getAdmin(HBaseAdmin.java:1342)
        at org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:206)
        at org.apache.hadoop.hive.hbase.HBaseStorageHandler.preCreateTable(HBaseStorageHandler.java:147)
        at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:462)
        at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:455)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:616)
        at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:74)
        at sun.proxy.$Proxy10.createTable(Unknown Source)
        at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:596)
        at org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:3776)
        at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:256)
        at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:144)
        at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
        at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1355)
        at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1139)
        at org.apache.hadoop.hive.ql.Driver.run(Driver.java:945)
        at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:259)
        at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216)
        at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:413)
        at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:756)
        at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:614)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:616)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:197)
)
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
hive>
</code>
Please note that, i ensured to set the following in hive-site.xml
<code>
<property>
<name>hive.aux.jars.path</name>
<value>file:///opt/mapr/hive/hive-0.11/lib/hive-hbase-handler-0.11-mapr.jar,file:///opt/mapr/hbase/hbase-0.94.5/hbase-0.94.5-mapr.jar,file:///opt/mapr/zookeeper/zookeeper-3.3.6/zookeeper-3.3.6.jar</value>
<description>A comma separated list (with no spaces) of the jar files required for Hive-HBase integration</description>
</property>

even i tried invoking "hive using auxpath as argument thinking that AUXPATH is not getting set)

$ hive --auxpath /opt/mapr/hive/hive-0.11/lib/hive-hbase-handler-0.11-mapr.jar,/opt/mapr/hbase/hbase-0.94.5/hbase-0.94.5-mapr.jar,/opt/mapr/zookeeper/zookeeper-3.3.6/zookeeper-3.3.6.jar,/opt/mapr/hive/hive-0.11/lib/guava-11.0.2.jar,/opt/mapr/hive/hive-0.11/lib/zookeeper-3.4.3.jar,/opt/mapr/hive/hive-0.11/lib/hive-serde-0.11-mapr.jar -hiveconf hbase.master=127.0.0.1:60000  

$ java -version
java version "1.6.0_27"
OpenJDK Runtime Environment (IcedTea6 1.12.6) (6b27-1.12.6-1ubuntu0.12.04.2)
OpenJDK 64-Bit Server VM (build 20.0-b12, mixed mode)

$ ls -l /opt/mapr/roles
total 0
-rwxr-xr-x 1 mapr mapr 0 Mar  5 08:14 cldb
-rwxr-xr-x 1 mapr mapr 0 Mar  5 08:14 fileserver
-rw-r--r-- 1 mapr mapr 0 Jul 24 16:36 hbinternal
-rwxr-xr-x 1 root root 0 Jul 24 16:36 hbmaster
-rwxr-xr-x 1 root root 0 Jul 24 16:36 hbregionserver
-rwxr-xr-x 1 mapr mapr 0 Mar  5 08:14 jobtracker
-rwxr-xr-x 1 mapr mapr 0 Mar  5 08:14 nfs
-rwxr-xr-x 1 mapr mapr 0 Jul 13 00:06 oozie
-rwxr-xr-x 1 mapr mapr 0 Mar  5 08:14 tasktracker
-rwxr-xr-x 1 mapr mapr 0 Mar  5 08:14 webserver
-rwxr-xr-x 1 mapr mapr 0 Mar  5 08:14 zookeeper

$ sudo maprcli service list
name         state  logpath                        displayname
fileserver   2      /opt/mapr/logs/mfs.log         FileServer
nfs          4      /opt/mapr/logs/nfsserver.log   NFS Gateway
webserver    2      /opt/mapr/logs/adminuiapp.log  Webserver
cldb         2      /opt/mapr/logs/cldb.log        CLDB
tasktracker  2      /opt/mapr/hadoop/logs          TaskTracker
jobtracker   2      /opt/mapr/hadoop/logs          JobTracker
hoststats    2      /opt/mapr/logs/hoststats.log   HostStats
</code>

Please help to resolve this
Thanks,
Satish
(satish.hegde@wipro.com)

Outcomes