AnsweredAssumed Answered

Integrating Mapr M7 table with Hive on Amazon EMR - MasterNotRunningException

Question asked by elleg on Jun 30, 2014
Latest reply on Jul 1, 2014 by elleg
Hello,

My m7 cluster was created on Amazon EMR (for testing purposes). I created a mapr table successfully, and now want to integrate it with Hive to allow for querying.

On the hive shell, I ran:

    CREATE EXTERNAL TABLE mapr_hive_test(key int, `all` string, `text` string, `date` bigint)
    STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
    WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,all:all,text:text,dt:dt")
    TBLPROPERTIES("hbase.table.name" = "/mapr/MapR_EMR.amazonaws.com/data/test");

I got the following exception:

    FAILED: Error in metadata: MetaException(message:org.apache.hadoop.hbase.MasterNotRunningException: Retried 10 times
at org.apache.hadoop.hbase.client.HBaseAdmin.<init>(HBaseAdmin.java:127)
at org.apache.hadoop.hive.hbase.HBaseStorageHandler.getHBaseAdmin(HBaseStorageHandler.java:74)
at org.apache.hadoop.hive.hbase.HBaseStorageHandler.preCreateTable(HBaseStorageHandler.java:148)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:467)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:460)

I read a bunch of similar discussions on the error, and tried to fix it by supplying the environment vars defined in hive-site.xml. But that doesn't seem to help.

      env HIVE_OPTS="-hiveconf hive.aux.jars.path=file:///opt/mapr/hive/hive-0.11.0/lib/hive-hbase-handler-0.11.0-mapr.jar,file:///opt/mapr/hbase/hbase-0.94.17/hbase-0.94.17-mapr-1405.jar,file:///opt/mapr/zookeeper/zookeeper-3.3.6/zookeeper-3.3.6.jar -hiveconf hbase.zookeeper.quorum=ip-10-72-222

I chose the default M7 installation while launching my Amazon cluster, and I also selected the Hive installation from there. Did I miss a step where I need to get the correct hive installation?

Here are the arguments from AWS for the Hive install step:

     JAR location: s3://elasticmapreduce/libs/script-runner/script-runner.jar
     Main class: None
     Arguments: s3://elasticmapreduce/libs/hive/hive-script --base-path s3://elasticmapreduce/libs/hive/ --install-hive --hive-versions 0.11.0.1

jps:

     hadoop@ip-10-72-222-97:~/bin$ jps
     5402 QuorumPeerMain
     24912 Jps
     3005 RolesController
     5590 WardenMain
     4884 CommandServer
     6452 CLDB
     14073 RunJar
     2140 Main
     12651 JobTracker
     8091 CommandServer


Thanks!

Outcomes