AnsweredAssumed Answered

Hive error: MasterNotRunningException

Question asked by preston on Dec 15, 2013
Latest reply on Dec 15, 2013 by preston
I am having trouble getting Hive to connect to HBase on EMR.

First I launched a MapR M7 cluster on Amazon EMR through the management console. It started fine.

    Creation date: 2013-12-14 01:02:23 (local time - UTC+9)
    AMI version: 2.4.2
    Hadoop distribution: mapr-m7
    Applications: Hive 0.11.0.1, Pig 0.11.1.1

Then I SSHed in and started the HBase Thrift service, because it doesn't seem to be running out of the box. (Is there a better way to do this?)

    /opt/mapr/hbase/hbase-0.94.12/bin/hbase-daemon.sh start thrift

My app connects to Thrift and works fine (after opening port 9090). It is able to create tables and read/write data. Similarly, 'hbase shell' works fine. But 'hive' is having trouble.

    $ hive

    Logging initialized using configuration in file:/home/hadoop/.versions/hive-0.11.0/conf/hive-log4j.properties
    Hive history file=[redacted]
    hive> CREATE EXTERNAL TABLE IF NOT EXISTS hbase_events
        >   (rowkey STRING, data MAP<STRING,STRING>)
        > STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
        > WITH SERDEPROPERTIES ('hbase.columns.mapping' = ':key,data:')
        > TBLPROPERTIES ('hbase.table.name' = 'events');

Here it hangs indefinitely, so I Ctrl+C.

    FAILED: Error in metadata: MetaException(message:org.apache.hadoop.hbase.MasterNotRunningException: Interrupted
     at org.apache.hadoop.hbase.client.HBaseAdmin.<init>(HBaseAdmin.java:121)
     at org.apache.hadoop.hive.hbase.HBaseStorageHandler.getHBaseAdmin(HBaseStorageHandler.java:74)
     at org.apache.hadoop.hive.hbase.HBaseStorageHandler.preCreateTable(HBaseStorageHandler.java:148)
     at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:467)
     at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:460)
     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
     at java.lang.reflect.Method.invoke(Method.java:606)
     at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:74)
     at com.sun.proxy.$Proxy10.createTable(Unknown Source)
     at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:600)
     at org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:3791)
     at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:258)
     at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:144)
     at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
     at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1355)
     at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1139)
     at org.apache.hadoop.hive.ql.Driver.run(Driver.java:945)
     at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:310)
     at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:231)
     at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:466)
     at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:819)
     at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:674)
     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
     at java.lang.reflect.Method.invoke(Method.java:606)
     at org.apache.hadoop.util.RunJar.main(RunJar.java:197)
    )
    FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask

Is the master process really not running?

    $ jps
    6194 CLDB
    2987 RolesController
    26834 Jps
    12603 ThriftServer
    7608 CommandServer
    12191 JobTracker
    5237 QuorumPeerMain
    2121 Main
    5430 WardenMain
    13546 RunJar

Indeed. The [MapR documentation][1] suggests that there should be an 'HMaster' in that list. I tried starting it.

    $ maprcli node services -hbmaster start -nodes [redacted]
    ERROR (10002) -  Service: hbmaster is not configured on node: [redacted]

I'm not really sure what I'm doing at this point.

  [1]: http://doc.mapr.com/display/MapR/Integrating+Hive+and+MapR+Tables

Outcomes