AnsweredAssumed Answered

Sqoop / Hue / Oozie / Hive failure

Question asked by ronaldm on Jan 22, 2018
Latest reply on Feb 2, 2018 by Murshid Chalaev

I am trying to run a sqoop job thru Hue/Oozie where it is importing data from MySQL to Hive.

Running the sqoop command via command line works fine. When run via Hue/Oozie, it fails...

Saw these:


2018-01-22 12:21:20,074 [uber-SubtaskRunner] INFO  org.apache.hadoop.hive.conf.HiveConf  
- Found configuration file
file:/tmp/hadoop-mapr/nm-local-dir/usercache/mapr/appcache/application_1516614625728_0011/container_e07_1516614625728_0011_01_000001/hive-site.xml 2018-01-22 12:21:21,334 [uber-SubtaskRunner] INFO  CliDriver  -  Logging initialized using configuration in
jar:file:/tmp/hadoop-mapr/nm-local-dir/filecache/74/hive-exec-2.1.1-mapr-1710.jar!/hive-log4j2.properties Async: true 2018-01-22 12:21:21,371 [uber-SubtaskRunner] INFO  hive.metastore  -
Trying to connect to metastore with URI
thrift://ip-172-31-19-232.ap-southeast-1.compute.internal:9083 2018-01-22 12:21:21,434 [uber-SubtaskRunner] INFO 
org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge25Sasl  -
Hive is configured for user authentication:  2018-01-22 12:21:21,434 [uber-SubtaskRunner] INFO 
org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge25Sasl  -
User authentication with method: CUSTOM 2018-01-22 12:21:21,434 [uber-SubtaskRunner] WARN 
org.apache.hadoop.security.rpcauth.RpcAuthRegistry  -
No RpcAuthMethod registerd for authentication method CUSTOM 2018-01-22 12:21:21,437 [uber-SubtaskRunner] WARN 
hive.ql.metadata.Hive  - Failed to register all functions. java.lang.RuntimeException:
Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient

When the sqoop job was modified to skip the Hive import, it works on Hue/Oozie.

I added hive-site.xml to the job. I may have to add something else.

 

I am running MapR 6.0 and MEP 4.0.

This is an unsecure cluster.

Cluster was installed using MapR GUI installer. No changes made to configuration.

Unsure why user authentication method became CUSTOM when hive-site.xml did not have any configuration set for this - unless the default became CUSTOM.

Outcomes