AnsweredAssumed Answered

Exception when using sqoop to hive via oozie

Question asked by thomaslarsson on Apr 22, 2014
Latest reply on Apr 23, 2014 by thomaslarsson
Hello.

I am evaluating MapR as an alternative to Cloudera and am now running an oozie workflow which contains a sqoop action that should create and populate a hive table.
When running this, I see the following exception which I don't know how to proceed with. I assume there is some misconfiguration from my part. This flow is working on our cloudera cluster although I am not sure whether some special configuration was done on that cluster (the flow was created before I started). The stacktrace below seems to be related to the MaprfsLogAppender which I can't look at since its proprietary.

The mapr version is 3.1.0.23703.GA.

Log output
----------
<pre>
2014-04-23 10:32:53,452 DEBUG org.apache.sqoop.hive.TableDefWriter: Create statement: CREATE TABLE IF NOT EXISTS `thedatabase.dummy` ( `col1` BIGINT, `col2` STRING) COMMENT 'Imported by sqoop on 2014/04/23 10:32:53' ROW FORMAT DELIMITED FIELDS TERMINATED BY '\001' LINES TERMINATED BY '\012' STORED AS TEXTFILE
2014-04-23 10:32:53,452 DEBUG org.apache.sqoop.hive.TableDefWriter: Load statement: LOAD DATA INPATH 'maprfs:/tmp/theuser/sqoop/dummy' INTO TABLE `thedatabase.dummy`
2014-04-23 10:32:53,453 INFO org.apache.sqoop.hive.HiveImport: Removing temporary files from import process: maprfs:/tmp/theuser/sqoop/dummy/_logs
2014-04-23 10:32:53,460 INFO org.apache.sqoop.hive.HiveImport: Loading uploaded data into Hive
2014-04-23 10:32:53,461 DEBUG org.apache.sqoop.hive.HiveImport: Using external Hive process.

2014-04-23 10:32:54,134 INFO org.apache.sqoop.hive.HiveImport: Exception in thread "main" java.lang.ExceptionInInitializerError
2014-04-23 10:32:54,135 INFO org.apache.sqoop.hive.HiveImport:  at org.apache.hadoop.util.RunJar.main(RunJar.java:143)
2014-04-23 10:32:54,135 INFO org.apache.sqoop.hive.HiveImport: Caused by: org.apache.commons.logging.LogConfigurationException: java.lang.ExceptionInInitializerError

(Caused by java.lang.ExceptionInInitializerError)
2014-04-23 10:32:54,135 INFO org.apache.sqoop.hive.HiveImport:  at org.apache.commons.logging.impl.LogFactoryImpl.newInstance(LogFactoryImpl.java:538)
2014-04-23 10:32:54,135 INFO org.apache.sqoop.hive.HiveImport:  at org.apache.commons.logging.impl.LogFactoryImpl.getInstance(LogFactoryImpl.java:235)
2014-04-23 10:32:54,135 INFO org.apache.sqoop.hive.HiveImport:  at org.apache.commons.logging.impl.LogFactoryImpl.getInstance(LogFactoryImpl.java:209)
2014-04-23 10:32:54,135 INFO org.apache.sqoop.hive.HiveImport:  at org.apache.commons.logging.LogFactory.getLog(LogFactory.java:351)
2014-04-23 10:32:54,135 INFO org.apache.sqoop.hive.HiveImport:  at org.apache.hadoop.conf.Configuration.<clinit>(Configuration.java:152)
2014-04-23 10:32:54,135 INFO org.apache.sqoop.hive.HiveImport:  ... 1 more
2014-04-23 10:32:54,135 INFO org.apache.sqoop.hive.HiveImport:

Caused by: java.lang.ExceptionInInitializerError
2014-04-23 10:32:54,135 INFO org.apache.sqoop.hive.HiveImport:  at com.mapr.log4j.MaprfsLogAppender.<clinit>(MaprfsLogAppender.java:121)
2014-04-23 10:32:54,135 INFO org.apache.sqoop.hive.HiveImport:  at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
2014-04-23 10:32:54,135 INFO org.apache.sqoop.hive.HiveImport:  at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
2014-04-23 10:32:54,135 INFO org.apache.sqoop.hive.HiveImport:  at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
2014-04-23 10:32:54,135 INFO org.apache.sqoop.hive.HiveImport:  at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
2014-04-23 10:32:54,135 INFO org.apache.sqoop.hive.HiveImport:  at java.lang.Class.newInstance(Class.java:374)
2014-04-23 10:32:54,135 INFO org.apache.sqoop.hive.HiveImport:  at org.apache.log4j.helpers.OptionConverter.instantiateByClassName(OptionConverter.java:330)
2014-04-23 10:32:54,136 INFO org.apache.sqoop.hive.HiveImport:  at org.apache.log4j.helpers.OptionConverter.instantiateByKey(OptionConverter.java:121)
2014-04-23 10:32:54,136 INFO org.apache.sqoop.hive.HiveImport:  at org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:664)
2014-04-23 10:32:54,136 INFO org.apache.sqoop.hive.HiveImport:  at org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:647)
2014-04-23 10:32:54,136 INFO org.apache.sqoop.hive.HiveImport:  at org.apache.log4j.PropertyConfigurator.configureRootCategory(PropertyConfigurator.java:544)
2014-04-23 10:32:54,136 INFO org.apache.sqoop.hive.HiveImport:  at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:440)
2014-04-23 10:32:54,136 INFO org.apache.sqoop.hive.HiveImport:  at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:476)
2014-04-23 10:32:54,136 INFO org.apache.sqoop.hive.HiveImport:  at org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConverter.java:471)
2014-04-23 10:32:54,136 INFO org.apache.sqoop.hive.HiveImport:  at org.apache.log4j.LogManager.<clinit>(LogManager.java:125)
2014-04-23 10:32:54,136 INFO org.apache.sqoop.hive.HiveImport:  at org.apache.log4j.Logger.getLogger(Logger.java:105)
2014-04-23 10:32:54,136 INFO org.apache.sqoop.hive.HiveImport:  at org.apache.commons.logging.impl.Log4JLogger.getLogger(Log4JLogger.java:229)
2014-04-23 10:32:54,136 INFO org.apache.sqoop.hive.HiveImport:  at org.apache.commons.logging.impl.Log4JLogger.<init>(Log4JLogger.java:65)
2014-04-23 10:32:54,136 INFO org.apache.sqoop.hive.HiveImport:  at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
2014-04-23 10:32:54,136 INFO org.apache.sqoop.hive.HiveImport:  at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
2014-04-23 10:32:54,136 INFO org.apache.sqoop.hive.HiveImport:  at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
2014-04-23 10:32:54,136 INFO org.apache.sqoop.hive.HiveImport:  at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
2014-04-23 10:32:54,136 INFO org.apache.sqoop.hive.HiveImport:  at org.apache.commons.logging.impl.LogFactoryImpl.newInstance(LogFactoryImpl.java:529)
2014-04-23 10:32:54,136 INFO org.apache.sqoop.hive.HiveImport:  ... 5 more
2014-04-23 10:32:54,136 INFO org.apache.sqoop.hive.HiveImport:

Caused by: java.lang.NullPointerException
2014-04-23 10:32:54,136 INFO org.apache.sqoop.hive.HiveImport:  at org.apache.hadoop.conf.Configuration.addDefaultResource(Configuration.java:365)
2014-04-23 10:32:54,136 INFO org.apache.sqoop.hive.HiveImport:  at org.apache.hadoop.conf.Configuration.addDefaultResource(Configuration.java:348)
2014-04-23 10:32:54,136 INFO org.apache.sqoop.hive.HiveImport:  at org.apache.hadoop.mapred.TaskTracker.<clinit>(TaskTracker.java:202)
2014-04-23 10:32:54,136 INFO org.apache.sqoop.hive.HiveImport:  ... 28 more

----------

Outcomes