AnsweredAssumed Answered

Error while executing /opt/mapr/spark/spark-1.2.1/bin/run-example org.apache.spark.examples.sql.hive.HiveFromSpark

Question asked by sangeetha on Mar 30, 2016
Latest reply on Jun 6, 2016 by pcurtis

while executing /opt/mapr/spark/spark-1.2.1/bin/run-example org.apache.spark.examples.sql.hive.HiveFromSpark

 

Spark assembly has been built with Hive, including Datanucleus jars on classpath

Picked up _JAVA_OPTIONS: -Djava.io.tmpdir=/opt/mapr/tmp/spark-tmp

Picked up _JAVA_OPTIONS: -Djava.io.tmpdir=/opt/mapr/tmp/spark-tmp

SLF4J: Class path contains multiple SLF4J bindings.

SLF4J: Found binding in [jar:file:/opt/mapr/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]

SLF4J: Found binding in [jar:file:/opt/mapr/spark/spark-1.2.1/lib/spark-assembly-1.2.1-hadoop2.5.1-mapr-1501.jar!/org/slf4j/impl/StaticLoggerBinder.class]

SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.

SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]

16/03/30 02:42:19 WARN Utils: Service 'SparkUI' could not bind on port 4040. Attempting port 4041.

16/03/30 02:42:25 ERROR Driver: Authorization failed:java.security.AccessControlException: action WRITE not permitted on path maprfs:/user for user hdpibdq. Use SHOW GRANT to get more details.

16/03/30 02:42:25 ERROR HiveContext:

======================

HIVE FAILURE OUTPUT

======================

Authorization failed:java.security.AccessControlException: action WRITE not permitted on path maprfs:/user for user hdpibdq. Use SHOW GRANT to get more details.

 

======================

END HIVE FAILURE OUTPUT

======================

 

Exception in thread "main" org.apache.spark.sql.execution.QueryExecutionException: java.security.AccessControlException: action WRITE not permitted on path maprfs:/user for user hdpibdq

        at org.apache.spark.sql.hive.HiveContext.runHive(HiveContext.scala:309)

        at org.apache.spark.sql.hive.HiveContext.runSqlHive(HiveContext.scala:276)

        at org.apache.spark.sql.hive.execution.NativeCommand.sideEffectResult$lzycompute(NativeCommand.scala:35)

        at org.apache.spark.sql.hive.execution.NativeCommand.sideEffectResult(NativeCommand.scala:35)

        at org.apache.spark.sql.execution.Command$class.execute(commands.scala:46)

        at org.apache.spark.sql.hive.execution.NativeCommand.execute(NativeCommand.scala:30)

        at org.apache.spark.sql.SQLContext$QueryExecution.toRdd$lzycompute(SQLContext.scala:425)

        at org.apache.spark.sql.SQLContext$QueryExecution.toRdd(SQLContext.scala:425)

        at org.apache.spark.sql.SchemaRDDLike$class.$init$(SchemaRDDLike.scala:58)

        at org.apache.spark.sql.SchemaRDD.<init>(SchemaRDD.scala:108)

        at org.apache.spark.sql.hive.HiveContext.sql(HiveContext.scala:94)

        at org.apache.spark.examples.sql.hive.HiveFromSpark$.main(HiveFromSpark.scala:48)

        at org.apache.spark.examples.sql.hive.HiveFromSpark.main(HiveFromSpark.scala)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:606)

        at org.apache.spark.deploy.SparkSubmit$.launch(SparkSubmit.scala:358)

        at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:75)

        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)

SLF4J: Class path contains multiple SLF4J bindings.

SLF4J: Found binding in [jar:file:/opt/mapr/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]

SLF4J: Found binding in [jar:file:/opt/mapr/spark/spark-1.2.1/lib/spark-assembly-1.2.1-hadoop2.5.1-mapr-1501.jar!/org/slf4j/impl/StaticLoggerBinder.class]

SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.

SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]

 

 

Please advise.

Outcomes