AnsweredAssumed Answered

Spark, Yarn, Oozie: Driver memory exception

Question asked by dodoman on Jun 3, 2016
Latest reply on Aug 2, 2016 by Karen Whipple

Hello together,

 

 

I am setting up a new mapr cluster. It is the first time I ran in the following issue which is not described anywhere and I am wondering if this is a bug or a missing configuration parameter which is not documented in mapr.

I installed a mapr 5.1 cluster with spark 1.5.2 , oozie 4.2 , hive 1.2 and yarn 2.7.0.

 

When I run spark jobs by commandline (spark-submit ) they work fine. When I run these jobs by oozie it ends up in:

In Oozie:

 

2016-06-02 20:00:03,105  WARN SparkActionExecutor:523 - SERVER[machine04] USER[mapr] GROUP[-] TOKEN[] APP[Spark] JOB[0000013-160602182858301-oozie-mapr-W] ACTION[0000013-160602182858301-oozie-mapr-W@spark-d909] Launcher exception: org.apache.spark.util.Utils$.DEFAULT_DRIVER_MEM_MB()I
java.lang.NoSuchMethodError: org.apache.spark.util.Utils$.DEFAULT_DRIVER_MEM_MB()I
        at org.apache.spark.deploy.yarn.ClientArguments.<init>(ClientArguments.scala:49)
        at org.apache.spark.scheduler.cluster.YarnClientSchedulerBackend.start(YarnClientSchedulerBackend.scala:53)
        at org.apache.spark.scheduler.TaskSchedulerImpl.start(TaskSchedulerImpl.scala:141)
        at org.apache.spark.SparkContext.<init>(SparkContext.scala:497)
        at org.apache.spark.api.java.JavaSparkContext.<init>(JavaSparkContext.scala:61)
        at org.apache.oozie.example.SparkFileCopy.main(SparkFileCopy.java:35)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)

 

It is that interesting because I don't get this error on each machine. I just get it on 2 of my 5 machines in the cluster. So i can never be sure if my jobs are running succesfully or not.

 

After following the stacktrace the only explanation are missing libraries because the value is set in the network package of spark.

 

Can someone provide me a list of the spark dependencies in the sharelib, the spark lib folder and the dependencies in the oozie share1 and 2 folder? This would be very nice.

Outcomes