AnsweredAssumed Answered

Error while saving data from spark to MapR DB

Question asked by anshul09013 on Aug 22, 2017
Latest reply on Oct 27, 2017 by mstang

Hi, 

I have created a sample spark application which reads the data from MapR stream and save it to MapR DB. I get below error while saving data to MapR DB. I am using Spark Version 2.1.0 and MapR DB version as 5.2.2.

 

Exception in thread "streaming-job-executor-0" java.lang.NoClassDefFoundError: com/mapr/db/impl/MapRDBImpl      at com.mapr.db.spark.dbclient.DBOlderClientImpl$.tableExists(DBOlderClientImpl.scala:43)      at com.mapr.db.spark.utils.MapRDBUtils$.checkOrCreateTable(MapRDBUtils.scala:24)      at com.mapr.db.spark.RDD.DocumentRDDFunctions.saveToMapRDBInternal(DocumentRDDFunctions.scala:34)      at com.mapr.db.spark.RDD.OJAIDocumentRDDFunctions.saveToMapRDB(DocumentRDDFunctions.scala:61)      at com.sparkkafka.uber.SparkKafkaConsumerGlobal$$anonfun$main$1$$anonfun$apply$1.apply$mcVI$sp(SparkKafkaConsumerGlobal.scala:80)      at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:160)      at com.sparkkafka.uber.SparkKafkaConsumerGlobal$$anonfun$main$1.apply(SparkKafkaConsumerGlobal.scala:75)      at com.sparkkafka.uber.SparkKafkaConsumerGlobal$$anonfun$main$1.apply(SparkKafkaConsumerGlobal.scala:70)      at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)      at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)      at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)      at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)      at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)      at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)      at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)      at scala.util.Try$.apply(Try.scala:192)      at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)      at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)      at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)      at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)      at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)      at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)      at java.lang.Thread.run(Thread.java:748)

Outcomes