AnsweredAssumed Answered

Streaming Data from Cassandra to MapR DB

Question asked by manikanthkoora on Apr 23, 2018
Latest reply on Apr 23, 2018 by maprcommunity

I can see the data in df as a tabular format and it looks good. But when I save the df(dataframe) to mapr db, only one column is getting saved and remaining columns are missing. I am unable to find the root cause of this issue. Thanks for the help

val ssc = new StreamingContext(spark.sparkContext, Seconds(10))
val cassandraRDD = ssc.cassandraTable[Users](KEY_SPACE_NAME, TABLE_NAME)
val dstream = new ConstantInputDStream(ssc, cassandraRDD)
def saveToMapRDB(tablename: String, createTable: Boolean = false, bulkInsert: Boolean = false, idFieldPath: String = "username"): Unit = {}
dstream.foreachRDD{ rdd =>
val df = rdd.toDF()
df.write.saveToMapRDB("/tmp/usersInfo","username")
}

Outcomes