AnsweredAssumed Answered

MapR streams configuration for a remote producer

Question asked by mahdi62b on Jan 31, 2017
Latest reply on Mar 14, 2017 by cathy

Hi,

 

I have to develop a remote producer for MapR streams so it should connect to MapR cluster from a remote machine but not sure what should be its , the below is code for a local producer on MapR cluster

 

ggsnArray.foreachRDD( rdd => {
System.out.println("# events = " + rdd.count())

rdd.foreachPartition( partition => {
// Print statements in this section are shown in the executor's stdout logs
val topic = "/sample-stream:ggsn"
val properties=new Properties()
properties.put("value.serializer","org.apache.kafka.common.serialization.StringSerializer")
properties.put("key.serializer","org.apache.kafka.common.serialization.StringSerializer")
properties.put("key.deserializer","org.apache.kafka.common.serialization.StrinDeserializer")
properties.put("value.deserializer","org.apache.kafka.common.serialization.StringDeserializer")

//properties.put("client.id","testgroup")

val producer = new KafkaProducer[String, String](properties)
partition.foreach( record => {
val data = record.toString
//data=data.concat(",STREAM-GGSN")

 

// As as debugging technique, users can write to DBFS to verify that records are being written out
// dbutils.fs.put("/tmp/test_kafka_output",data,true)
val message = new ProducerRecord[String, String](topic, data)
producer.send(message)

})
producer.close()
})

})

 

 

How to configure above code to work on a remote machine outside of the cluster? 

Outcomes