AnsweredAssumed Answered

Kakfka Connect HDFS Sink and MapR FS

Question asked by mandoskippy on Jan 23, 2017
Latest reply on Mar 6, 2017 by maprcommunity

I am looking at using the Kafka Connect HDFS Sink (HDFS Connector — Confluent Platform 2.0.0 documentation )  to take data on a Kafka cluster (out of my control) and move it to a MapR Cluster in my control.  Basically, I am looking for specific instructions on how to make this work, and in the absence of that, is there any good documentation on how to take something that expects to talk to the HDFS API and help it to talk to MapR FS in a straight forward way. 

 

This may seem like a basic question, but I have been using Spark 2.x with the Hadoop provided distribution with MapR and through lots of trial and error, I have gotten it working well, without having to rebuild spark. (Lots of updating class path locations etc).  Is this true for most things in Java where if you provide different classpath locations, it will work without rebuilding?  I guess that my base question... 

 

Thanks!

Outcomes