AnsweredAssumed Answered

MapR maven dependency conflicts with existing Hadoop dependency

Question asked by niranjan on Sep 19, 2016
Latest reply on Oct 30, 2017 by snayeem


I'm working on a project which has a dependency on Apache Hadoop related jars, in this project I also have a module which reads from or writes to HDFS. 

I tried testing this module against MapR-FS and it works fine (I've installed the client as required). But in order to make this work I did the following 

I had to remove Apache related Hadoop jars from my classpath, having both MapR client dependency /opt/mapr/lib and Apache Hadoop jars in classpath resulted in the following exception  


   No FileSystem for scheme: maprfs.


I could understand there is a jar conflict but is there a way or some configuration that I could specify in my module or client so that it could force my application to load correct dependency? 


For instance would something like this help?


Configuration conf = new Configuration();
FileSystem fs = FileSystem.get( conf);

The above didn't work actually but looking for a suggestion something like above.


 Can anyone help me to find out the why there is a conflict or explain me what is happening here? Is this a general class loading issue? 


Sanjamala Nayeem Ted Dunning MC Srivas Nabeel Moidu