AnsweredAssumed Answered

/opt/mapr/lib/spring jars conflict with my own job with Mapr5.1

Question asked by perttvil on Jun 13, 2016
Latest reply on Feb 22, 2018 by maprcommunity

I just installed mapr 5.1 cluster, and I noticed that /opt/mapr/lib has spring 3 jars. Mapr5.0 did not have these jars.


Spring 3 does not work with java 8 (problems with asm). And I am using spring 4 with java 8 in my spark job. My spark job adds spring 4 jars to the job classpath and everything worked well with mapr 5.0.


It seems /opt/mapr/lib/spring*3* jars are added to spark job classpath automatically and those jars are in the start of classpath, so even though my spark job includes spring 4 jars, spark 3 jars are used from /opt/mapr/lib => Job does not work.


This might have something to do with LD_LIBRARY_PATH environment variable set in and other config

scripts under hadoop directory.


There are no references to this directory under /opt/mapr/spark/spark-1.6.0/ directory so mapr distribution must add then to classpath somewhere.


Is there a safe way to remove spring 3 jars from spark job classpath?