I just installed mapr 5.1 cluster, and I noticed that /opt/mapr/lib has spring 3 jars. Mapr5.0 did not have these jars.
Spring 3 does not work with java 8 (problems with asm). And I am using spring 4 with java 8 in my spark job. My spark job adds spring 4 jars to the job classpath and everything worked well with mapr 5.0.
It seems /opt/mapr/lib/spring*3* jars are added to spark job classpath automatically and those jars are in the start of classpath, so even though my spark job includes spring 4 jars, spark 3 jars are used from /opt/mapr/lib => Job does not work.
This might have something to do with LD_LIBRARY_PATH environment variable set in hadoop_config.sh and other config
scripts under hadoop directory.
There are no references to this directory under /opt/mapr/spark/spark-1.6.0/ directory so mapr distribution must add then to classpath somewhere.
Is there a safe way to remove spring 3 jars from spark job classpath?