Mahout jobs fail with java.lang.IncompatibleClassChangeError after upgrading from MapR v3.x to MapR v4.x

Document created by jbubier Employee on Feb 7, 2016Last modified by jbubier Employee on Feb 7, 2016
Version 2Show Document
  • View in full screen mode

Author: Jonathan Bubier

 

Original Publication Date: February 4, 2015

 

After upgrading from MapR v3.x to MapR v4.x Mahout map-reduce jobs may fail with an exception similar to the following:

 

Exception in thread "main" java.lang.IncompatibleClassChangeError: Found interface org.apache.hadoop.mapreduce.JobContext, but class was expected

at org.apache.mahout.common.HadoopUtil.getCustomJobName(HadoopUtil.java:174)

at org.apache.mahout.clustering.streaming.mapreduce.StreamingKMeansDriver.runMapReduce(StreamingKMeansDriver.java:464)

at org.apache.mahout.clustering.streaming.mapreduce.StreamingKMeansDriver.run(StreamingKMeansDriver.java:419)

at org.apache.mahout.clustering.streaming.mapreduce.StreamingKMeansDriver.run(StreamingKMeansDriver.java:240)

at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)

at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)

at org.apache.mahout.clustering.streaming.mapreduce.StreamingKMeansDriver.main(StreamingKMeansDriver.java:491)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:606)

at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)

at org.apache.hadoop.util.ProgramDriver.run(ProgramDriver.java:145)

at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:153)

This exception shows that getCustomJobName() in HadoopUtil.java in the Mahout code is referencing the JobContext API and expected it as a class but found it as an interface.  In Hadoop 1.x the JobContext API is a class and in Hadoop 2.x it is an interface so the exception indicates that Mahout is using Hadoop 1 APIs while the MapR cluster is using Hadoop 2 APIs. 

 

All ecosystem packages made available by MapR have a version specifically built for MapR v4.x and Hadoop 2.  These packages are available from MapR's repository at http://package.mapr.com/releases/ecosystem-4.x/. When an ecosystem package is installed the jar files built for the installed Hadoop version are deployed to ensure version compatibility.  When the MapR core packages are upgraded from v3.x to v4.x all installed ecosystem packages must be reinstalled using their 4.x equivalent to deploy the jar files built for Hadoop 2 which is provided in MapR v4.x. In the case described above the MapR core packages has been upgraded but the ecosystem packages have not been modified.

 

To resolve the issue with Mahout as described above remove the mapr-mahout package wherever it is installed and install the corresponding version from MapR's v4.x ecosystem repository.  During the package installation a message similar to the following should be observed which indicates the Hadoop 2 jar files were deployed by the package installer:

# rpm -ivh mapr-mahout-0.9.201409041745-1.noarch.rpm 
Preparing... ########################################### [100%]
1:mapr-mahout ########################################### [100%]
POST_YARN=1, replacing MR1 jars

 

After reinstalling the mapr-mahout package verify the issue is resolved by re-running your Mahout jobs.  If the jobs still fail with the java.lang.IncompatibleClassChangeError please contact MapR Support at support@mapr.com or by opening a case via the Support Portal to investigate further.

Attachments

    Outcomes