MapReduce job fails with error "NoClassDefFound DirectFileOutputCommitter" while submitting job

Document created by wade on Feb 27, 2016
Version 1Show Document
  • View in full screen mode

Author: Jitendra Yadav, last modified by Hao Zhu on May 7, 2015

 

Original Publication Date: May 1, 2015

 

Environment
Hbase 0.94.X

Symptom


While running MapReduce jobs to access MapR-DB table it is getting below error, although the client app is compiled with proper Hbase classes.

Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/mapreduce/lib/output/DirectFileOutputCommitter 
at com.aexp.ims.cmph4.RowCounter.run(RowCounter.java:84)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.mapreduce.lib.output.DirectFileOutputCommitter
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)

Root Cause
Class loader doesn't find the definition of class "
DirectFileOutputCommitter".

Solution

DirectFileOutputCommitter class is in amazon-s3.jar.Please set the classpath to include this jar.

Attachments

    Outcomes