AnsweredAssumed Answered

Hbase Bulk load - Map Reduce job failing on MapR

Question asked by shashi.vish on Jun 2, 2015
Latest reply on Jun 10, 2015 by nabeel
I have map reduce job for hbase bulk load. Job is converting data into Hfiles and loading into hbase but after certain map % job is failing. Below is the exception that I am getting.

Error: /var/mapr/local/tm4/mapred/nodeManager/spill/job_1433110149357_0005/attempt_1433110149357_0005_m_000000_0/spill83.out.index
    at org.apache.hadoop.mapred.SpillRecord.<init>(
    at org.apache.hadoop.mapred.MapRFsOutputBuffer.mergeParts(
    at org.apache.hadoop.mapred.MapRFsOutputBuffer.flush(
    at org.apache.hadoop.mapred.MapTask$NewOutputCollector.close(
    at org.apache.hadoop.mapred.MapTask.runNewMapper(
    at org.apache.hadoop.mapred.YarnChild$
    at Method)
    at org.apache.hadoop.mapred.YarnChild.main(

Its say's file not found exception but when i searched this file in above mentioned directory , i was able to locate it.

Only thing that i noticed in job that for small set of data it is working fine but as data grows job starts failing.Currently I am using Mapr 10 node cluster for this.

Let me know if anyone has faced this issue.