AnsweredAssumed Answered


Question asked by stormcrow on Jul 8, 2013
Latest reply on Jul 9, 2013 by gera
Reduce tasks for most cluster jobs all failing with the following error:

    Diagnostic Messages for this Task:
    Error: java.lang.NumberFormatException: For input string: "2244811652"
            at java.lang.NumberFormatException.forInputString(
            at java.lang.Integer.parseInt(
            at java.lang.Integer.valueOf(
            at org.apache.hadoop.fs.PathId.getFidInfo(
            at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$MapOutputLocation.<init>(
            at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$GetMapEventsThread.getMapCompletionEvents(
            at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$
    FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask

It looks to us like our "fileID" of wherever the Reduce task is trying
to send output have exceeded the size of a 32 bit signed integer
(2.1B)....note it's trying to parse an int from "2244811652" (2.2B).

Have we reached the maximum number of files on a volume somewhere or

What exactly is PathId.getFidInfo() trying to read?