AnsweredAssumed Answered

Having more output than set reducers in hive

Question asked by sunny on Jan 6, 2014
Latest reply on Jan 6, 2014 by vkorukanti
Hi Team,

I have a mapr implementaion and using hive-0.11. I am executing a query which has some joins and a union all condition after setting these 2 properties in hive.

Set mapred.reduce.tasks=8;
Set hive.exec.reducers.max=8;
Below are the stats for mapper and reducers for each stage (3 stage Job)
Stage1: No. of mapper=152,No. of Reducers=8 (Expected)
Stage2: No. of mapper=146,No. of Reducers=8 (Expected)
Stage3: No. of mapper=137,No. of Reducer not specified, setting number of reducer=0 (not expected)

Kindly guide why the reducer has been automatically set to 0 in last stage as it should be 8 only as was set in hive.