AnsweredAssumed Answered

Mapper, Reducer allocation to a job

Question asked by clpong on Jun 19, 2015
hi,
I have a job that running periodically - monthly.
And I notice that sometimes the number of mappers, and reducer that allocated to job might be different from time to time (eg: might be this month, and next month).

1.) How's the calculation for map input records vs mapper and reducer allocation?

Examples:
map input records: 36,951,844 - job_201505051724_1425 - 692 mappers, 235 reducers
map input records:21,596,267 - job_201506102200_0843 - 410 mappers, 235 reducers
map input records:4,279,728 - job_201506042321_0060 - 82 mappers, 200 reducers

2.) Any idea why mappers, and reducers are at pending status?
Example, it assigned 410 mappers, but 10 mappers are at pending status.

3.) Will the pending mappers cause reducers to pending status as well?
In this particular case, after 410 mappers successfully run, reducers started the work, but still there are about 95 reducer pending.


Outcomes