AnsweredAssumed Answered

understanding question about RAM

Question asked by gabi_k on Apr 24, 2013
Latest reply on May 8, 2013 by gera
Hi, 
sometimes we got Java Heap Space error, and than we are increasing the JVM RAM limit (by Xmx value). 
i have 2 questions about it please: 

1) how can i know how much mapers and reducers should i have in cluster after i changing the Xmx value? i meant if i have node with 32G RAM, and have Xmx value for example to 5G, does it mean i can use only 6 maps and reducers (5*6=30G) ? 

2) why does it happend at all? i tried to read all this forum and didnt find the answer. if i have query on hive which reading a lot of data, i know the job settings the number of mappers which go and read the data from the MFS. each chunk of data goes to map, and up to the RAM. why i need to increase the Xmx , can't hive just split the data to more mappers? lets say i have job with 1000 mappers, and i got the HEAP space issue, can't hive give me 2000 mappers and it will fix the issue (each map will take less RAM, because it use less data) ?

Thanks!

Outcomes