AnsweredAssumed Answered

How to Increase HistoryServer memory (Heap) in MapR distribution?

Question asked by dafox on May 3, 2016
Latest reply on May 12, 2016 by dafox



We need to increase the Xmx for HistoryServer service but it does not work as expected.


We increased memory for HistoryServer as described in hadoop documentatin, just add following line to and restart warden service:



The problem is change is not taking effect after warden restart. See `ps aux` output:

mapr  19565  0.1  0.1 3079240 996208 ? Sl   Apr29   8:22 /usr/java/latest/bin/java -Dproc_historyserver -Xmx1000m -Dhadoop.log.dir=/opt/mapr/hadoop/hadoop-2.7.0/logs 
 -Dhadoop.log.file=hadoop.log <other -D...>,NullAppender 


The reason why we need to increase Xmx is OutOfMemoryException in HistoryServer service (/opt/mapr/roles/historyserver) and we cannot see mapreduce-jobs logs in Hadoop UI. Probably we had huge log file for some job.

2016-04-15 14:33:42,281 INFO org.apache.hadoop.mapreduce.v2.hs.JobHistory: Start
ing scan to move intermediate done files
2016-04-15 14:44:15,386 WARN org.apache.hadoop.ipc.Server: IPC Server handler 4 
on 10020, call org.apache.hadoop.mapreduce.v2.api.HSClientProtocolPB.getJobRepor
t from Call#50610 Retry#0
java.lang.OutOfMemoryError: Java heap space


So what we are doing wrong? Is there some MapR specific step required to make this work?