We need to increase the Xmx for HistoryServer service but it does not work as expected.
We increased memory for HistoryServer as described in hadoop documentatin, just add following line to yarn-env.sh and restart warden service:
The problem is change is not taking effect after warden restart. See `ps aux` output:
mapr 19565 0.1 0.1 3079240 996208 ? Sl Apr29 8:22 /usr/java/latest/bin/java -Dproc_historyserver -Xmx1000m -Djava.net.preferIPv4Stack=true -Dhadoop.log.dir=/opt/mapr/hadoop/hadoop-2.7.0/logs -Dhadoop.log.file=hadoop.log <other -D...> -Dhadoop.security.logger=INFO,NullAppender org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer
The reason why we need to increase Xmx is OutOfMemoryException in HistoryServer service (/opt/mapr/roles/historyserver) and we cannot see mapreduce-jobs logs in Hadoop UI. Probably we had huge log file for some job.
2016-04-15 14:33:42,281 INFO org.apache.hadoop.mapreduce.v2.hs.JobHistory: Start ing scan to move intermediate done files 2016-04-15 14:44:15,386 WARN org.apache.hadoop.ipc.Server: IPC Server handler 4 on 10020, call org.apache.hadoop.mapreduce.v2.api.HSClientProtocolPB.getJobRepor t from 10.0.0.1:37709 Call#50610 Retry#0 java.lang.OutOfMemoryError: Java heap space
So what we are doing wrong? Is there some MapR specific step required to make this work?