Author: Jonathan Bubier
Original Publication Date: December 1, 2014
When MapR's warden service starts on a node it calculates the service memory allocation based on the configured roles and the configuration in /opt/mapr/conf/warden.conf. For the MFS service the memory allocation is determined based on the values for the following configuration parameters:
The MFS service is guaranteed to be allocated at least 'service.command.mfs.heapsize.min' and capped by the lower of 'service.command.mfs.heapsize.percent' and 'service.command.mfs.heapsize.max'. The "min" and "max" parameter define a memory lower and upper bound respectively in MB.
Note: The upper bound on MFS memory, whether set by service.command.mfs.heapsize.percent or service.command.mfs.heapsize.max is not an absolute limit. It can be exceeded by up to 25% before the MFS process is restarted. For example, if the upper bound on MFS memory is calculated to be 10000MB the memory used by MFS can be up to 12500MB. Take this additional memory into account when determining the MFS memory limits to avoid exhausting the available system memory.
The current memory allocation for MFS can be seen by looking at the command line arguments for the MFS process. Ex:
# ps -ef | grep mfs
mapr 20108 1 0 Nov03 ? 00:40:55 /opt/mapr/server/mfs -b -p 5660 -n inode:10:log:6:meta:10:dir:15:small:15 -m 2756 -g -O /opt/mapr/conf/mapr-clusters.conf -V
Note the '-m' argument which specifies the memory allocation in MB, 2756 MB in the above example. If the memory allocated to MFS needs to be adjusted, it can be controlled in the following ways.
To allocate a specific amount of memory to MFS set the 'service.command.mfs.heapsize.min' and 'service.command.mfs.heapsize.max' parameters to equivalent values in /opt/mapr/conf/warden.conf. For example:
This configuration guarantees that MFS is allocated 40000MB assuming the node has sufficient memory available. Use caution when manually specifying memory amounts for MFS to ensure there is sufficient memory available for the OS and for the remaining MapR services on the node. After modifying the value of these parameters restart warden for the change to take effect.
To set an upper limit on the memory allocated to MFS set the parameter 'service.command.mfs.heapsize.max' in /opt/mapr/conf/warden.conf. Based on the other services configured on the node there is no guarantee MFS will be allocated the amount of memory specified but it will be allocated no more than this amount. After modifying the value of this parameter restart warden for the change to take effect.
In MapR v4.0.1 and later the system memory allocated to MFS can also be limited via the parameter 'service.command.mfs.heapsize.maxpercent'. This parameter specifies a percentage of the system memory that is the upper limit on MFS memory. If both 'service.command.mfs.heapsize.max' and 'service.command.mfs.heapsize.maxpercent' are defined in /opt/mapr/conf/warden.conf the parameter resulting in the lower memory allocation takes effect. For example:
With the above configuration on a node with 192GB of system memory the memory allocated to MFS will be capped at 40000MB.
If after setting the above parameters to control the MFS memory usage there are unexpected results please contact MapR Support using the Support Portal or by contacting firstname.lastname@example.org. Please also collect a support-dump using /opt/mapr/support/tools/mapr-support-dump.sh for MapR Support to review the MapR configuration and system diagnostics.