AnsweredAssumed Answered

Best Practices for yarn yarn.nodemanager.resource.io-spindles / disk

Question asked by Daniar on Jun 14, 2017
Latest reply on Jun 19, 2017 by MichaelSegel

Hello,

 

After reading the following article and documentations:

 

https://mapr.com/blog/best-practices-yarn-resource-management/

http://maprdocs.mapr.com/home/AdministratorGuide/ResourceAllocation-MRv2Apps.html
http://maprdocs.mapr.com/home/AdministratorGuide/ResourceAllocation-YARNContainer.html

 

I have questions regarding yarn.nodemanager.resource.io-spindles property.

1. If I understand right this is a mapr yarn specific property. If I do not run MapReduce1 Jobs and mainly use spark on yarn, is there any best practices how to set this property in that case?

 

2. yarn.nodemanager.resource.io-spindles = [# of disk on the node] – [# of disks assigned to process MapReduce v1 jobs] will be the second argument equal to zero if MapReduce2 is set as default.

3. Is there any suggestions how to set queue disk property in fair-scheduler.xml e.g.

<queue name="myQueue">
<minResources>8192 mb, 8 vcores, ??? disks </minResources>
<maxResources>96000 mb, 48 vcores, ??? disks </maxResources>

Many thanks,

Daniar

Outcomes