AnsweredAssumed Answered

Best Practices for yarn / disk

Question asked by Daniar on Jun 14, 2017
Latest reply on Jun 19, 2017 by MichaelSegel



After reading the following article and documentations:


I have questions regarding property.

1. If I understand right this is a mapr yarn specific property. If I do not run MapReduce1 Jobs and mainly use spark on yarn, is there any best practices how to set this property in that case?


2. = [# of disk on the node] – [# of disks assigned to process MapReduce v1 jobs] will be the second argument equal to zero if MapReduce2 is set as default.

3. Is there any suggestions how to set queue disk property in fair-scheduler.xml e.g.

<queue name="myQueue">
<minResources>8192 mb, 8 vcores, ??? disks </minResources>
<maxResources>96000 mb, 48 vcores, ??? disks </maxResources>

Many thanks,