AnsweredAssumed Answered

Live Lock issue with YARN's Fair Scheduler

Question asked by Velumani on Aug 9, 2016
Latest reply on Aug 12, 2016 by ryanvictory

Hi,

     I am using YARN to deploy Spark jobs in my 4 node aws cluster. When I submit the job, the Application state is not moving from "ACCEPTED". In the Yarn Resource Manager  web console, the apps are in Pending  queue and Final Status is UNDEFINED.

 

     The cluster have enough resource and no parallel jobs running. Apparently the RM unable to allocate containers. When I check http://<resource manager ip>:8088/cluster/scheduler, the used resource % is 0 for all users and all those pending apps are in ACCEPTED state.

 

     YARN uses Fair Scheduler. 

 

     Any help would be greatly appreciated !!!

Outcomes