I am using YARN to deploy Spark jobs in my 4 node aws cluster. When I submit the job, the Application state is not moving from "ACCEPTED". In the Yarn Resource Manager web console, the apps are in Pending queue and Final Status is UNDEFINED.
The cluster have enough resource and no parallel jobs running. Apparently the RM unable to allocate containers. When I check http://<resource manager ip>:8088/cluster/scheduler, the used resource % is 0 for all users and all those pending apps are in ACCEPTED state.
YARN uses Fair Scheduler.
Any help would be greatly appreciated !!!