AnsweredAssumed Answered

Spark on YARN without client VM process?

Question asked by john.humphreys on Oct 19, 2017
Latest reply on Oct 25, 2017 by cathy

When I run a spark job on YARN in cluster mode, the driver and workers are all in YARN.


There is still a process on the VM that ran Spark submit which is using resources though.  Can this be avoided?


When small clients launch many jobs, these processes can get pretty burdensome resource wise.