I would like to ask if root(user) instead of mapr admin can submit YARN/SPARK jobs?
Mark Dendrix Garcia Just wanted to add though that if you still wish to submit as root (UID 0 < UID 500) you could add the specific user in the allowed.system.users list in the /opt/mapr/hadoop/hadoop-<version>/etc/hadoop/container-executor.cfg file. Then, copy the file to all nodes and run the job again.
Hope that helps.
Hi Mark Dendrix Garcia,
I am not sure what you meant by "root" here. If you are referring to running Spark on Yarn, then here are more info: https://spark.apache.org/docs/latest/running-on-yarn.html
root user instead of mapr user
From what I read, you can install Spark on Yarn as root or using sudo; however, you can't test the installation as the root user. Please review http://maprdocs.mapr.com/home/AdvancedInstallation/InstallSparkonYARN.html.
We need to use non-root user to submit any Spark/YARN jobs.
id rootuid=0(root) gid=0(root) groups=0(root)
Job submitted with root user may fail will following exception,
Diagnostics: Application application_1504219316584_0006 initialization failed (exitCode=255) with output: Requested user root is not whitelisted and has id 0,which is below the minimum allowed 500
Thanks guys for this
You're most welcome Mark Dendrix Garcia.
Retrieving data ...