AnsweredAssumed Answered

oozie error: Permission denied

Question asked by akusanagi Employee on Apr 23, 2012
Latest reply on Apr 23, 2012 by nabeel
I setup mapr-oozie on one of my cluster nodes. When the user root runs the oozie example code, it finishes successfully. However, when the other user (mapr) runs it, it fails generating the following error.

    java.io.IOException: Open failed for file: /var/mapr/cluster/mapred/jobTracker/staging/mapr/.staging/job_201204241610_0001/job.split, error: Permission denied (13)
     at com.mapr.fs.MapRClient.open(MapRClient.java:189)
     at com.mapr.fs.MapRFileSystem.open(MapRFileSystem.java:323)
     at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:460)
     at org.apache.hadoop.mapred.MapTask.getSplitDetails(MapTask.java:336)
     at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:368)
     at org.apache.hadoop.mapred.MapTask.run(MapTask.java:327)
     at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
     at java.security.AccessController.doPrivileged(Native Method)
     at javax.security.auth.Subject.doAs(Subject.java:396)
     at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1109)
     at org.apache.hadoop.mapred.Child.main(Child.java:264)

It seems that the job executes as the user mapr, but it cannot read its job staging files which are created by root and accessible only by the user root. By setting the hadoop.proxyuser.oozie.groups property, the user root should impersonate the user mapr, however it doesn't seem to work here.

I tried the following steps:

(1) Create the user mapr on all the nodes

    [root] groupadd -g 600 mapr
    [root] useradd -u 600 -g 600 mapr

(2) Edit /opt/mapr/hadoop/hadoop-0.20.2/conf/mapred-site.xml on all the nodes

    <property>
      <name>hadoop.proxyuser.root.hosts</name>
      <value>*</value>
    </property>
    <property>
      <name>hadoop.proxyuser.root.groups</name>
      <value>mapr,root</value>
    </property>

(3) Unzip oozie examples and run it as the user root --> it finishes successfully

    [root] cd /opt/mapr/oozie/oozie-3.0.0/
    [root] unzip oozie-examples.tar.gz
    [root] hadoop fs -put examples examples
    [root] oozie job --oozie http://localhost:11000/oozie -config examples/apps/map-reduce/job.properties -DjobTracker=<jobtracker node>:9001 -run

(4) Run it as the user mapr --> it fails

    [mapr] cd /opt/mapr/oozie/oozie-3.0.0/
    [mapr] hadoop fs -put examples examples
    [mapr] oozie job --oozie http://localhost:11000/oozie -config examples/apps/map-reduce/job.properties -DjobTracker=<jobtracker node>:9001 -run

Any comments are greatly appreciated.

Outcomes