AnsweredAssumed Answered

org.apache.hadoop.security.AccessControlException: User mapr(user id 1008)  has been denied access- MapR Client

Question asked by Velumani on Jun 3, 2016
Latest reply on Jun 4, 2016 by Velumani

Hi,

     I have installed MapR Client in centos and trying to access aws cluster resource. If I run the hadoop fs -ls command I can see the cluster directories.

 

[mapr@aws-dev-srv-04 batch-spark]$ hadoop fs -ls /mapr/dev.aws.test.com

Found 15 items

tr--------   3 root root          2 2016-05-11 20:02 /mapr/dev.aws.test.com/Test

drwxr-xr-x   - 5000 root          3 2016-05-30 09:16 /mapr/dev.aws.test.com/apps

-rw-rw-r--   3 5000 5000  158869168 2016-05-13 17:51 /mapr/dev.aws.test.com/cardissuing.csv

drwxrwxr-x   - 5000 5000          0 2016-06-03 09:06 /mapr/dev.aws.test.com/destination

drwxr-xr-x   - 5000 root         18 2016-06-03 11:10 /mapr/dev.aws.test.com/hbase

drwxr-xr-x   - root root          3 2016-04-12 15:13 /mapr/dev.aws.test.com/installer

drwxr-xr-x   - root root          0 2016-05-10 06:05 /mapr/dev.aws.test.com/kylin

drwxr-xr-x   - 5000 5000          3 2016-05-20 07:24 /mapr/dev.aws.test.com/oozie

drwxr-xr-x   - 5000 root          0 2016-04-12 15:06 /mapr/dev.aws.test.com/opt

drwxrwxr-x   - 5000 5000         15 2016-06-03 07:14 /mapr/dev.aws.test.com/poc

drwxrwxr-x   - 5000 5000          4 2016-05-10 14:57 /mapr/dev.aws.test.com/projects

drwxrwxr-x   - 5000 5000          3 2016-06-03 10:32 /mapr/dev.aws.test.com/tables

drwxrwxrwx   - 5000 root          1 2016-05-31 12:47 /mapr/dev.aws.test.com/tmp

drwxr-xr-x   - 5000 root          3 2016-05-10 10:30 /mapr/dev.aws.test.com/user

drwxr-xr-x   - 5000 5000          1 2016-04-12 15:05 /mapr/dev.aws.test.com/var

 

 

But when I try to submit a spark job using Spring YarnTasklet, I am getting access denied exception 

 

[2016-06-03 11:03:58.691] boot - 14246 ERROR [main] --- AbstractStep: Encountered an error executing step initScript in job TweetTopHashtags

org.apache.hadoop.security.AccessControlException: User mapr(user id 1008)  has been denied access to create application_1464696944391_0053

        at com.mapr.fs.MapRFileSystem.makeDir(MapRFileSystem.java:1157)

        at com.mapr.fs.MapRFileSystem.mkdirs(MapRFileSystem.java:1177)

        at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:1913)

        at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:634)

        at org.apache.spark.deploy.yarn.Client.prepareLocalResources(Client.scala:356)

        at org.apache.spark.deploy.yarn.Client.createContainerLaunchContext(Client.scala:727)

        at org.apache.spark.deploy.yarn.Client.submitApplication(Client.scala:142)

        at org.apache.spark.deploy.yarn.Client.run(Client.scala:1021)

        at org.springframework.data.hadoop.batch.spark.SparkYarnTasklet.execute(SparkYarnTasklet.java:79)

        at org.springframework.batch.core.step.tasklet.TaskletStep$ChunkTransactionCallback.doInTransaction(TaskletStep.java:406)

        at org.springframework.batch.core.step.tasklet.TaskletStep$ChunkTransactionCallback.doInTransaction(TaskletStep.java:330)

        at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:133)

        at org.springframework.batch.core.step.tasklet.TaskletStep$2.doInChunkContext(TaskletStep.java:271)

        at org.springframework.batch.core.scope.context.StepContextRepeatCallback.doInIteration(StepContextRepeatCallback.java:81)

        at org.springframework.batch.repeat.support.RepeatTemplate.getNextResult(RepeatTemplate.java:374)

        at org.springframework.batch.repeat.support.RepeatTemplate.executeInternal(RepeatTemplate.java:215)

        at org.springframework.batch.repeat.support.RepeatTemplate.iterate(RepeatTemplate.java:144)

        at org.springframework.batch.core.step.tasklet.TaskletStep.doExecute(TaskletStep.java:257)

        at org.springframework.batch.core.step.AbstractStep.execute(AbstractStep.java:200)

        at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:148)

        at org.springframework.batch.core.job.AbstractJob.handleStep(AbstractJob.java:392)

        at org.springframework.batch.core.job.SimpleJob.doExecute(SimpleJob.java:135)

        at org.springframework.batch.core.job.AbstractJob.execute(AbstractJob.java:306)

        at org.springframework.batch.core.launch.support.SimpleJobLauncher$1.run(SimpleJobLauncher.java:135)

        at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50)

        at org.springframework.batch.core.launch.support.SimpleJobLauncher.run(SimpleJobLauncher.java:128)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:498)

        at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:302)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)

        at org.springframework.batch.core.configuration.annotation.SimpleBatchConfiguration$PassthruAdvice.invoke(SimpleBatchConfiguration.java:127)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)

        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:207)

        at com.sun.proxy.$Proxy33.run(Unknown Source)

        at com.springdeveloper.demo.SparkYarnApplication.run(SparkYarnApplication.java:29)

        at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:707)

        at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:691)

        at org.springframework.boot.SpringApplication.afterRefresh(SpringApplication.java:678)

        at org.springframework.boot.SpringApplication.doRun(SpringApplication.java:343)

        at org.springframework.boot.SpringApplication.run(SpringApplication.java:273)

        at org.springframework.boot.SpringApplication.run(SpringApplication.java:980)

        at org.springframework.boot.SpringApplication.run(SpringApplication.java:969)

        at com.springdeveloper.demo.SparkYarnApplication.main(SparkYarnApplication.java:23)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:498)

        at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:53)

        at java.lang.Thread.run(Thread.java:745)

 

Also when I try to create table in MapR-DB using OJAI, I am getting same issue. 

 

2016-06-03 11:35:54,5155 ERROR Client fs/client/fileclient/cc/client.cc:2367 Thread: 15583 Unlink failed for file FILECONFIG, error Permission denied(13)

2016-06-03 11:35:54,5156 ERROR JniCommon fs/client/fileclient/cc/jni_MapRClient.cc:951 Thread: 15583 remove: File /tables/FILECONFIG, rpc error, Permission denied(13)

Exception in thread "main" com.mapr.db.exceptions.AccessDeniedException: User mapr(user id 1008)  does not have access to /tables/FILECONFIG

        at com.mapr.db.exceptions.ExceptionHandler.handle(ExceptionHandler.java:59)

        at com.mapr.db.impl.AdminImpl.deleteTable(AdminImpl.java:401)

        at com.mapr.db.impl.AdminImpl.deleteTable(AdminImpl.java:391)

        at com.mapr.db.MapRDB.deleteTable(MapRDB.java:95)

        at com.mapr.db.importer.streamParser.run(streamParser.java:34)

        at com.mapr.db.importer.streamParser.main(streamParser.java:23)

Caused by: org.apache.hadoop.security.AccessControlException: User mapr(user id 1008)  does not have access to /tables/FILECONFIG

        at com.mapr.fs.MapRFileSystem.delete(MapRFileSystem.java:1144)

        at com.mapr.fs.MapRFileSystem.delete(MapRFileSystem.java:1175)

        at com.mapr.db.impl.AdminImpl.deleteTable(AdminImpl.java:399)

        ... 4 more

Help me in understanding what is missing in client setup

Outcomes