AnsweredAssumed Answered

Failed to renew token: Kind MR_DELEGATION_TOKEN

Question asked by shrilesh.naik on Feb 26, 2018
Latest reply on Apr 20, 2018 by maprcommunity

Facing this issue while submitting sample oozie job - Setup and Run Oozie Examples 

 

We have valid token generated. Oozie is configured to run with "mapr" user.

 

facing below error :

 

2018-02-26 11:35:05,613 WARN ActionStartXCommand:523 - SERVER[x.x.x.x] USER[mapr] GROUP[-] TOKEN[] APP[SparkFileCopy] JOB[0000003-180226111711614-oozie-mapr-W] ACTION[0000003-180226111711614-oozie-mapr-W@spark-node] Error starting action [spark-node]. ErrorType [TRANSIENT], ErrorCode [JA009], Message [JA009: org.apache.hadoop.yarn.exceptions.YarnException: Failed to submit application_1519057374135_0064 to YARN : Failed to renew token: Kind: MR_DELEGATION_TOKEN, Service: 0.0.0.0:10020, Ident: (owner=mapr, renewer=mapr, realUser=, issueDate=1519644895225, maxDate=1520249695225, sequenceNumber=53, masterKeyId=8)]
org.apache.oozie.action.ActionExecutorException: JA009: org.apache.hadoop.yarn.exceptions.YarnException: Failed to submit application_1519057374135_0064 to YARN : Failed to renew token: Kind: MR_DELEGATION_TOKEN, Service: 0.0.0.0:10020, Ident: (owner=mapr, renewer=mapr, realUser=, issueDate=1519644895225, maxDate=1520249695225, sequenceNumber=53, masterKeyId=8)
at org.apache.oozie.action.ActionExecutor.convertExceptionHelper(ActionExecutor.java:457)
at org.apache.oozie.action.ActionExecutor.convertException(ActionExecutor.java:441)
at org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:1198)
at org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:1369)
at org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:234)
at org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:65)
at org.apache.oozie.command.XCommand.call(XCommand.java:287)
at org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:332)
at org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:261)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:179)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException: org.apache.hadoop.yarn.exceptions.YarnException: Failed to submit application_1519057374135_0064 to YARN : Failed to renew token: Kind: MR_DELEGATION_TOKEN, Service: 0.0.0.0:10020, Ident: (owner=mapr, renewer=mapr, realUser=, issueDate=1519644895225, maxDate=1520249695225, sequenceNumber=53, masterKeyId=8)
at org.apache.hadoop.mapred.YARNRunner.submitJob(YARNRunner.java:327)
at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:243)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1290)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1287)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1631)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1287)
at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:575)
at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:570)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1631)
at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:570)
at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:561)
at org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:1183)
... 11 more
Caused by: org.apache.hadoop.yarn.exceptions.YarnException: Failed to submit application_1519057374135_0064 to YARN : Failed to renew token: Kind: MR_DELEGATION_TOKEN, Service: 0.0.0.0:10020, Ident: (owner=mapr, renewer=mapr, realUser=, issueDate=1519644895225, maxDate=1520249695225, sequenceNumber=53, masterKeyId=8)
at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.submitApplication(YarnClientImpl.java:278)
at org.apache.hadoop.mapred.ResourceMgrDelegate.submitApplication(ResourceMgrDelegate.java:291)
at org.apache.hadoop.mapred.YARNRunner.submitJob(YARNRunner.java:311)
... 26 more
 

Outcomes