AnsweredAssumed Answered

Spark on MapR 6 generates authentication error

Question asked by rbukarev on Feb 19, 2018
Latest reply on Feb 21, 2018 by MichaelSegel

I've got MapR 6 cluster, on which I installed Spark using MapR Installer.

I have compiled a Scala program and tried to execute it with Spark:

$SPARK_HOME/bin/spark-submit --properties-file spark-override.conf --master yarn --class myClass myprogram.jar

 

Here, spark-override.conf is a copy of spark-defaults.conf with the property for using ssl turned off.

I have used maprlogin to create my ticket.

 

The exceptions chain I'm getting contains this:

Exception in thread "main" java.io.IOException: org.apache.hadoop.security.authentication.client.AuthenticationException: Authentication failed, status: 401, message: Authentication required
at org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl$5.run(TimelineClientImpl.java:476)
at org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl$TimelineClientConnectionRetry.retryOn(TimelineClientImpl.java:182)
at org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl.operateDelegationToken(TimelineClientImpl.java:489)
at org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl.getDelegationToken(TimelineClientImpl.java:386)
at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.getTimelineDelegationToken(YarnClientImpl.java:356)
at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.addTimelineDelegationToken(YarnClientImpl.java:337)
at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.submitApplication(YarnClientImpl.java:257)
at org.apache.spark.deploy.yarn.Client.submitApplication(Client.scala:175)

 

Did I miss something in Spark configuration or in my program build?

Outcomes