AnsweredAssumed Answered

Tez on mapr4.1

Question asked by skhurana333 on Jul 30, 2016
Latest reply on Nov 14, 2016 by aalvarez

Hello

 

 

On mapR single node cluster, I compiled Tez as per the documentation on apache tez site.

 

The only thing I changed from documentation was in tez-site.xml, where I gave the value of tez.use.cluster.hadoop-libs as true. I did it because mapR has its own jars for hadoop components. And without these tez might not work.

 

When I run any MR job or hive query I get this below error in the job on RM link. (DIGEST-MD5: digest response format violation. Mismatched URI: default/; expecting: null/default)

 

I am using mapr 4.1 and tez 0.7.0. mapR 4.1 uses hive 1.2.0.

Any pointer or suggestion what might be causing this issue or where should I look?

 

 

2016-07-30 09:58:38,737 INFO [main] ipc.Server: Stopping server on 45868

2016-07-30 09:58:38,737 INFO [IPC Server Responder] ipc.Server: Stopping IPC Server Responder

2016-07-30 09:58:38,739 INFO [IPC Server listener on 45868] ipc.Server: Stopping IPC Server listener on 45868

2016-07-30 09:58:38,739 INFO [IPC Server Responder] ipc.Server: Stopping IPC Server Responder

2016-07-30 09:58:38,739 ERROR [main] app.DAGAppMaster: Error starting DAGAppMaster

org.apache.tez.dag.api.TezUncheckedException: javax.security.sasl.SaslException: DIGEST-MD5: digest response format violation. Mismatched URI: default/; expecting: null/default [Caused by org.apache.hadoop.ipc.RemoteException(javax.security.sasl.SaslException): DIGEST-MD5: digest response format violation. Mismatched URI: default/; expecting: null/default]

  at org.apache.tez.dag.app.rm.YarnTaskSchedulerService.serviceStart(YarnTaskSchedulerService.java:384)

  at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)

  at org.apache.tez.dag.app.rm.TaskSchedulerEventHandler.serviceStart(TaskSchedulerEventHandler.java:353)

  at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)

  at org.apache.tez.dag.app.DAGAppMaster$ServiceWithDependency.start(DAGAppMaster.java:1573)

  at org.apache.tez.dag.app.DAGAppMaster$ServiceThread.run(DAGAppMaster.java:1591)

Caused by: javax.security.sasl.SaslException: DIGEST-MD5: digest response format violation. Mismatched URI: default/; expecting: null/default [Caused by org.apache.hadoop.ipc.RemoteException(javax.security.sasl.SaslException): DIGEST-MD5: digest response format violation. Mismatched URI: default/; expecting: null/default]

  at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)

  at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)

  at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)

  at java.lang.reflect.Constructor.newInstance(Constructor.java:526)

  at org.apache.hadoop.yarn.ipc.RPCUtil.instantiateException(RPCUtil.java:53)

  at org.apache.hadoop.yarn.ipc.RPCUtil.unwrapAndThrowException(RPCUtil.java:104)

  at org.apache.hadoop.yarn.api.impl.pb.client.ApplicationMasterProtocolPBClientImpl.registerApplicationMaster(ApplicationMasterProtocolPBClientImpl.java:109)

  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

  at java.lang.reflect.Method.invoke(Method.java:606)

  at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)

  at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)

  at com.sun.proxy.$Proxy24.registerApplicationMaster(Unknown Source)

  at org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl.registerApplicationMaster(AMRMClientImpl.java:222)

  at org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl.registerApplicationMaster(AMRMClientImpl.java:214)

  at org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl.registerApplicationMaster(AMRMClientAsyncImpl.java:138)

  at org.apache.tez.dag.app.rm.YarnTaskSchedulerService.serviceStart(YarnTaskSchedulerService.java:368)

  ... 5 more

Caused by: org.apache.hadoop.ipc.RemoteException(javax.security.sasl.SaslException): DIGEST-MD5: digest response format violation. Mismatched URI: default/; expecting: null/default

  at org.apache.hadoop.ipc.Client.call(Client.java:1468)

  at org.apache.hadoop.ipc.Client.call(Client.java:1399)

  at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232)

  at com.sun.proxy.$Proxy23.registerApplicationMaster(Unknown Source)

  at org.apache.hadoop.yarn.api.impl.pb.client.ApplicationMasterProtocolPBClientImpl.registerApplicationMaster(ApplicationMasterProtocolPBClientImpl.java:106)

  ... 16 more

2016-07-30 09:58:38,741 INFO [Thread-2] app.DAGAppMaster: DAGAppMasterShutdownHook invoked


 

Outcomes