AnsweredAssumed Answered

Hiveserver2 leak of memory

Question asked by rvelfre on Mar 17, 2016
Latest reply on Mar 24, 2016 by mufeed

Hello !

 

I'm facing an issue after migrating from MapR 3 to MapR 5.

Our hiveserver2 (that we updated from 0.13-1409 to 0.13-1510) throws a java heap space error every 5 hours so I assume there is a memory leak somewhere...

 

In the following log :  /opt/mapr/hive/hive-0.13/logs/mapr/hive.log

The same ERROR message is throwing many times/sec and says :

 

2016-03-17 17:35:03,823 ERROR [HiveServer2-Handler-Pool: Thread-16]: server.TThreadPoolServer (TThreadPoolServer.java:run(215)) - Error occurred during processing of message.
java.lang.RuntimeException: org.apache.thrift.transport.TTransportException

at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$HiveSaslServerTransportFactory.getTransport(HadoopThriftAuthBridge.java:179)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:189)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.thrift.transport.TTransportException
at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$HiveSaslServerTransportFactory$1.receiveSaslMessage(HadoopThriftAuthBridge.java:196)
at org.apache.thrift.transport.TSaslServerTransport.handleSaslStartMessage(TSaslServerTransport.java:125)
at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:253)
at org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:41)
at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$HiveSaslServerTransportFactory.getTransport(HadoopThriftAuthBridge.java:176)
... 4 more

 

It seems to be related to the SALS authentication. For you information, we’re running a non-secure cluster.

We use default hive.server2.authentication properties in hive-site.xml (which is SASL mode).

 

Has anyone facing this issue ?

 

Thanks for your help !

Outcomes