AnsweredAssumed Answered

HiveServer2 Logging

Question asked by dimamah on Jan 25, 2014
Hey, 
We are considering to start using HiveServer2 and its seems that the logging capabilities are really limited as the server runs on a single port. 
I’ll explain the problem more accurately : 
Today, we are using HiveServer where each user gets a specific port (or a set of ports) then, we start X hiveservers , 1 for each port in the following manner 
`nohup hive --service hiveserver -p $2 -v --hiveconf hive.root.logger=INFO,console >> ~/hive.$2 2>&1 &`

For example : 
`nohup hive --service hiveserver -p 12354 -v --hiveconf hive.root.logger=INFO,console >> ~/hive.12354 2>&1 &`

This way, hiveserver runs in the background, the logging is redirected to the “console†but then its redirected to a file. 
So each port has its own log file, assuming there are no concurrent executions on the same port, each file is a sequential execution log. 

This is really convenient to debugging executions on the HiveServer as the full stacktrace isn’t passed through the JDBC and mostly we get a “MapredTask†error. 
The full errors are found in the logs and debugging is easy.

Now in the HiveServer2, I couldn’t find any way to separate  the logging per session. 
In fact i see that there is a [Jira][1] open for this issue. 

Is there any work around for this? I want each session / connection in its own log file, higher resolution is also acceptable (like a log per session) 
In fact, looking at hive-exec-log4j.properties it seems that logging is per query : 
`hive.log.file=${hive.query.id}.log` 
But this is a different log full with counter data / task progress when i’m looking for the standard logging as seem in the CLI. 
I think that this is defined by `hive.log.file` in hive-log4j.properties which is set to : 
`hive.log.file=hive.log`
Trying to set it also as the previous doesn’t help as `${hive.query.id}` resolves to nothing.

Any advice appreciated.


  [1]: https://issues.apache.org/jira/browse/HIVE-5924

Outcomes