AnsweredAssumed Answered

Hive - java.lang.RuntimeException: java.lang.StackOverflowError

Question asked by sureshgopalan on Dec 14, 2017
Latest reply on Jan 10, 2018 by maprcommunity

Hi,

 

We ran a query like below in hive cli

UNION select count(*) from QA_CDS_JDA.T_POC_TN1POLNR where CNTL_CATG_IND  like '%?%'
UNION select count(*) from QA_CDS_JDA.T_POC_TN1POLNR where STA_CODE  like '%?%'
UNION select count(*) from QA_CDS_JDA.T_POC_TN1POLNR where ENS_COOR_CD  like '%?%'
UNION select count(*) from QA_CDS_JDA.T_POC_TN1POLNR where CREATE_TIMESTAMP  like '%?%'
UNION select count(*) from QA_CDS_JDA.T_POC_TN1POLNR where UPDATE_TIMESTAMP  like '%?%'

........like this there are 692 lines

 

Query failed when doing Semantic Analysis

2017-12-14 11:38:01,756 INFO  [HiveServer2-Handler-Pool: Thread-29297]: parse.ParseDriver (ParseDriver.java:parse(209)) - Parse Completed
2017-12-14 11:38:01,757 INFO  [HiveServer2-Handler-Pool: Thread-29297]: log.PerfLogger (PerfLogger.java:PerfLogEnd(148)) - </PERFLOG method=parse start=1513269481722 end=1513269481757 duration=35 from=org.apache.hadoop.hive.ql.Driver>
2017-12-14 11:38:01,758 INFO  [HiveServer2-Handler-Pool: Thread-29297]: log.PerfLogger (PerfLogger.java:PerfLogBegin(121)) - <PERFLOG method=semanticAnalyze from=org.apache.hadoop.hive.ql.Driver>
2017-12-14 11:38:01,758 INFO  [HiveServer2-Handler-Pool: Thread-29297]: parse.CalcitePlanner (SemanticAnalyzer.java:analyzeInternal(10203)) - Starting Semantic Analysis
2017-12-14 11:38:01,803 INFO  [HiveServer2-Handler-Pool: Thread-29297]: log.PerfLogger (PerfLogger.java:PerfLogEnd(148)) - </PERFLOG method=compile start=1513269481721 end=1513269481803 duration=82 from=org.apache.hadoop.hive.ql.Driver>
2017-12-14 11:38:01,803 WARN  [HiveServer2-Handler-Pool: Thread-29297]: thrift.ThriftCLIService (ThriftCLIService.java:ExecuteStatement(497)) - Error executing statement:
java.lang.RuntimeException: java.lang.StackOverflowError
        at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:83)
        at org.apache.hive.service.cli.session.HiveSessionProxy.access$000(HiveSessionProxy.java:36)
        at org.apache.hive.service.cli.session.HiveSessionProxy$1.run(HiveSessionProxy.java:63)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1595)
        at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:59)
        at com.sun.proxy.$Proxy10.executeStatementAsync(Unknown Source)
        at org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:274)
        at org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:491)
        at org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1317)
        at org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1302)
        at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
        at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
        at org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:56)
        at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:285)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.StackOverflowError
        at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.doPhase1QBExpr(SemanticAnalyzer.java:426)
        at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.doPhase1QBExpr(SemanticAnalyzer.java:441)
        at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.processSubQuery(SemanticAnalyzer.java:917)
        at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.doPhase1(SemanticAnalyzer.java:1258)
        at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.doPhase1(SemanticAnalyzer.java:1462)
        at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.doPhase1QBExpr(SemanticAnalyzer.java:430)
        at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.doPhase1QBExpr(SemanticAnalyzer.java:441)
        at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.processSubQuery(SemanticAnalyzer.java:917)

 

 

We checked it by running EXPLAIN.  Same error. Query worked when there are fewer lines ( 550)

How to resolve this error? Can we tune some settings in hive ?

Outcomes