AnsweredAssumed Answered

Table Join exception on Hive 0.9.0

Question asked by pmenon on Feb 26, 2013
Latest reply on Mar 3, 2013 by prasadb
I have two external Hive tables using different deserializers that, when I perform any join between them, throws an exception.  One table was brought in through Sqoop, the other is a one created through flume.  Using Hive 0.9.0 on M3 and [this][1] answer didn't fix the issue.  Can anyone help?

The Sqoop command I used to bring in a SQL table:
<pre>
sqoop import --connect "jdbc:sqlserver://..." --username user -P --verbose --warehouse-dir maprfs:///data/tables --num-mappers 1 -table my_table</pre>


The syntax of the join query:
<pre>
select * from
my_table_one one
inner join my_table_two two on two.id = one.id;
</pre>

The exception:
<pre>
java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row {"key1":"","key2":"","datecreated":"2012-12-10 15:54:14.59"}
at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:161)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:405)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:336)
at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1126)
at org.apache.hadoop.mapred.Child.main(Child.java:264)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row {"key1":"","key2":"","datecreated":"2012-12-10 15:54:14.59"}
at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:548)
at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:143)
... 8 more
Caused by: java.lang.ClassCastException: org.apache.hadoop.hive.serde2.lazy.LazyStruct cannot be cast to [Ljava.lang.Object;
at org.apache.hadoop.hive.serde2.objectinspector.UnionStructObjectInspector.getStructFieldsDataAsList(UnionStructObjectInspector.java:141)
at org.apache.hadoop.hive.ql.exec.LateralViewJoinOperator.processOp(LateralViewJoinOperator.java:128)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762)
at org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:70)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762)
at org.apache.hadoop.hive.ql.exec.LateralViewForwardOperator.processOp(LateralViewForwardOperator.java:37)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762)
at org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:83)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762)
at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:529)
... 9 more
</pre>


  [1]: http://answers.mapr.com/questions/3965/table-join-issue-after-upgrade-to-hive-090

Outcomes