AnsweredAssumed Answered

Hive MaprDB(HBase) Integrated table using Serde - group by error

Question asked by jayshewale on May 3, 2016
Latest reply on Jul 13, 2016 by Karen Whipple

I am getting following error when on Hive -MapR-DB integrated table using serde. if i do select * it works file but it fails when i am trying to do any aggregation, order by, etc operations. any help will be appreciated.

select columna,count(*) from tablename group by columna ;

I get an exception as below

Error: java.lang.NullPointerException

at org.apache.hadoop.hive.serde2.ColumnProjectionUtils.toReadColumnIDString(ColumnProjectionUtils.java:149)

at org.apache.hadoop.hive.serde2.ColumnProjectionUtils.appendReadColumns(ColumnProjectionUtils.java:89)

at org.apache.hadoop.hive.serde2.ColumnProjectionUtils.appendReadColumns(ColumnProjectionUtils.java:102)

at org.apache.hadoop.hive.ql.io.HiveInputFormat.pushProjectionsAndFilters(HiveInputFormat.java:480)

at org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:234)

at org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getRecordReader(CombineHiveInputFormat.java:573)

at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.<init>(MapTask.java:172)

at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:414)

at org.apache.hadoop.mapred.MapTask.run(MapTask.java:347)

at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)

at java.security.AccessController.doPrivileged(Native Method)

at javax.security.auth.Subject.doAs(Subject.java:422)

at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1566)

at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)

The hive table is uses org.apache.hadoop.hive.hbase.HBaseSerDe and storage handler is org.apache.hadoop.hive.hbase.HBaseStorageHandler. All the Columns are defined as string in the create definition.

Is this because the serde is not able to convert the value(May be Null values etc) ? I tried to put a Null check as well as below which again gave me the same error

select columna,count(*) from tablename group by columna having columna<> NULL ;

Also i could see in the job log some extra info as below

  1. java.lang.NoSuchMethodException: <unbound>=ReduceSinkDesc.getReducerTraits();

java.lang.IllegalStateException: The outer element does not return value

java.lang.IllegalStateException: The outer element does not return value

java.lang.NullPointerException: target should not be null

java.lang.NoSuchMethodException: <unbound>=TableScanDesc.setNeededColumnIDs(ArrayList);

java.lang.NoSuchMethodException: <unbound>=TableScanDesc.setNeededColumns(ArrayList);

java.lang.NoSuchMethodException: <unbound>=TableScanOperator.setReferencedColumns(Ar

Outcomes