Hive shell errors "java.lang.OutOfMemoryError: Java heap space"

Document created by wade on Feb 27, 2016
Version 1Show Document
  • View in full screen mode

Author: Jitendra Yadav, last modified by Hao Zhu on May 7, 2015

Original Publication Date: May 5, 2015

 

Environment

Hive all releases

Symptom

 

While running any select query from hive shell is throwing OOM errors:

 

hive> select * from test_table limit 1;

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space

  at java.util.Arrays.copyOfRange(Arrays.java:3221)

  at java.lang.String.<init>(String.java:233)

  at java.lang.StringBuffer.toString(StringBuffer.java:602)

  at org.datanucleus.store.rdbms.sql.SQLText.toSQL(SQLText.java:402)

  at org.datanucleus.store.rdbms.request.FetchRequest.<init>(FetchRequest.java:271)

  at org.datanucleus.store.rdbms.RDBMSPersistenceHandler.getFetchRequest(RDBMSPersistenceHandler.java:345)

  at org.datanucleus.store.rdbms.RDBMSPersistenceHandler.fetchObject(RDBMSPersistenceHandler.java:323)

  at org.datanucleus.state.AbstractStateManager.loadFieldsFromDatastore(AbstractStateManager.java:1120)

  at org.datanucleus.state.JDOStateManager.loadSpecifiedFields(JDOStateManager.java:2908)

  at org.datanucleus.state.JDOStateManager.isLoaded(JDOStateManager.java:3214)

Root Cause

 

There could be two issues here either hive shell or hive metastore service is throwing OOM .

 

Solution

 

Increase the JVM heap size through HADOOP_CLIENT_OPTS variable in hive-env.sh or in user environment.

export  HADOOP_CLIENT_OPTS=" -Xmx2048m"

After setting this variable, restart the hive metastore service or hive shell.

Attachments

    Outcomes