How to add auxiliary JARs into Hive

Document created by Hao Zhu Employee on Feb 18, 2016
Version 1Show Document
  • View in full screen mode

Author: Hao Zhu

Original Publication Date: March 25, 2015

Goal:

Sometimes we need to add auxiliary JARs into Hive classpath.For example, using customized SERDE JARs or Hive integration with other applications.This article shows how to add JARs into Hive classpath.

Solution:

Firstly the JAR files should be accessible by all hadoop nodes.

1. Hive server(HS1 or HS2)

Modify /opt/mapr/hive/hive-<version>/conf/hive-site.xml to add parameter hive.aux.jars.path.

 

The value is comma separated.

<property>

  <name>hive.aux.jars.path</name>

  <value>/path_to_jar,/path_to_jar2</value>

</property>

For example:

<property>

  <name>hive.aux.jars.path</name>

  <value>/opt/mapr/hive/hive-0.13/aux/hive-json-serde-0.2.jar</value>

</property>

And then restart the Hive server.

For HS1, kill the hive process and :

nohup hive --service hiveserver &

For HS2:

maprcli node services -name hs2 -action stop -nodes <hostname> 
maprcli node services -name hs2 -action start -nodes <hostname>

2.Hive shellHere are 3 ways.a. Add the JAR in hive shell:

ADD JAR <jar_path>;

For example:

hive> ADD JAR /opt/mapr/hive/hive-0.13/aux/hive-json-serde-0.2.jar;

Added /opt/mapr/hive/hive-0.13/aux/hive-json-serde-0.2.jar to class path

Added resource: /opt/mapr/hive/hive-0.13/aux/hive-json-serde-0.2.jar

hive> list jars;

/opt/mapr/hive/hive-0.13/aux/hive-json-serde-0.2.jar

b. Use option "--auxpath":

hive --auxpath <path_to_jar>,<path_to_jar2>

For example:

hive --auxpath /opt/mapr/hive/hive-0.13/aux/hive-json-serde-0.2.jar

c. Use environment variable HIVE_AUX_JARS_PATH before launching hive command:

export HIVE_AUX_JARS_PATH=<path_to_jar>,<path_to_jar2>

For example:

export HIVE_AUX_JARS_PATH=/opt/mapr/hive/hive-0.13/aux/hive-json-serde-0.2.jar

Attachments

    Outcomes