How to switch Impala file client from MapR C++ Client to Hadoop JAVA Client

Document created by Hao Zhu Employee on Feb 18, 2016
Version 1Show Document
  • View in full screen mode

Author: Hao Zhu

Original Publication Date: February 6, 2015

 

Env:

Impala 1.4.1MapR 4.0.1 with Hadoop 2.4.1

Goal:

Impala on MapR uses MapR C++ file client by default.For example:

/opt/mapr/impala/impala-1.4.1/lib/libhdfs.so -> /opt/mapr/lib/libMapRClient.so

/opt/mapr/impala/impala-1.4.1/lib/libhdfs.so.0 -> /opt/mapr/lib/libMapRClient.so

/opt/mapr/impala/impala-1.4.1/lib/libhdfs.so.0.0.0 -> /opt/mapr/lib/libMapRClient.so

For troubleshooting purpose, sometimes we need to change the file client to Hadoop java file client as below:

/opt/mapr/impala/impala-1.4.1/lib/libhdfs.so -> /opt/mapr/hadoop/hadoop-2.4.1/lib/native/libhdfs.so

/opt/mapr/impala/impala-1.4.1/lib/libhdfs.so.0 -> /opt/mapr/hadoop/hadoop-2.4.1/lib/native/libhdfs.so

/opt/mapr/impala/impala-1.4.1/lib/libhdfs.so.0.0.0 -> /opt/mapr/hadoop/hadoop-2.4.1/lib/native/libhdfs.so

Solution:

Firstly stop all impala services, and then here are 2 shell scripts to change the file client to:1. MapR C++ File Client

clush -a rm /opt/mapr/impala/impala-1.4.1/lib/libhdfs.so

clush -a rm /opt/mapr/impala/impala-1.4.1/lib/libhdfs.so.0

clush -a rm /opt/mapr/impala/impala-1.4.1/lib/libhdfs.so.0.0.0

clush -a ln -s /opt/mapr/lib/libMapRClient.so /opt/mapr/impala/impala-1.4.1/lib/libhdfs.so

clush -a ln -s /opt/mapr/lib/libMapRClient.so /opt/mapr/impala/impala-1.4.1/lib/libhdfs.so.0

clush -a ln -s /opt/mapr/lib/libMapRClient.so /opt/mapr/impala/impala-1.4.1/lib/libhdfs.so.0.0.0

clush -a ls -altr /opt/mapr/impala/impala-1.4.1/lib/libhdfs.so*

2. Hadoop JAVA File Client

clush -a rm /opt/mapr/impala/impala-1.4.1/lib/libhdfs.so

clush -a rm /opt/mapr/impala/impala-1.4.1/lib/libhdfs.so.0

clush -a rm /opt/mapr/impala/impala-1.4.1/lib/libhdfs.so.0.0.0

clush -a ln -s /opt/mapr/hadoop/hadoop-2.4.1/lib/native/libhdfs.so /opt/mapr/impala/impala-1.4.1/lib/libhdfs.so

clush -a ln -s /opt/mapr/hadoop/hadoop-2.4.1/lib/native/libhdfs.so /opt/mapr/impala/impala-1.4.1/lib/libhdfs.so.0

clush -a ln -s /opt/mapr/hadoop/hadoop-2.4.1/lib/native/libhdfs.so /opt/mapr/impala/impala-1.4.1/lib/libhdfs.so.0.0.0

clush -a ls -altr /opt/mapr/impala/impala-1.4.1/lib/libhdfs.so*

Note: Please always consult MapR support before taking above actions.

Attachments

    Outcomes