AnsweredAssumed Answered

MapReduce Accessing / Manipulating HBase tables

Question asked by rthimmana on Jan 4, 2013
Latest reply on Jan 7, 2013 by gera
Hi ALL

In our Development phase, we used the HBase Client to access / manipulate / summarize data from HBase Tables running on MapR. So, the Client and HBase Server are on different systems.

Now, we have to get the same to scale. Doing this in the form of Map-Reduce jobs is ideal as we can cut down on all the network data transfers, mainly while calculating daily summaries (totals, totals by various groupings etc)

**On MapR HBase Server**

While running the Java application from within Eclipse - everything works fine (no Map Reduce)

While running the same code through the MapReduce Framework from within Eclipse, we get a UnsatisfiedLinkError: no MapRClient in java.library.path

Stack Trace
java.lang.UnsatisfiedLinkError: no MapRClient in java.library.path

at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1681)
at java.lang.Runtime.loadLibrary0(Runtime.java:840)
at java.lang.System.loadLibrary(System.java:1047)
at com.mapr.fs.MapRFileSystem.<clinit>(MapRFileSystem.java:1298)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:264)
at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1028)
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1079)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1413)
at org.apache.hadoop.fs.FileSystem.access$100(FileSystem.java:69)
at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:1453)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1435)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:232)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:115)
at org.apache.hadoop.mapred.JobClient.createRPCProxy(JobClient.java:524)
at org.apache.hadoop.mapred.JobClient.init(JobClient.java:502)
at org.apache.hadoop.mapred.JobClient.<init>(JobClient.java:483)
at org.apache.hadoop.mapreduce.Job$1.run(Job.java:489)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:416)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1126)
at org.apache.hadoop.mapreduce.Job.connect(Job.java:487)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:475)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:506)

This is expected as the MapR Client related packages are not installed on the MapR HBase Server Node

Question:
Can the HBase Client and HBase (Server) be installed on the same machine? I remember seeing a Install Notes Warning which discourages this. Please advise?

Outcomes