AnsweredAssumed Answered

LocalCluster with access to real MapR

Question asked by mistanke on Mar 30, 2017
Latest reply on Apr 1, 2018 by vaibhav7994

Hi.

 

In our Storm topology, some bolts need access to MapR wide column tables. For testing in the LocalCluster we need to add jaas.conf and ensure the right configuration before first Zookeeper client is started using this code:

static {     beforeStormClusterStart(); }  public static void beforeStormClusterStart() {     URL jaasConfFile = StormTestUtil.class.getClassLoader().getResource("jaas.conf");     System.setProperty("java.security.auth.login.config", jaasConfFile.toExternalForm());          Configuration.getConfiguration();     System.setProperty("zookeeper.sasl.client", "false"); }

The jaas.conf looks like this:

hadoop_simple {   org.apache.hadoop.security.login.GenericOSLoginModule required;   org.apache.hadoop.security.login.HadoopLoginModule required; };

We have hbase-client and hbase-common in version 0.98.12-mapr-1602-m7-5.1.0, corresponding to the mapr cluster installed in 5.1.0. Also we are forcing zookeeper dependency in version 3.4.6, as newer versions break the LocalCluster too. The hadoop-auth is in version 2.7.0-mapr-1602 coming from the hbase-client.

 

It works well when using storm-core 0.9.6, but fails when trying to upgrade the dependency to 1.0.3. Preserving this configuration, we see an error from either from mapr client unable to access the cluster (when the code above is omitted), or 'InvalidACL for /storm' returned from the LocalCluster in memory Zookeeper, when the configuration is the above.

 

Does anyone have an experience with running storm 1.0.x LocalCluster with topology accessing real mapr cluster? Or better run into this issue and solved it?

Outcomes