AnsweredAssumed Answered

PySpark to MaprDB Failure

Question asked by VivekG on Jul 18, 2018
Latest reply on Jul 18, 2018 by VivekG

Hi Team,

I had installed mapr [5.2.2.44680.GA] on my Centos 6.5 single node cluster. All services are up and running.

We were performing an exercise of running a maprstream producer --->>----maprstream consumer --->>---maprdb

As first step wanted to write data to maprdb using pyspark and got the following issue, need your advice as i'm not sure where does the 5.2.1 version is getting exported?

Error

[

2018-07-18 10:26:47,6244 ERROR JniCommon fs/client/fileclient/cc/jni_MapRClient.cc:616 Thread: 5644 Mismatch found for java and native libraries java build version 5.2.1.42646.GA, native build version 5.2.2.44680.GA java patch vserion $Id: mapr-version: 5.2.1.42646.GA 42646:812878ab1269 $, native patch version $Id: mapr-version: 5.2.2.44680.GA 44680:b0ba7dcefd80 $
2018-07-18 10:26:47,6245 ERROR JniCommon fs/client/fileclient/cc/jni_MapRClient.cc:626 Thread: 5644 Client initialization failed.
Traceback (most recent call last):
File "/home/mapr/hadoop/BBI/DEV_VAL/SparkStreaming/spark/PySpark/StreamingMaprDB/mqttKafka2/lib/python3.6/site-packages/maprdb/utils.py", line 98, in wrapper
ret = f(*args, **kwargs)
File "/home/mapr/hadoop/BBI/DEV_VAL/SparkStreaming/spark/PySpark/StreamingMaprDB/mqttKafka2/lib/python3.6/site-packages/maprdb/connection.py", line 79, in create
j_table = self.MapRDB.createTable(name)
jpype._jexception.DBExceptionPyRaisable: com.mapr.db.exceptions.DBException: createTable() failed.,

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/mapr/hadoop/BBI/DEV_VAL/SparkStreaming/spark/PySpark/StreamingMaprDB/mqttKafka2/lib/python3.6/site-packages/maprdb/utils.py", line 101, in wrapper
raise MapRDBError(str(e)) from e
maprdb.utils.MapRDBError: com.mapr.db.exceptions.DBException: createTable() failed.,

]

Codebase:

[

>>> import maprdb
>>> db = maprdb.connect()
>>> db.create('/user/mapr/iot_temp_humid')

]

Kafka Version: 0.9.0

Java Version: openjdk version "1.8.0_171"
OpenJDK Runtime Environment (build 1.8.0_171-b10)
OpenJDK 64-Bit Server VM (build 25.171-b10, mixed mode)

 

Ref: Developing MapR-ES Python Applications 

Outcomes