AnsweredAssumed Answered

Kafka Connect distributed throws UnsatisfiedLinkError: com.mapr.fs.jni.MapRClient.initSpoofedUser

Question asked by Géry on Jun 7, 2018
Latest reply on Jun 26, 2018 by Géry

Hi,

 

I got MapR Kafka Connect working on a MapR client in standalone mode with plain Kafka.

But I can't launch MapR Kafka Connect in distributed mode, I get the following error :

2018-06-07 15:50:28,675] ERROR Uncaught exception in herder work thread, exiting: (org.apache.kafka.connect.runtime.distributed.DistributedHerder:218)
java.lang.RuntimeException: Error occurred while invoking com.mapr.streams.impl.admin.MarlinAdminClientImpl:createInternal.
==> java.lang.UnsatisfiedLinkError: com.mapr.fs.jni.MapRClient.initSpoofedUser(Ljava/lang/String;ILjava/lang/String;I)I.
at org.apache.kafka.clients.mapr.GenericHFactory.runMethod(GenericHFactory.java:69)
at org.apache.kafka.clients.admin.AdminClient.create(AdminClient.java:73)
at org.apache.kafka.connect.util.TopicAdmin.<init>(TopicAdmin.java:167)
at org.apache.kafka.connect.storage.KafkaOffsetBackingStore$1.run(KafkaOffsetBackingStore.java:98)
at org.apache.kafka.connect.util.KafkaBasedLog.start(KafkaBasedLog.java:146)
at org.apache.kafka.connect.storage.KafkaOffsetBackingStore.start(KafkaOffsetBackingStore.java:109)
at org.apache.kafka.connect.runtime.Worker.start(Worker.java:144)
at org.apache.kafka.connect.runtime.AbstractHerder.startServices(AbstractHerder.java:100)
at org.apache.kafka.connect.runtime.distributed.DistributedHerder.run(DistributedHerder.java:205)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.apache.kafka.clients.mapr.GenericHFactory.runMethod(GenericHFactory.java:63)
... 13 more
Caused by: java.lang.UnsatisfiedLinkError: com.mapr.fs.jni.MapRClient.initSpoofedUser(Ljava/lang/String;ILjava/lang/String;I)I
at com.mapr.fs.jni.MapRClient.initSpoofedUser(Native Method)
at com.mapr.fs.MapRFileSystem.initConfig(MapRFileSystem.java:541)
at com.mapr.fs.MapRFileSystem.initialize(MapRFileSystem.java:346)
at com.mapr.fs.MapRFileSystem.initialize(MapRFileSystem.java:336)
at com.mapr.streams.impl.admin.MarlinAdminImpl.<init>(MarlinAdminImpl.java:80)
at com.mapr.streams.impl.admin.MarlinAdminClientImpl.<init>(MarlinAdminClientImpl.java:98)
at com.mapr.streams.impl.admin.MarlinAdminClientImpl.createInternal(MarlinAdminClientImpl.java:93)
... 18 more

 

Can MapR Kafka Connect work in distributed mode with plain Kafka as it needs internal topics to work ?

 

In MapR connect-distributed.properties I see :

offset.storage.topic=/var/mapr/.__mapr_connect:offsets
# Flush much faster than normal, which is useful for testing/debugging
offset.flush.interval.ms=10000
config.storage.topic=/var/mapr/.__mapr_connect:configs
status.storage.topic=/var/mapr/.__mapr_connect:status

 

which make me think that MapR Kafka Connect can only work with MapR-ES in distributed mode. Is it correct ?

 

My configuration is :
mapr-client 6.0.1.20180404222005.GA-1 amd64 MapR: Client package
mapr-kafka 1.0.1.201803222306 all MapR: Kafka package
mapr-kafka-connect-hdfs 4.0.0.201804061412

 

Thanks

Outcomes