AnsweredAssumed Answered

Error while using HDFS connector

Question asked by Sanjay0004 on Jan 16, 2018
Latest reply on Feb 7, 2018 by maprcommunity

Hi All,

 

I am trying to stream the data from MapR-ES to Hive Database using kafka-connect-hdfs (HDFS connector and Hive Integration). Below is the quickstart-hdfs.properties file parameters:

 

name=hdfs-connector-hive

hive.integration=true

hive.database=db3

hive.conf.dir=/opt/mapr/hive/hive-2.1/conf

hive.metastore.uris=thrift://localhost:9083

schema.compatibility=BACKWARD

connector.class=io.confluent.connect.hdfs.HdfsSinkConnector

tasks.max=1

#topics=test_hdfs

topics=/TABLE_EVENT:TOPIC

hdfs.url=maprfs:///

#format.class=io.confluent.connect.hdfs.parquet.ParquetFormat

flush.size=1

#rotate.interval.ms=1000

 

I have taken reference from this link:

https://maprdocs.mapr.com/home/Kafka/Connect-hdfs-example-fromKafka.html

 

I am running into error, below is the attached screenshot of error logs:

 

 

I can see the table created in the Hive Database but table is empty. Please have a look, and let me know if I am missing something, and how to resolve this issue.

 

Best Regards,

Sanjay

Outcomes