AnsweredAssumed Answered

MapR Kafka Connect HDFS flush issues

Question asked by EmielKremers on Jul 12, 2018

Hi all,

We are successfully reading a Kafka topic and writing it to HDFS. It’s creating a lot of files, we want to change this behavior using the flush parameter. If we use any other setting then 1, it fails.

 

The stacktrace reports it’s trying to open the same temp file twice. Anybody have a solution to this problem?

 

Thanks in advance for any help.

 

Kafka 1.0.1 (MEP 5.0)

Kafka Connect HDFS 4.0.0 (MEP 5.0)

Outcomes