We are successfully reading a Kafka topic and writing it to HDFS. It’s creating a lot of files, we want to change this behavior using the flush parameter. If we use any other setting then 1, it fails.
The stacktrace reports it’s trying to open the same temp file twice. Anybody have a solution to this problem?
Thanks in advance for any help.
Kafka 1.0.1 (MEP 5.0)
Kafka Connect HDFS 4.0.0 (MEP 5.0)