AnsweredAssumed Answered

Mapr streams: adjusting kafka producer timeout when cluster is unreachable.

Question asked by arp4504 on Aug 30, 2017
Latest reply on Sep 25, 2017 by maprcommunity

Hi,

 

We want to reduce mapr stream producer timeouts when the mapr cluster is unreachable. We made the client unreachable from the cluster using iptables rules.  However, we see that despite reducing values for kafka producer configuration parameters (like "message.timeout.ms", "request.timeout.ms" and other parameters given here : librdkafka/CONFIGURATION.md at master · edenhill/librdkafka · GitHub  )  each message is timing out after a fixed time period (currently, a little over a 6 minutes). Attaching timeout logs.   

 

2017-08-30 11:26:19,7124 ERROR Cidcache fs/client/fileclient/cc/cidcache.cc:2192 Thread: 18748 MoveToNextCldb: No CLDB entries, cannot run, sleeping 5 seconds!
2017-08-30 11:26:24,7127 ERROR Client fs/client/fileclient/cc/client.cc:1205 Thread: 18748 Failed to initialize client for cluster randomhost.com, error Connection reset by peer(104)
2017-08-30 11:29:33,7404 ERROR Cidcache fs/client/fileclient/cc/cidcache.cc:2192 Thread: 18748 MoveToNextCldb: No CLDB entries, cannot run, sleeping 5 seconds!
2017-08-30 11:29:38,7406 ERROR Client fs/client/fileclient/cc/client.cc:1205 Thread: 18748 Failed to initialize client for cluster randomhost.com, error Connection reset by peer(104)
2017-08-30 11:29:38,7669 ERROR StreamsClient fs/client/marlin/cc/marlinclient.cc:291 Thread: 18748 Cannot open cluster,randomhost.com
2017-08-30 11:29:38,7669 ERROR StreamsClient fs/client/marlin/cc/producer/marlinproducer.cc:511 Thread: 18748 FetchTopicMetadatAndPossiblyCreateTopic: error opening stream /MapR_Random_Streams (22)
2017-08-30 11:29:38,7669 ERROR StreamsClient fs/client/marlin/cc/producer/marlinproducer.cc:252 Thread: 18748 Cannot get number of partitions, err 22
2017-08-30 11:29:38,7669 ERROR StreamsProducer fs/client/marlin/cc/api/streams_producer.cc:112 Thread: 18748 fetching topic /MapR_Random_Streams:MapR-RandomTopic metadata failed (err 22)

 

We referred - Developing a MapR Streams C Application for our producer program. 

Is there any way to reduce message timeout in such a case?

 

Note : We can reduce the timeout to approximately 1 minute, by setting connection parameter "fs.mapr.rpc.timeout" in JAVA client. However, this parameter does not work in mapr stream C client. We tried using hdfsSetRpcTimeout() method to set this parameter in C and change "fs.mapr.rpc.timeout" in core-site.xml, as shown here :Configuring RPC timeouts in MapR FS  but that is also not working. Is there a way to set this parameter in kafka api?

 

Software versions:

 

Apache kafka API version : 0.0.9

Mapr version : 5.2.2

 

 

mapr_streams kafka api mapr-streaming mapr-fs apache kafka kafka streams

Outcomes