AnsweredAssumed Answered

Debugging slow puts to MapR-DB

Question asked by gesgeorge on Sep 5, 2016
Latest reply on Sep 14, 2016 by gesgeorge

I'm trying to track down the reason behind the sudden drop in put rates to two Mapr-DB tables defined in the same volume. I was very surprised by the sharp drop and I would like to understand how I can trace the cause.

 

When I see that writes are slow in my application, I see logs indicating lots of pending flushes and HighLatency writes in mfs.log-5 in /opt/mapr/logs. I'm not sure what to make of the pending flushes since its not clear what's the underlying cause.

I did some tests on my cluster now and I'm not seeing any such pending flushes in the logs and the write rate is pretty good. (~32k puts per sec across the cluster). In this small test cluster (4 nodes), I've seen somewhere between 50 - 60k puts per sec across the cluster when I've a fire hose of events being written to the cluster.

 

Could you point me to what I should be looking for to track this done? I've attached the mfs log from the time of the issue. See logs for 2016-09-02. This drop in puts was the reason I raised What is the impact of using more than one Async HBase client?

Attachments

Outcomes