AnsweredAssumed Answered

Increasing the speed at which blocks are replicated

Question asked by davidjericho on Apr 15, 2013
Latest reply on Apr 16, 2013 by davidjericho
I've got a cluster that's taking some time to replicate data. At most, it appears I'm doing about 10MByte/s while replicating to a recently added node.

Between the nodes using either a web download of a file or iperf, I get 10Gbits/s throughput reliably so bandwidth isn't the issue. Speed when writing to the MapR NFS server on the localhost, and over a network is fast as well.

The cluster appears to be be transferring at about 600 kbyte/s to 1 Mbyte/s per replication thread to a new node, and my mfs.log is full of messages like:

2013-04-09 00:02:42,3559 INFO x.x.0.0:0 Returning EAGAIN for FSID 2094340271159275139,,  replicacid 2617, srccid 2617 because container need slow resync and all slowresync slots are full

Looking at network monitoring tools, and even the node information, the node is replicating at roughly 16 times 1MB/s, give or take.

I know in non-MapR Hadoop land, hdfs-site.xml has a setting that can be modified to increase this number, but MapR M5 does not have this file.

Can I increase the concurrency to get aggregate throughput, increase the throughput put thread, or do both to get throughput? I have plenty of network whitespace I'd like to see utilised, as fast replication is of interest.