AnsweredAssumed Answered

MapR Posix Client is very slow when copying data to MapR cluster

Question asked by hphan on Jul 28, 2016
Latest reply on Mar 7, 2018 by maprcommunity


Hi all,

I have installed MapR Posix Client on the node outside of our MapR cluster (version 5.1) and then tried to copy the data from the client node to the cluster. However, the performance is so slow.

The tasks I have done are:

1- install MapR client and MapR Posix client on the client node
2- mount the Hadoop file system to the client node, at /mapr/<cluster-name>

3- copy data from client node to MapR cluster by the command cp, such as:

cp -R a-folderD-in-client /mapr/<cluster-name>/folderA/

 

However, the performance for this copy is very slow. It took us closed to 6h hours to finish the copy of 300GB folder

This is contrast with the use Hadoop fs copy command. In the client node, if we you: hadoop fs -copyFromLocal a-folderD-in-client /folderA, to copy the same folder to MapR cluster, the performance is very good with only 50 minutes.

 

I have no clues for this. Could you please give me some advice about this?

 

Thanks and regards,

 

 

Hien Phan.

Outcomes