can we use linux mapr-client for running distcp aganist the multiple clusters, if yes please provide what changes we need to do this.
Hello Bharath. That being your case, you'd typically add something like below in the /opt/mapr/conf/mapr-clusters.conf on your MapR client node,
a.cluster.com secure=false cldb-a:7222b.cluster.com secure=false cldb-b:7222c.cluster.com secure=false cldb-c:7222d.cluster.com secure=false cldb-d:7222
Here a.cluster.com will be your default cluster, i.e., any client commands like 'hadoop fs -ls' will land onto the cluster to execute the command if the cluster name is not explicitly mentioned in the syntax.
Also, you may want to be aware of the client compatibility matrix defined at Hadoop Protocol Versions for MapR Software.
Hope that helps.
Invite Jimit J. Shah to help to answer.
/opt/mapr/conf/mapr-clusters.conf (mapr-clusters.conf ) is where the cluster definitions would go in, the top level entry being considered as the default cluster to connect to. Were you looking for an answer beyond that? If so, please share more details on what is currently not working for you.
Hi Mufeed Usman
Thanks for the response. We have multiple clusters A( Mapr 5.0),B( Mapr 5.1),C( Mapr 5.2),D( Mapr 5.2) which are not secure clusters.
We want to use mapr-client node to run distcp jobs. Do we need to add clusters A( Mapr 5.0),B( Mapr 5.1),C( Mapr 5.2),D( Mapr 5.2) information in mapr-client node or anything else is required.
I hope this explains our use case.
Hello Mufeed Usman ,
can we run distcp jobs on client node to Cluster A -> B, C->D without adding Cluster A information in Cluster B(/opt/mapr/conf/mapr.cluster.conf), Cluster C information in Cluster D (/opt/mapr/conf/mapr.cluster.conf) i.e.,
for example :
clientnode# hadoop distcp maprfs:///mapr/A/TMP2 maprfs/B/TMP2
clientnode# hadoop distcp maprfs:///mapr/C/TMP2 maprfs/D/TMP2
The client node's .conf should contain both the cluster info. The target or destination need not have info about the source.
Thank you Shishir Prakash for confirming.
Retrieving data ...