AnsweredAssumed Answered

Proper way to do distcp via maprfs

Question asked by mmercer on Aug 21, 2013
Latest reply on Aug 22, 2013 by Ted Dunning
We have one single cluster running 2 separate volumes (data and depot).  I am attempting to do a distcp between the two.  The cluster name is dev.

Now in the mapr documentation, this is accomplished via:

hadoop distcp -p -update "maprfs://</cluster></path>" "maprfs://</cluster></path>"

In our given case, I am trying:

    root@dhm0:~# hadoop distcp "maprfs:///data/exports3/r/depot" "maprfs:///depot"
    13/08/21 14:22:50 INFO tools.DistCp: srcPaths=[maprfs:/data/exports3/r/depot]
    13/08/21 14:22:50 INFO tools.DistCp: destPath=maprfs:/depot
    13/08/21 14:22:50 INFO fs.JobTrackerWatcher: Current running JobTracker is: dhm0.quantifind.com/10.10.5.189:9001

As you can see, it does suggest that its been handed off to the jobtracker... But the jobtracker never actually gets it.  I believe this is because the distcp itself is being done wrong, but I am very limited in being able to troubleshoot from here.

I have of course checked the /opt/mapr/hadoop/hadoop-0.20.2/logs/jobtracker logs, and I do not even see it, so at this point, I am stuck.

Any guidance would be great.  We are running version 1.29

Thanks

Outcomes