AnsweredAssumed Answered

DistCP issues between CDH3U0 -> MapR M5

Question asked by jcarroll on Sep 8, 2011
Latest reply on Sep 9, 2011 by jcarroll
I'm attempting to move our Hive Data Warehouses from our current CDH3 cluster to MapR for testing. I am not able to move the files successfully after reading the documentation. This is the command that I run, and the error that I receive.
COMMAND
----
hadoop distcp -i "hftp://hadoop.colo:50070/user/hive/warehouse/media/authored_month=2011-04" "maprfs://192.168.200.148:7222/user/hivwarehouse/media/authored_month=2011-04"

ERROR
----
11/09/08 16:15:28 INFO mapred.JobClient: Task Id : attempt_201109071638_0009_m_000022_0, Status : FAILED on node dn18-r175u9.colo.
java.io.IOException: Can not get the relative path: base = maprfs://192.168.200.148:7222/user/hive/warehouse/media/_distcp_logs_g4k9wn/_temporary/_attempt_201109071638_0009_m_000022_0 child = maprfs://192.168.200.146:7222/user/hive/warehouse/media/_distcp_logs_g4k9wn/_temporary/_attempt_201109071638_0009_m_000022_0/part-00022
        at org.apache.hadoop.mapred.FileOutputCommitter.getFinalPath(FileOutputCommitter.java:196)
        at org.apache.hadoop.mapred.FileOutputCommitter.moveTaskOutputs(FileOutputCommitter.java:152)
        at org.apache.hadoop.mapred.FileOutputCommitter.moveTaskOutputs(FileOutputCommitter.java:172)
        at org.apache.hadoop.mapred.FileOutputCommitter.commitTask(FileOutputCommitter.java:132)
        at org.apache.hadoop.mapred.OutputCommitter.commitTask(OutputCommitter.java:221)
        at org.apache.hadoop.mapred.Task.commit(Task.java:1011)
        at org.apache.hadoop.mapred.Task.done(Task.java:875)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:329)
        at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:396)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1074)
        at org.apache.hadoop.mapred.Child.main(Child.java:262)

11/09/08 16:15:29 INFO mapred.JobClient:  map 98% reduce 0%
...

Outcomes