AnsweredAssumed Answered

Copy Task Fails

Question asked by shp on Nov 10, 2014
Hi ,

I am having copy task which copies files( using DistCp) from one bucket to another bucket. After file copied , file is deleted from src path.
However, in my job copy task fails and file is not getting deleted from src path. Due to this reason file gets processed twice and creates duplicate records. In copy task its giving me  below exception :

java.io.IOException: Copied: 16 Skipped: 0 Failed: 1
at com.gfk.dm.api.helper.DistCpInput$CopyFilesMapper.close(DistCpInput.java:654)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:57)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:441)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:377)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
at org.apache.hadoop.mapred.Child.main(Child.java:249)


Please suggest me.

Outcomes