AnsweredAssumed Answered

Moving selected files from one volume to a second

Question asked by stefanpapp on Feb 23, 2018
Latest reply on Feb 27, 2018 by MichaelSegel

I move files from one MapR volume to another volume. The volumes are located on different nodes. 

 

The files are in a directory tree:

 

DIR 1

-          File 1

-          File 2

-          DIR 2

o   File3

 

The challenge is that only a subset of the files have to be copied. For example file 1 and file 3 could be copied but not file 2. The files that need to be copied are provided by a list.


hadoop disp -f <filelist> <src> <dest> would flatten the hierarchy. One solution is described at https://stackoverflow.com/questions/29077831/how-can-i-execute-hadoop-distcp-f-command-properl.  However, calling hadoop distcp for each directory as described in the stackoverflow article leads to a lot of overhead. What would be the MapR solution to move selective files from A to B. 

Outcomes