AnsweredAssumed Answered

Spark saveAsNewAPIHadoopFile with SequenceFIleOutputFormat does not create output

Question asked by jid1 on Apr 10, 2015
Latest reply on Apr 13, 2015 by jid1

I am reading a bunch of files and create an RDD out of them. Then I use RDD.saveAsNewAPIHadoopFile(...., SequenceFileOutputFormat.class, new Configuration()) to save it back to MapR-fs. I can that the target directory is generated and a _temporary folder created. Once the job is done, the target directory contains only _SUCCESS, but without any files.
I tried the same code on Apache and it works. As there is no error, is there anything that you would recommend for this issue?

I debugged a little further and my WritableComparable.write(DataOutput out) is being called and data are written into it, but no file is created (Other than the _SUCCESS)

PS. Using Spark 1.2.1 with MapR 2.4.1