AnsweredAssumed Answered

java.io.IOException: Could not create FileClient

Question asked by karthikSpark on Feb 17, 2017
Latest reply on Apr 24, 2017 by maprcommunity

Hi all,

I'm trying to persist my data frame in to a different host using 

i have added both the clusters to mapr.clusters.conf file just in case.

val conf = new Configuration()
conf.set("fs.defaultFS","maprfs://another-host:7222")
val filepath = "/mfs/filepath/";
val fs =FileSystem.get(URI.create(filepath), conf);

 

myDF.write.format("parquet").mode(org.apache.spark.sql.SaveMode.Append).save("maprfs://another-host.com:7222//3_0/records")

This works fine when i do it in spark shell.

But when i perform the same with spark-submit the above error comes up.

How to solve this, my spark application is in scala-2.10.Mapr-5.2,spark-1.6.1 versions.

Community Manager bgajjela Raju Bairishetti. debjyotiroy _  Artur Sukhenko

I have found some solution in the mapr community, java.io.IOException: Could not create FileClient but dont know how to use this in my scala application.

Any help can will be appreciated.

thanks in advance.

Outcomes