AnsweredAssumed Answered

mapreduce job configuration  and conversion.

Question asked by jimmy888 on Jul 26, 2011
Latest reply on Aug 8, 2017 by sagarsoni
Hi, There:
  I am trying to get my java based code to work with mapr distribution, I have configuration like this:


        Configuration config = new Configuration();
        config.set("fs.default.name", "hdfs://t-namenodes1.cloud.ppops.net");
        config.set("mapred.job.tracker", "t-namenodes1.cloud.ppops.net:8021");
    
    then I can use FileSystem.get(config) method to get the a FileSystem, and I can use
  JobConf jobConf = new JobConf(configuration);  to let the java code to know which job tracker to submit the job .

With mapr, I realized that I need to substitute the hadoop core jar file and put the maprfs-0.1.jar in my java path, and I also need to change the URL to maprfs://t-namenodes1.cloud.ppops.net:7222  and
t-namenodes1.cloud.ppops.net:9001 . however, after the change, I still can't successfully run the job. The FileSystem.get() method  seems not to recognice the maprfs:// url.

Does anybody know what I should do to convert the apache based mapreduce job to mapr based job ?



Outcomes