AnsweredAssumed Answered

Jobs not getting submitted to yarn

Question asked by harinder on Sep 24, 2014
Latest reply on Sep 25, 2014 by harinder
Hi, i am working with Mapr 4.0.1 and when i submit the Job from my code its getting submitted to the old job tracker and not yarn. I am setting the following also in my `Configuration` object:-

    conf.addResource(new Path(parentDirPath + "core-site.xml"));
    conf.addResource(new Path(parentDirPath + "hdfs-site.xml"));
    conf.addResource(new Path(parentDirPath + "mapred-site.xml"));
    conf.set("mapreduce.framework.name", "yarn");
    conf.set("yarn.resourcemanager.address", "hostname:8032");
    conf.set("fs.maprfs.impl", "com.mapr.fs.MapRFileSystem");

The files i am picking are at the following location:-
/opt/mapr/hadoop/hadoop-2.4.1/etc/hadoop

But i suppose only `mapred-site.xml` is useful in this case as it contains the following

    <property>
        <name>mapreduce.jobhistory.address</name>
        <value>myIP:10020</value>
      </property>
      <property>
        <name>mapreduce.jobhistory.webapp.address</name>
        <value>myIP:19888</value>
      </property>

Rest of the files are empty. And the there is no yarn-site.xml at this location.

One thing i noticed is that the Jobconf  already has the below values:-

    conf.get("fs.default.name")--- "maprfs:///
    conf.get("mapred.job.tracker")--- "maprfs:///

Would these be causing the Job not being submitted to YARN? As these values loaded by default configuration, I think or i am missing something here.

Outcomes