Running a test of the fair scheduler in YARN with
[myuser@mapr001 user]$ hadoop jar /opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.0-mapr-1710.jar pi 4 100
I see that the scheduler page in the YARN web UI shows that a new queue is created called "root.myuser", which (I think) is the default behavior for the scheduler. However, it should not be using the default scheduler rules from the initial mapr install. My /opt/mapr/hadoop/hadoop-2.7.0/etc/hadoop/fair-scheduler.xml file looks like
[mapr@mapr001 hadoop]$ cat fair-scheduler.xml
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License. See accompanying LICENSE file.
<!-- Note: all queues will be children of the root queue even if root queue not listed -->
<rule name="specified" create="false"/>
<rule name="primaryGroup" create="false"/>
<rule name="default" queue="public"/>
[mapr@mapr001 hadoop]$ echo $MAPR_HOME
So, from this, I would think that the job submitted by "myuser" should be run in a queue called "public" and certainly not a queue named after this user. From the mapr docs (Configuring the Fair Scheduler), the fair-scheduler.xml file should have taken effect around 10 seconds after it was changed, which does not seem to be the case here. Restarting the cluster also does not seem to fix the problem. Does anyone know what is going on here? Thanks.