AnsweredAssumed Answered

Select Column query in Hive CLI not working, Resource Manager hangs.

Question asked by vishal on Apr 29, 2015
Latest reply on Nov 20, 2015 by sandeep.dolia
We have MapR single node cluster running, we want to execute a select column query on a table.
Upon executing the query on Hive CLI,it just hangs, when we searched in Resource Manager logs it paused with message  State change fro
m SUBMITTED to SCHEDULED , we have solved the same problem in CDH by increasing
yarn.nodemanager.resource.memory-mb property. But we are unable to achieve the same in MapR.

Following are the details-


    hive> select zoneid from zones;
    Total jobs = 1
    Launching Job 1 out of 1
    Number of reduce tasks is set to 0 since there's no reduce operator
    Starting Job = job_1430318464056_0001, Tracking URL = http://ip-10-38-28-231:808
    8/proxy/application_1430318464056_0001/
    Kill Command = /opt/mapr/bin/hadoop job  -kill job_1430318464056_0001

yarn-mapr-resourcemanager-ip-10-38-28-31.log

    2015-04-29 10:57:24,552 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp
    .attempt.RMAppAttemptImpl: appattempt_1430319251078_0001_000001 State change fro
    m NEW to SUBMITTED
    2015-04-29 10:57:24,552 INFO org.apache.hadoop.yarn.server.resourcemanager.sched
    uler.fair.FairScheduler: Added Application Attempt appattempt_1430319251078_0001
    _000001 to scheduler from user: ec2-user
    2015-04-29 10:57:24,552 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp
    .attempt.RMAppAttemptImpl: appattempt_1430319251078_0001_000001 State change fro
    m SUBMITTED to SCHEDULED
    001

Settings in yarn-site.xml

    <configuration>
      <!-- Resource Manager MapR HA Configs -->
      <property>
        <name>yarn.resourcemanager.ha.custom-ha-enabled</name>
        <value>true</value>
        <description>MapR Zookeeper based RM Reconnect Enabled. If this is true, set
     the failover proxy to be the class MapRZKBasedRMFailoverProxyProvider</description>
      </property>
      <property>
        <name>yarn.client.failover-proxy-provider</name>
        <value>org.apache.hadoop.yarn.client.MapRZKBasedRMFailoverProxyProvider</value>
        <description>Zookeeper based reconnect proxy provider. Should be set if and
    only if mapr-ha-enabled property is true.</description>
      </property>
      
      <property>
        <name>yarn.resourcemanager.recovery.enabled</name>
        <value>true</value>
        <description>RM Recovery Enabled</description>
      </property>
    
      <property>
           <name>yarn.scheduler.minimum-allocation-mb</name>
           <value>2048</value>
           <description>Minimum limit of memory to allocate to each container reques
    t at the Resource Manager.</description>
       </property>
    
          <property>
           <name>yarn.scheduler.maximum-allocation-mb</name>
           <value>4096</value>
           <description>Maximum limit of memory to allocate to each container request at the Resource Manager.</description>
          </property>
    
       <property>
           <name>yarn.nodemanager.resource.memory-mb</name>
           <value>8192</value>
           <description>Physical memory, in MB, to be made available to running cont
    ainers</description>
       </property>
    <property>
    <name>yarn.app.mapreduce.am.resource.mb</name>
    <value>2048</value>
    </property>
    </configuration>





Outcomes