How to configure JobTracker High Availability in Hue

Document created by sreedhar Employee on Feb 8, 2016
Version 1Show Document
  • View in full screen mode

Author: Sreedhar Alagonda

 

Original Publication Date: November 26, 2014

 

MapR provides high availability for the JobTracker service. If the active JobTracker fails a configured standby JobTracker is restarted automatically to resume map-reduce job processing with minimal to no interruption.  As Hue is able to launch map-reduce jobs via its various editors and dashboards it is necessary that Hue can submit jobs to the active JobTracker wherever it is running.  Below is the configuration needed for Hue to take advantage of JobTracker high availability.

 

In  /opt/mapr/hue/hue-<version>/desktop/conf/hue.ini , in the section  [[mapred-clusters ]]  define all standby Jobtrackers to support high availability by creating an [[ha]] section for each standby Jobtracker. The following provides an example for defining the current active JobTracker and the standby JobTrackers.

 

[[mapred_clusters]]

 

[[[default]]]

# Enter the host on which you are running the Hadoop JobTracker

jobtracker_host=10.250.50.80 

# The port where the JobTracker IPC listens on

jobtracker_port=9001

# JobTracker logical name.

## logical_name=

# Thrift plug-in port for the JobTracker

## thrift_port=9290

# Whether to submit jobs to this cluster

submit_to=True

 

# Change this if your MapReduce cluster is Kerberos-secured

security_enabled=false

 

# Settings about this MR1 cluster. If you install MR1 in a

# different location, you need to set the following.

 

# Defaults to $HADOOP_MR1_HOME or /usr/lib/hadoop-0.20-mapreduce

hadoop_mapred_home=/opt/mapr/hadoop/hadoop-0.20.2

 

# Defaults to $HADOOP_BIN or /usr/bin/hadoop

hadoop_bin=/opt/mapr/hadoop/hadoop-0.20.2/bin/hadoop

 

# Defaults to $HADOOP_CONF_DIR or /etc/hadoop/conf

hadoop_conf_dir=/opt/mapr/hadoop/hadoop-0.20.2/conf

 

# HA support by specifying multiple clusters

# e.g.

Attachments

    Outcomes