Running multiple httpfs servers on multiple nodes on cluster

Document created by sreedhar Employee on Feb 8, 2016
Version 1Show Document
  • View in full screen mode

Author: Sreedhar Alagonda

 

Original Publication Date: April 28, 2015

 

ISSUE:

 

At present  we can not run multiple httpfs servers on  the cluster. Mapr-warden manages all services and httpfs is one of them. Warden  does not allow multiple httpfs to launch on multiple servers because by default in warden-httpfs.conf file the service for httpfs is listed as "services=httpfs:1"

 

cat /opt/mapr/conf/conf.d/warden.httpfs.conf

services=httpfs:1

service.displayname=Httpfs

service.command.start=/opt/mapr/httpfs/httpfs-1.0/sbin/httpfs.sh start

service.command.stop=/opt/mapr/httpfs/httpfs-1.0/sbin/httpfs.sh stop

service.command.type=BACKGROUND

service.command.monitorcommand=/opt/mapr/httpfs/httpfs-1.0/sbin/httpfs.sh status

service.port=14000

service.ui.port=14000

service.uri=about

service.logs.location=/opt/mapr/httpfs/httpfs-1.0/logs

service.process.type=JAVA

service.env="MAPR_MAPREDUCE_MODE=default"

Environment:

 

All MapR versions

 

 

Solution:

 

To execute httpfs on multiple nodes in your cluster, warden.httpfs.conf  file needs to modified. The line with "services=httpfs:1"  should be set to "service=httpfs:<node number>.  If you want to run httpfs on three nodes out of five node cluster then 

Set first node  in warden.httpfs.conf  to include "services=httpfs:1"

Set second node  in warden.httpfs.conf  to include "services=httpfs:2"

Set third node in warden.httpfs.conf  to include "services=httpfs:3"

 

Sample config for node3:

 

services=httpfs:3

service.displayname=Httpfs

service.command.start=/opt/mapr/httpfs/httpfs-1.0/sbin/httpfs.sh start

service.command.stop=/opt/mapr/httpfs/httpfs-1.0/sbin/httpfs.sh stop

service.command.type=BACKGROUND

service.command.monitorcommand=/opt/mapr/httpfs/httpfs-1.0/sbin/httpfs.sh status

service.port=14000

service.ui.port=14000

service.uri=about

service.logs.location=/opt/mapr/httpfs/httpfs-1.0/logs

service.process.type=JAVA

service.env="MAPR_MAPREDUCE_MODE=default"

 

Attachments

    Outcomes