Manual AWS Deployment

Document created by wochanda Employee on Aug 23, 2017Last modified by nsen on Oct 26, 2017
Version 15Show Document
  • View in full screen mode

Manually Deploying MapR in AWS

This document assumes you are familiar with the AWS console and can navigate and create resources there.


The high level tasks are:



Create AWS Key-pair

See on creating key-pairs



Create Policies and Roles

If there are restriction in creating roles and policies, you can add the AWS API credentials on the installer node. See section on “Setup the Installer”


There is cloud-formation template that can create these for you: Launch Stack


Otherwise, follow these steps:

  1. Create Policy

       The policy statement would be:  


            "Version": "2012-10-17",

            "Statement": [


                    "Effect": "Allow",

                    "Action": [
























                 "Resource": "*"





    2. Create Role with the policy created above.


Note: You as a user must have the IAM:PassRole permission





Create VPC, subnets, Internet Gateway, NAT Gateway, Security Groups

You can use an existing VPC if that satisfies the following requirements:

  • Instances can be created in a subnet that can talk to the Internet (to download MapR packages)
  • Instances can talk to each other within the VPC


If not use can use the following steps to create the VPC, subnets, Internet Gateway, NAT Gateway, Security Groups.


There is cloud-formation template that can create these for you: Launch Stack


Otherwise, follow these steps:

  1. Create VPC
    1. Make sure that "DNS Resolution" and "DNS Hostnames" are enabled for the VPC
  2. Create two subnets
    1. These will be configured such that one can be accessed from the Internet (public-subnet), another can talk to the Internet but cannot be accessed from outside (private-subnet)
  3. Create Internet Gateway
  4. Attach Internet Gateway to the VPC
  5. Create Elastic IP
  6. Create NAT Gateway - attach it to the public subnet
  7. Update default Route Table for VPC to allow Internet access via Internet Gateway
  8. Create a new Route
    1. Update the route to allow the private-subnet to talk to the Internet via NAT Gateway
    2. Associate private subnet to the new route
  9. Create Security groups
    1. SSH Access
      1. Ports: 22
    2. Web Access
      1. Ports: 9443, 8443, 8042, 8047, 8088, 8888, 18080, 19888

Create VPC

DNS Resolution and DNS Hostnames are enabled


Create two subnets in the VPC you created above (you need to do this step twice - once for each subnet). The subnet CIDR should be different and non-overlapping for the two subnets. So, if your VPC CIDR is, your public subnet CIDR could be and private subnet CIDR could be


Create Internet Gateway


Attach Internet Gateway to VPC


Create NAT Gateway - choose the public subnet



Update default route table for VPC to route all trafic to Internet Gateway



Create a new route table for private subnet



Update the route table to route all traffic to the NAT gateway


Associate route table with the private subnet


Create Security Groups

Add Inbound security rules

Create a Launch Configuration

From AWS console, create launch configuration:

  • Choose AMI

  • Choose machine type

  • Provide launch configuration name

    • Choose either the 2nd or 3rd option for IP Address Type under Advanced options

  • Add storage

    • Increase the root disk size to minimum of 128G

    • Add as many disks, along with their size, for data disks on the nodes

  • Select security groups

    • The default security group for the VPC that will allow the instances to talk to each other

    • The security group created to allow web access to the MapR services

  • Select the AWS Keypair to be used


You can go on to creating an auto scaling group with the launch configuration created from the final screen.


Create an Auto-scale Group

From AWS console, create auto scaling group:

  • Set auto-scale group properties

    • Give it a name

    • Select the VPC and the private-subnet

    • Choose 0 instances to start with (and let the MapR installer manage it)

  • Select to keep the group at initial size

  • After auto-scale group is created, edit the group to Suspend the following 4 services

    • Terminate, ReplaceUnhealthy, AZRebalance, HealthCheck


Suspend auto-scale management processes:


Create an EC2 instance for the installer

From AWS console, create an EC2 instance using an AMI with supported OS:

  1. Assign it to the VPC created and public-subnet. Assign a public IP address.

  2. Give role to be able to execute AWS commands without credentials

    1. Alternatively, create aws credentials files on the installer (see next section)

  3. Min disk size: 128G

  4. Assign the following security groups:

    1. Default security group to allow the node to talk to all nodes in the cluster

    2. The security group created to allow web access to the MapR services

    3. The security group created to allow SSH access to the nodes



Setup MapR installer

SSH to mapr installer and as root run:


Note: If the installer has not been setup with an IAM Role to allow it to execute AWS commands, follow the section "Create AWS credentials file" first.

Note: replace <clustername> with the name of you wish to give your cluster

  1. wget
  2. chmod +x
  3. ./ -y
  4. cd /opt/mapr/installer
  5. source build/installer/bin/activate
  6. cd data
  7. ssh-keygen -b 2048 -t rsa -f <clustername> -q -N '' -C "maprinstaller@maprcluster"
  8. aws ec2 import-key-pair --key-name <clustername> --region us-east-2 --public-key-material "`cat <clustername>.pub`"
  9. chmod 400 <clustername>
  10. touch config.yml
  11. chown mapr:mapr <clustername> <clustername>.pub config.yml
  12. vi config.yml

Copy and paste the content below. Update the values as per your environment.


 mapr_core_version: 5.2.2


 ssh_id: centos

 ssh_key_file: /opt/mapr/installer/data/<clustername>

 cluster_name: <clustername>

 mep_version: 3.0.1


   id: AWS


     aws_region: us-east-2

     auto_scaling_group: mapr-test-asg

     count: 3

     key_name: <clustername>

     disk_type: gp2

     disk_size: 100

     disk_count: 3

 hosts: []

  1. sudo -u mapr ../bin/mapr-installer-cli import -f -n --config -t config.yml
  2. passwd mapr


Next, launch the installer and complete the rest of the installation via the installer. The installer would be listening on:



Create AWS credentials file

If you did not create an AWS role to be used by the MapR installer, you must create an AWS credentials file with AWS access key and secret. Information on how to obtain access key and secret can be found at


  1. mkdir -p ~/home/mapr/.aws
  2. vi ~/home/mapr/.aws/credentials


aws_access_key_id = <your aws access_key>

aws_secret_access_key = <your aws secret key>