snormand

Playing with local hadoop cluster

Discussion created by snormand on May 30, 2016

Over MapR sandbox i was looking for a simple solution out of cloud (meaning connectivity and little cost) to play (learning purpose) with a little Hadoop cluster.

 

A really nice one :

http://software.danielwatrous.com/install-and-configure-a-multi-node-hadoop-cluster-using-ansible/http://

https://github.com/dwatrous/hadoop-multi-server-ansible

 

It let in a chain scripted way :

  • Through Vagrant create, configure 3 VirtualBox VM (cpu, RAM, ports configuration, ...)
  • Through Ansible configure them as hadoop node (jdk install, ...)
  • Once done you have under virtual box  1 hadoop master node and 2 data nodes ready to run MapReduce stuff. So nice !!!!

 

To me, was also a way to discover Vagrant and Ansible that let integrates VM product (VirtualBox, VMWare, ...)

Here are two introduction to those technologies

http://fr.slideshare.net/Juliendubreuil/vagrant-concept

http://fr.slideshare.net/headmin/ansible-meetup-hamburg

 

Next way trying to adapt it to MapR ?

Outcomes