bretlowery

Dr. Elephant for MapR

Blog Post created by bretlowery on Jan 5, 2017

Dr. Elephant (GitHub - linkedin/dr-elephant ) is a performance monitoring and tuning tool originally developed by LinkedIn for Apache Hadoop and Apache Spark. It automatically gathers metrics, runs analysis on them, and presents them in a simple green-yellow-red dashboard for easy consumption. It analyzes the Hadoop and Spark jobs using a set of pluggable, configurable, rule-based heuristics that provide insights on how a job performed, and then uses the results to make suggestions about how to tune the job to make it perform more efficiently, allowing Hadoop and Spark users to self-diagnose and treat common performance issues with Hadoop and Spark.

 

I've developed a modified version that works with the MapR distros of both Hadoop (tested on 2.7.0) and Spark (tested on 1.6.1). You can download it from GitHub here

Outcomes