AnsweredAssumed Answered

MapR is 50% slower than Hadoop in major TPC-H queries in a fat node cluster

Question asked by feeblefakie on Sep 13, 2012
Latest reply on Sep 28, 2012 by jason
Hi,

I am benchmarking MapR M3 and comparing with Hadoop.
I have run TPC-H queries though Hive and
found out that MapR is basically slower than Hadoop more than 50%.

Basic Hadoop configuration is the same and some MapR specific configurations are different.
(like mapred.reduce.slowstart.completed.maps, mapr recommends it as 0.95, so
I leave it as it is, but 0.05 for Hadoop.)

One of the unique things in the experiment is that our cluster is a little bit fatter than
normal Hadoop cluster setting.
We have (2U) 15 nodes and each node equipped with
16 physical cores (32 logical) and 64GB RAM and 26 disk drives.
(Each disk drive is 2.5 inch 10K rpm SAS drive)
Usual Hadoop cluster is tens of 1U node with 2-4 disk drives, maybe.

We create one logical volume for 26 drives with RAID5(hardware) and
divide it into 2 logical volumes by LVM, one for Hadoop and one for MapR.
Each logical volume gets 2000 MB/s and both are the same because I/O performance is bounded by RAID controller.

The cluster is composed of 1 namenode and 14 datanode for Hadoop (same for Mapr with different component name)
and 1 jobtracker and 14 tasktrackers.
Namenode and jobtracker are located one node, other 14 nodes each owns 1 datanode and 1 tasktracker.
I have run experiments with TPC-H dataset and queries.
Scale factor is 5600, which is almost 5.6TB of data, about 400GB of data for each datanode.
Hive queries we use is from the following site. <https://issues.apache.org/jira/browse/HIVE-600>

For example, for TPC-H Q1, which is a simple selectioin query for lineitem and aggregation and sorting afterwards,
mapr takes 1719 (s) and Hadoop takes 975 (s).
For Q3, which is simple 3-way join queries, MapR takes  6855 (s) and Hadoop takes 4330 (s).

I stored all the traces of each job, which I can send you in person if you look into this issue,
and as long as I can tell from the traces,
MapR reading I/O performance by mapper is slower than HDFS reading I/O performance.
MapR gets about 300MB/s and Hadoop gets 350-400 MB/s.
(Both system is bounded by CPU, that is why so far from ideal 2000 MB/s.)
Also, reduce phase seems take longer in Mapr than Hadoop,
which is I have no idea why.

Major configuration is listed as follows.
I can send both MapR and Hadoop mapred-site.xml configurations in person if you like. (and other configrations, too)

<pre>
mapred.tasktracker.map.tasks.maximum: 16
mapred.tasktracker.reduce.tasks.maximum: 16
mapred.map.child.java.opts: -Xmx1024m -Xms512m
mapred.reduce.child.java.opts:  -Xmx1024m -Xms512m
io.sort.mb: 256
io.sort.factor: 100
</pre>

The thing is we run both systems with almost the same configuration.
So the basic difference is the filesystem, MapRFS and HDFS.
MapRFS are anounced to be faster than HDFS about 3 times,
but my experiments show it is not even faster at all.

What is the problem ?
I can give you any information for finding out the cause.

Outcomes