AnsweredAssumed Answered

Spark Cluster Benchmarking

Question asked by atandon on Oct 12, 2015
Latest reply on Mar 23, 2016 by kzielins
Hi All,

I wanted to execute HiBench 4.0 benchmarks on Spark 1.3.1/ Spark 1.4.1 installed on MapR 4.1 cluster.
As Hibench will fail or may be tweaking is required to get it going (not sure though), when it will run DFSIO as it would expect HDFS and in this case of MapR distribution, MapRFS would be there. 

Request you to please suggest if the above understanding is correct and advise which tool will be best suitable for Spark cluster bench-marking. The intent is to bench-mark the resource consumption metrics for various workloads (CPU, memory, disk, network Utilization etc).

Thanks in advance for your help.