AnsweredAssumed Answered

Need to buy Drives: Discussion of Pros and Cons of Various Drives?

Question asked by mandoskippy on Oct 28, 2012
Latest reply on Oct 28, 2012 by mandoskippy
I need to purchase some harddrives for my cluster.  It's a small test/dev cluster that I like to play with, and budget is a huge concern.  Thus, in needing to purchase drives, I wanted to toss out some "classes" of drives I am seeing in the tech stores, and get some opinions on pros and cons of some of these with MapR. Should I aim for one type of drive vs. another etc.

I've listed some "thoughts" I've had in reading reviews etc and the putting them into the context of what I know about MapR.  I looked at a few things, including the WD lineup, but may apply to other brands. I'd love feedback from MapR engineers on this; given a clean slate, where you do you think performance would gain vs. reliability vs. cost etc.  Does MapR do things that make it better or worse than standard HDFS for certain types of drives?

Desktop vs. Server:  Desktop Drives tends to suffer from the "Raid dropping" syndrome where a command timeout happens and a Raid Array will drop a drive because it didn't wait long enough.  This seems to be one of the biggest differentiators between server and desktop drives.  But for something like MapR where you can set the timeout yourself, is the Raid worth it?

RPMs: How big of a different do RPMs really make? Does MapR take advantage of ATA streaming or anything like that?   

Desktop vs. Mobile drives: For servers that have limited space, do you need to spring for the expensive 2.5 inch server drives or are the 2.5 inch mobile drives acceptable because of of MapR can utilize them? Obviously there is a MTBF issue to think about, but that may be acceptable if performance and price is ok given the replicative nature of Hadoop/MaprFS in general.

Now some thoughts on reading about WDs different lines...

WD Green: Low Power Low Noise, Low Speed, Low Performance? I hear lots of issues in this Eco Line, I am guessing should be avoided for MapR. (5400 RPM I think). Forums state suffer from Disk Time out errors.  This can be addressed in mfs.conf, but maybe we should avoid?

WD Blue: Standard Desktop User: 5400 RPM I think A bit better then Green, but I am guessing the 5400 RPM is a red flag, may suffer from the same disk time out errors. 

WD Black: Supposedly for performance users, 7200 RPMs etc.  Notes say not good for RAID, but how would they hold up with MapR FS, especially since we can set the disk timeout.  These better than say Green or Blue?  Dunno

WD Red: This one intrigues me. These are supposedly designed for NAS devices and support "ATA Streaming"  Not sure what that is, but supposedly they deal with with raid (i.e. they don't time out well) maybe that would be good for MapR? Not sure.  Does MapR supposed the ATA streaming at the Drive level?  I think these spin at 5400rpm, but that may be deceiving based on some of the reviews I read. 

WD Raid Edition (RE): These supposed to be the High end drives (and you pay a premium for them) performance and Raid ready. I am not sure the performance here would be great for the extra price. Especially since you can configure the timeout per node.  Thoughts