AnsweredAssumed Answered

Suitability for image processing

Question asked by nullpainter on Oct 15, 2012
Latest reply on Oct 16, 2012 by Ted Dunning
Hi, and apologies for those who subscribe to the Hadoop users mailing list and may have seen a similar question.

I am investigating options for scaling an archive of around 100Tb of image data. These images are typically TIFF files of around 50-100Mb each and need to be made available online. As well as storing the master files, we will be wishing to perform image conversion to create thumbnails, PDFs etc. I'm attracted by the distributed, scalable and redundant nature of HDFS but don't currently have many MapReduce needs.

I have concluded that a native Hadoop solution won't fly - image conversion and MapReduce don't seem the best bedfellows, and I'd really prefer to use ImageMagick rather than roll my own.

I see that MapR provides NFS support, which sounds very encouraging for our needs. On the surface, this seems the best of both worlds. Does anyone see any issues in this approach? Store our master files in HDFS and get scalability etc., and use ImageMagick to do image conversion where required over NFS?

Many thanks,